|
@@ -1,139 +0,0 @@
|
|
|
-{
|
|
|
- "cells": [
|
|
|
- {
|
|
|
- "cell_type": "markdown",
|
|
|
- "metadata": {},
|
|
|
- "source": [
|
|
|
- "# Results"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "markdown",
|
|
|
- "metadata": {},
|
|
|
- "source": [
|
|
|
- "## Beaver_01\n",
|
|
|
- "1734 Lapse images, 695 Motion images.\n",
|
|
|
- "\n",
|
|
|
- "| Approach | Configuration | Best AUC | TNR @TPR $\\geq$ 0.9 | TNR @TPR $\\geq$ 0.95 | TNR @TPR $\\geq$ 0.99 | approx train time | approx eval time |\n",
|
|
|
- "| --- | --- | ---: | ---: | ---: | ---: | ---: | ---: |\n",
|
|
|
- "| 1a - Basic Frame Differencing | abs var | 0.7414 | 0.4865 | 0.4189 | 0.2432 | 0 | 1:00 min |\n",
|
|
|
- "| | $\\sigma=2$, sq var | 0.8986 | 0.7162 | 0.6081 | 0.5270 | 0 | 1:30 min |\n",
|
|
|
- "| | $\\sigma=4$, sq var | 0.9156 | 0.7973 | 0.6486 | 0.5676 | 0 | 1:30 min |\n",
|
|
|
- "| 1b - Histogram Comparison | p-mean | 0.6707 | | | | | |\n",
|
|
|
- "| 2 - Background Estimation | sq var | 0.7897 | 0.6622 | 0.5946 | 0.2703 | 0 | < 1 min |\n",
|
|
|
- "| | $\\sigma=2$, sq var | 0.8735 | 0.7973 | 0.7162 | 0.4865 | 0 | 1:00 min |\n",
|
|
|
- "| | $\\sigma=4$, sq var | 0.8776 | 0.7838 | 0.7027 | 0.4459 | 0 | 1:00 min |\n",
|
|
|
- "| 3 - BOW | $k=1024, kp=30$ | 0.7698 | 0.3929 | 0.3800 | 0.0757 | | |\n",
|
|
|
- "| | $k=2048, kp=30$ | 0.7741 | 0.4976 | 0.3382 | 0.0564 | | |\n",
|
|
|
- "| | $k=4096, kp=30$ | 0.7837 | 0.5797 | 0.2866 | 0.0451 | 4:00 h | 2:10 min |\n",
|
|
|
- "| | $k=2048, kp=40$ | 0.7611 | 0.3317 | 0.1610 | 0.1320 | 1:10 h | 1:30 min |\n",
|
|
|
- "| 3 - BOW +motion | $k=1024, kp=30$, +motion | 0.7056 | 0.2432 | 0.2222 | 0.0821 | | |\n",
|
|
|
- "| | $k=2048, kp=30$, +motion | 0.7390 | 0.3172 | 0.3092 | 0.0612 | | |\n",
|
|
|
- "| | $k=4096, kp=30$, +motion | 0.7542 | 0.3768 | 0.2963 | 0.0515 | 5:30 h | 2:10 min |\n",
|
|
|
- "| | $k=2048, kp=40$, +motion | 0.7388 | 0.1852 | 0.1820 | 0.0467 | 2:40 h | 2:20 min |\n",
|
|
|
- "| 3 - BOW random | $k=2048, kp=30$, random | 0.8002 | 0.6296 | 0.5588 | 0.0258 | 8 min | 1:30 min |\n",
|
|
|
- "| | $k=4096, kp=30$, random | 0.8022 | 0.6602 | 0.2738 | 0.1353 | 8 min | 2:00 min |\n",
|
|
|
- "| | $k=8192, kp=30$, random | 0.7973 | 0.6151 | 0.3913 | 0.2061 | 8 min | 3:30 min |\n",
|
|
|
- "| | $k=2048, kp=20$, random | 0.7943 | 0.5862 | 0.5539 | 0.2399 | 15 min | 3:00 min |\n",
|
|
|
- "| | $k=4096, kp=20$, random | 0.8088 | 0.6329 | 0.5459 | 0.2432 | 15 min | 4:00 min |\n",
|
|
|
- "| 4 - Autoencoder | Deep2 | 0.8678 | 0.5946 | 0.5000 | 0.0405 | 7:00 min | < 0:30 min |\n",
|
|
|
- "| | Deep2 +Noise(.015) | 0.8930 | 0.7432 | 0.4459 | 0.0000 | 7:00 min | < 0:30 min |\n",
|
|
|
- "| | Deep2 +Noise(.015) +Sparse(1e-4) | 0.8445 | 0.2703 | 0.1081 | 0.0541 | 7:00 min | 1:00 min |\n",
|
|
|
- "| | Deep3 | 0.8663 | 0.7703 | 0.5946 | 0.1081 | 7:00 min | < 0:30 min |\n",
|
|
|
- "| | Deep3 +Noise(.015) | 0.8542 | 0.8486 | 0.4324 | 0.0946 | 7:00 min | < 0:30 min |\n",
|
|
|
- "| | Deep3 +Noise(.015) +Sparse(1e-4) | 0.7608 | 0.0811 | 0.0676 | 0.0405 | 7:00 min | 1:00 min |\n",
|
|
|
- "| | Deep +Noise +Sparse Loss (lr=1e-4, 200 epochs, reg=0.1) | 0.7479 | 0.2086 | 0.1138 | 0.0008 | 8:30 min | 1:30 min |\n",
|
|
|
- "| | Deep +Noise +Sparse KDE | 0.9209 | 0.8514 | 0.6892 | 0.1216 | 6 min | < 0:30 min |"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "markdown",
|
|
|
- "metadata": {},
|
|
|
- "source": [
|
|
|
- "\n",
|
|
|
- "## Marten_01\n",
|
|
|
- "2462 Lapse images (with many doubles), 3105 Motion images.\n",
|
|
|
- "\n",
|
|
|
- "| Approach | Configuration | Best AUC | TNR @TPR $\\geq$ 0.9 | TNR @TPR $\\geq$ 0.95 | TNR @TPR $\\geq$ 0.99 | approx train time | approx eval time |\n",
|
|
|
- "| --- | --- | ---: | ---: | ---: | ---: | ---: | ---: |\n",
|
|
|
- "| 1a - Basic Frame Differencing | sq var | 0.6363 | 0.0244 | 0.0215 | 0.0160 | 0 | 5 min |\n",
|
|
|
- "| | $\\sigma=2$, sq var | 0.8004 | 0.3236 | 0.1606 | 0.0434 | 0 | 6 min |\n",
|
|
|
- "| | $\\sigma=4$, sq var | 0.8030 | 0.3536 | 0.2031 | 0.0801 | 0 | 6 min |\n",
|
|
|
- "| 2 - Background Estimation | sqmean | 0.5056 | 0.0295 | 0.0219 | 0.0169 | 0 | 2:30 min |\n",
|
|
|
- "| | $\\sigma=4$, sqvar | 0.7403 | 0.2090 | 0.1150 | 0.0253 | 0 | 4:00 min |\n",
|
|
|
- "| 3 - BOW | $k = 4096, kp = 30$, random | 0.6619 | 0.4973 | 0.2186 | 0.1298 | 12 min | 13 min |\n",
|
|
|
- "| 4 - Autoencoder | Deep3 | 0.6912 | 0.1471 | 0.0721 | 0.0013 | 10:00 min | 1:00 min |\n",
|
|
|
- "| | Deep3 +Noise(.015) | 0.7582 | 0.3767 | 0.0206 | 0.0206 | 10:00 min | 1:00 min |\n",
|
|
|
- "| | Deep3 +Noise(.015) +Sparse(1e-4) | 0.6120 | 0.1753 | 0.1037 | 0.0013 | 10:00 min | 1:00 min |\n",
|
|
|
- "| | Deep2 | 0.7207 | 0.1745 | 0.0464 | 0.0198 | 20:00 min | < 0:30 min |\n",
|
|
|
- "| | Deep2 +Noise(.015) | 0.7200 | 0.1884 | 0.0881 | 0.0088 | 20:00 min | < 0:30 min |\n",
|
|
|
- "| | Deep2 +Noise(.015) +Sparse(1e-4) | 0.6553 | 0.0194 | 0.0114 | 0.0038 | 20:00 min | < 0:30 min |\n",
|
|
|
- "| | Deep +Noise +Sparse Loss (lr=1e-4, 200 epochs, reg=0.1) | 0.7479 | 0.2086 | 0.1138 | 0.0008 | 8:30 min | 1:30 min |"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "markdown",
|
|
|
- "metadata": {},
|
|
|
- "source": [
|
|
|
- "## GFox_03\n",
|
|
|
- "**Generated set:** The Lapse set was randomly selected from the labeled Motion set. This was necessary due to a lack of Lapse images (only one per day instead of per hour).\n",
|
|
|
- "\n",
|
|
|
- "**Lapse generation procedure:** Take a random set of consecutively taken Motion images. If all images are annotated as images, add the whole set to Lapse and remove it from Motion. Repeat until at least 1200 Lapse images were selected.\n",
|
|
|
- "\n",
|
|
|
- "| Approach | Configuration | Best AUC | TNR @TPR $\\geq$ 0.9 | TNR @TPR $\\geq$ 0.95 | TNR @TPR $\\geq$ 0.99 | approx train time | approx eval time |\n",
|
|
|
- "| --- | --- | ---: | ---: | ---: | ---: | ---: | ---: |\n",
|
|
|
- "| 1a - Basic Frame Differencing | n.a. | | | | | | |\n",
|
|
|
- "| 2 - Background Estimation | sqmean | 0.4745 | 0.0903 | 0.0493 | 0.0189 | 0 | 4:00 min |\n",
|
|
|
- "| | $\\sigma=4$, sqmean | 0.4793 | 0.0650 | 0.0434 | 0.0096 | 0 | 4:30 min |\n",
|
|
|
- "| 3 - BOW | $k = 4096, kp = 30$, random | 0.9743 | 0.9715 | 0.8333 | 0.4837 | 3:00 min | 8:00 min |\n",
|
|
|
- "| 4 - Autoencoder | Deep2 | 0.9713 | 0.9510 | 0.9049 | 0.6701 | 6:00 min | < 0:30 min |\n",
|
|
|
- "| | Deep2 (Loss) | 0.9861 | 0.9892 | 0.9470 | 0.6481 | 6:00 min | < 0:30 min |\n",
|
|
|
- "| | Deep2 +Noise(.015) | 0.9684 | 0.9274 | 0.8347 | 0.6838 | 6:00 min | < 0:30 min |\n",
|
|
|
- "| | Deep2 +Noise(.015) (Loss) | 0.9858 | 0.9852 | 0.9551 | 0.6344 | 6:00 min | < 0:30 min |\n",
|
|
|
- "| | Deep2 +Noise(.015) +Sparse(1e-4) | 0.9749 | 0.9631 | 0.8668 | 0.6320 | 6:00 min | 1:00 min |\n",
|
|
|
- "| | Deep2 +Noise(.015) +Sparse(1e-4) (Loss) | 0.9847 | 0.9916 | 0.9474 | 0.6140 | 6:00 min | 1:00 min |\n",
|
|
|
- "| | Deep3 | 0.9012 | 0.8455 | 0.7472 | 0.6746 | 6:00 min | < 0:30 min |\n",
|
|
|
- "| | Deep3 (Loss) | 0.9887 | 0.9932 | 0.9835 | 0.7913 | 6:00 min | < 0:30 min |\n",
|
|
|
- "| | Deep3 +Noise(.015) | 0.9042 | 0.8387 | 0.7476 | 0.6465 | 6:00 min | < 0:30 min |\n",
|
|
|
- "| | Deep3 +Noise(.015) (Loss) | 0.9889 | 0.9920 | 0.9715 | 0.8363 | 6:00 min | < 0:30 min |\n",
|
|
|
- "| | Deep3 +Noise(.015) +Sparse(1e-4) | 0.9356 | 0.8732 | 0.8351 | 0.1597 | 6:00 min | 1:00 min |\n",
|
|
|
- "| | Deep3 +Noise(.015) +Sparse(1e-4) (Loss) | 0.9829 | 0.9872 | 0.9117 | 0.6726 | 6:00 min | 1:00 min |\n",
|
|
|
- "| | Deep +Noise +Sparse KDE (lr=1e-4) | 0.9684 | 0.9579 | 0.9041 | 0.4964 | 5:00 min | < 0:30 min |"
|
|
|
- ]
|
|
|
- },
|
|
|
- {
|
|
|
- "cell_type": "code",
|
|
|
- "execution_count": null,
|
|
|
- "metadata": {},
|
|
|
- "outputs": [],
|
|
|
- "source": []
|
|
|
- }
|
|
|
- ],
|
|
|
- "metadata": {
|
|
|
- "kernelspec": {
|
|
|
- "display_name": "Python 3.6.9 64-bit",
|
|
|
- "language": "python",
|
|
|
- "name": "python3"
|
|
|
- },
|
|
|
- "language_info": {
|
|
|
- "codemirror_mode": {
|
|
|
- "name": "ipython",
|
|
|
- "version": 3
|
|
|
- },
|
|
|
- "file_extension": ".py",
|
|
|
- "mimetype": "text/x-python",
|
|
|
- "name": "python",
|
|
|
- "nbconvert_exporter": "python",
|
|
|
- "pygments_lexer": "ipython3",
|
|
|
- "version": "3.6.9"
|
|
|
- },
|
|
|
- "orig_nbformat": 4,
|
|
|
- "vscode": {
|
|
|
- "interpreter": {
|
|
|
- "hash": "31f2aee4e71d21fbe5cf8b01ff0e069b9275f58929596ceb00d14d90e3e16cd6"
|
|
|
- }
|
|
|
- }
|
|
|
- },
|
|
|
- "nbformat": 4,
|
|
|
- "nbformat_minor": 2
|
|
|
-}
|