Papers for Video Anomaly Detection

Papers for Video Anomaly Detection released code collections.

For any addition or bug please open an issue, pull requests or e-mail me by fjchange@hotmail.com

Recent Updated

• AAAI 2022
• CVPR 2022

Datasets

0. UMN Download link
1. UCSD Download link
2. Subway Entrance/Exit Download link
3. CUHK Avenue Download link
   • HD-Avenue Skeleton-based
4. ShanghaiTech Download link
   • HD-ShanghaiTech Skeleton-based
5. UCF-Crime (Weakly Supervised)
   • UCFCrime2Local (a subset of UCF-Crime but with spatial annotations.) Download_link, Ano-Locality
   • Spatial-Temporal Annotations Download_link Background-Bias
6. Traffic-Train
7. Belleview
8. Street Scene (WACV 2020) Street Scenes, Download link
9. IITB-Corridor (WACV 2020) Rodrigurs.etl
10. XD-Violence (ECCV 2020) XD-ViolenceDownload link
11. ADOC (ACCV 2020) ADOCDownload_link
12. UBnormal (CVPR 2022) [UBnormal] Project Link Open-Set

The Datasets below are about Traffic Accidents Anticipating in Dashcam videos or Surveillance videos

1. CADP (CarCrash Accidents Detection and Prediction)

2. DAD paper, Download link

3. A3D paper, Download link

4. DADA Download link

5. DoTA Download_link

6. Iowa DOT Download_link

7. Driver_Anomaly Project_link

---

Unsupervised

2016

1. [Conv-AE] Learning Temporal Regularity in Video Sequences, CVPR 16. Code

2017

1. [Hinami.etl] Joint Detection and Recounting of Abnormal Events by Learning Deep Generic Knowledge, ICCV 2017. (Explainable VAD)
2. [Stacked-RNN] A revisit of sparse coding-based anomaly detection in stacked rnn framework, ICCV 2017. code
3. [ConvLSTM-AE] Remembering history with convolutional LSTM for anomaly detection, ICME 2017.Code
4. [Conv3D-AE] Spatio-Temporal AutoEncoder for Video Anomaly Detection,ACM MM 17.
5. [Unmasking] Unmasking the abnormal events in video, ICCV 17.
6. [DeepAppearance] Deep appearance features for abnormal behavior detection in video

2018

1. [FramePred] Future Frame Prediction for Anomaly Detection — A New Baseline, CVPR 2018. code
2. [ALOOC] Adversarially Learned One-Class Classifier for Novelty Detection, CVPR 2018. code
3. Detecting Abnormality Without Knowing Normality: A Two-stage Approach for Unsupervised Video Abnormal Event Detection, ACM MM 18.

2019

1. [Mem-AE] Memorizing Normality to Detect Anomaly: Memory-augmented Deep Autoencoder for Unsupervised Anomaly Detection, ICCV 2019.code
2. [Skeleton-based] Learning Regularity in Skeleton Trajectories for Anomaly Detection in Videos, CVPR 2019.code
3. [Object-Centric] Object-Centric Auto-Encoders and Dummy Anomalies for Abnormal Event Detection, CVPR 2019.
4. [Appearance-Motion Correspondence] Anomaly Detection in Video Sequence with Appearance-Motion Correspondence, ICCV 2019.code
5. [AnoPCN]AnoPCN: Video Anomaly Detection via Deep Predictive Coding Network, ACM MM 2019.

2020

1. [Street-Scene] Street Scene: A new dataset and evaluation protocol for video anomaly detection, WACV 2020.
2. [Rodrigurs.etl]) Multi-timescale Trajectory Prediction for Abnormal Human Activity Detection, WACV 2020.
3. [GEPC] Graph Embedded Pose Clustering for Anomaly Detection, CVPR 2020.code
4. [Self-trained] Self-trained Deep Ordinal Regression for End-to-End Video Anomaly Detection, CVPR 2020.
5. [MNAD] Learning Memory-guided Normality for Anomaly Detection, CVPR 2020. code
6. [Continual-AD]] Continual Learning for Anomaly Detection in Surveillance Videos,CVPR 2020 Worksop.
7. [OGNet] Old is Gold: Redefining the Adversarially Learned One-Class Classifier Training Paradigm, CVPR 2020. code
8. [Any-Shot] Any-Shot Sequential Anomaly Detection in Surveillance Videos,CVPR 2020 workshop.
9. [Few-Shot]Few-Shot Scene-Adaptive Anomaly DetectionECCV 2020 Spotlight code
10. [CDAE]Clustering-driven Deep Autoencoder for Video Anomaly DetectionECCV 2020
11. [VEC]Cloze Test Helps: Effective Video Anomaly Detection via Learning to Complete Video EventsACM MM 2020 Oral code
12. [ADOC][A Day on Campus – An Anomaly Detection Dataset for Events in a Single Camera] ACCV 2020
13. [CAC]Cluster Attention Contrast for Video Anomaly Detection ACM MM 2020
14. [STC-Graph]Scene-Aware Context Reasoning for Unsupervised Abnormal Event Detection in Videos ACM MM 2020

2021

1. [AMCM]Appearance-Motion Memory Consistency Network for Video Anomaly Detection AAAI 2021
2. [SSMT,Self-Supervised-Multi-Task]Anomaly Detection in Video via Self-Supervised and Multi-Task Learning CVPR 2021
3. [HF2-VAD]A Hybrid Video Anomaly Detection Framework via Memory-Augmented Flow Reconstruction and Flow-Guided Frame PredictionICCV 2021 Oral
4. [ROADMAP]Robust Unsupervised Video Anomaly Detection by Multipath Frame PredictionTNNLS 2021
5. [AEP]Abnormal Event Detection and Localization via Adversarial Event Prediction TNNLS 2021

2022

1. [Casual]A Causal Inference Look At Unsupervised Video Anomaly DetectionAAAI 2022
2. [BDPN]Comprehensive Regularization in a Bi-directional Predictive Network for Video Anomaly DetectionAAAI 2022
3. [GCL]Generative Cooperative Learning for Unsupervised Video Anomaly DetectionCVPR 2022

Weakly-Supervised

2018

1. [Sultani.etl] Real-world Anomaly Detection in Surveillance Videos, CVPR 2018 code

2019

1. [GCN-Anomaly] Graph Convolutional Label Noise Cleaner:Train a Plug-and-play Action Classifier for Anomaly Detection, CVPR 2019, code
2. [MLEP] Margin Learning Embedded Prediction for Video Anomaly Detection with A Few Anomalies, IJCAI 2019code.
3. [IBL] Temporal Convolutional Network with Complementary Inner Bag Loss For Weakly Supervised Anomaly Detection. ICIP 19.
4. [Motion-Aware] Motion-Aware Feature for Improved Video Anomaly Detection. BMVC 19.

2020

1. [Siamese] Learning a distance function with a Siamese network to localize anomalies in videos, WACV 2020.
2. [AR-Net] Weakly Supervised Video Anomaly Detection via Center-Guided Discrimative Learning, ICME 2020.code
3. [‘XD-Violence’] Not only Look, but also Listen: Learning Multimodal Violence Detection under Weak Supervision ECCV 2020
4. [CLAWS] CLAWS: Clustering Assisted Weakly Supervised Learning with Normalcy Suppression for Anomalous Event Detection ECCV 2020

2021

1. [MIST] MIST: Multiple Instance Self-Training Framework for Video Anomaly Detection CVPR 2021 Project Page
2. [RTFM] Weakly-supervised Video Anomaly Detection with Contrastive Learning of Long and Short-range Temporal Features ICCV 2021Code
3. [STAD]Weakly-Supervised Spatio-Temporal Anomaly Detection in Surveillance Video IJCAI 2021
4. [WSAL]Localizing Anomalies From Weakly-Labeled VideosTIP 2021 Code
5. [CRFD]Learning Causal Temporal Relation and Feature Discrimination for Anomaly DetectionTIP 2021

2022

1. [MSL]Self-Training Multi-Sequence Learning with Transformer for Weakly Supervised Video Anomaly DetectionAAAI 2022

Supervised

2019

1. [Background-Bias]Exploring Background-bias for Anomaly Detection in Surveillance Videos, ACM MM 19.
2. [Ano-Locality]Anomaly locality in video suveillance.

Others

2020

1. [Few-Shot]Few-Shot Scene-Adaptive Anomaly Detection ECCV 2020code

---

Reviews / Surveys

1. An Overview of Deep Learning Based Methods for Unsupervised and Semi-Supervised Anomaly Detection in Videos, J. Image, 2018.page
2. DEEP LEARNING FOR ANOMALY DETECTION: A SURVEY, paper
3. Video Anomaly Detection for Smart Surveillance paper
4. A survey of single-scene video anomaly detection, TPAMI 2020 paper.

Books

1. Outlier Analysis. Charu C. Aggarwal

Specific Scene

---

Generally, anomaly detection in recent research is based on datasets from pedestrians (likes UCSD, Avenue, ShanghaiTech, etc.) or UCF-Crime (real-world anomaly). However, some focus on the specific scene is as follows.

Traffic

CVPR workshop, AI City Challenge series.

First-Person Traffic

​ Unsupervised Traffic Accident Detection in First-Person Videos, IROS 2019.

Driving

​ When, Where, and What? A New Dataset for Anomaly Detection in Driving Videos. GitHub

Old-man Fall Down

Fighting/Violence

1. Localization Guided Fight Action Detection in Surveillance Videos. ICME 2019.

Social/ Group Anomaly

1. Social-BiGAT: Multimodal Trajectory Forecasting using Bicycle-GAN and Graph Attention Networks, Neurips 2019.

Related Topics:

1. Video Representation (Unsupervised Video Representation, reconstruction, prediction etc.)
2. Object Detection
3. Pedestrian Detection
4. Skeleton Detection
5. Graph Neural Networks
6. GAN
7. Action Recognition / Temporal Action Localization
8. Metric Learning
9. Label Noise Learning
10. Cross-Modal/ Multi-Modal
11. Dictionary Learning
12. One-Class Classification / Novelty Detection / Out-of-Distribution Detection
13. Action Recognition.
    • Human in Events: A Large-Scale Benchmark for Human-centric Video Analysis in Complex Events. ACM MM 2020 workshop.

Performance Evaluation Methods

1. AUC
2. PR-AUC
3. Score Gap
4. False Alarm Rate on Normal with 0.5 as threshold (Weakly supervised, proposed in CVPR 18)

As discussed in Issue #12, the reported results below will be Micro-AUC”, if the paper provide Macro-AUC”, which will be tagged with *.

Performance Comparison on UCF-Crime

Model	Reported on Conference/Journal	Supervised	Feature	Encoder-based	32 Segments	AUC (%)	FAR@0.5 on Normal (%)
Sultani.etl	CVPR 18	Weakly	C3D RGB	X	√	75.41	1.9
IBL	ICIP 19	Weakly	C3D RGB	X	√	78.66	–
Motion-Aware	BMVC 19	Weakly	PWC Flow	X	√	79.0	–
GCN-Anomaly	CVPR 19	Weakly	TSN RGB	√	X	82.12	0.1
ST-Graph	ACM MM 20	Un	–	√	X	72.7
Background-Bias	ACM MM 19	Fully	NLN RGB	√	X	82.0	–
CLAWS	ECCV 20	Weakly	C3D RGB	√	X	83.03	–
MIST	CVPR 21	Weakly	I3D RGB	√	X	82.30	0.13
RTFM	ICCV 21	Weakly	I3D RGB	X	√	84.03	–
WSAL	TIP 21	Weakly	I3D RGB	X	√	85.38	–
CRFD	TIP 21	Weakly	I3D RGB	X	√	84.89	–
MSL	AAAI 22	Weakly	C3D RGB	√	X	82.85	–
MSL	AAAI 22	Weakly	I3D RGB	√	X	85.30	–
MSL	AAAI 22	Weakly	VideoSwin-RGB	√	X	85.62	–
GCL	CVPR 22	Weakly	ResNext	√	X	79.84	–
GCL	CVPR 22	Un	ResNext	√	X	71.04	–

Performance Comparison on ShanghaiTech

Model	Reported on Conference/Journal	Supervision	Feature	Encoder-based	AUC(%)	FAR@0.5 (%)
Conv-AE	CVPR 16	Un	–	√	60.85	–
stacked-RNN	ICCV 17	Un	–	√	68.0	–
FramePred	CVPR 18	Un	–	√	72.8	–
FramePred*	IJCAI 19	Un	–	√	73.4	–
Mem-AE	ICCV 19	Un	–	√	71.2	–
MNAD	CVPR 20	Un	–	√	70.5	–
VEC	ACM MM 20	Un	–	√	74.8	–
ST-Graph	ACM MM 20	Un	–	√	74.7	–
CAC	ACM MM 20	Un	–	√	79.3
AMMC	AAAI 21	Un	–	√	73.7	–
SSMT	CVPR 21	Un	–	√	82.4	–
HF2-VAD	ICCV 21	Un	–	√	76.2	–
ROADMAP	TNNLS 21	Un	–	√	76.6	–
BDPN	AAAI 22	Un	–	√	78.1	–
MLEP	IJCAI 19	10% test vids with Video Anno	–	√	75.6	–
MLEP	IJCAI 19	10% test vids with Frame Anno	–	√	76.8	–
Sultani.etl	ICME 2020	Weakly (Re-Organized Dataset)	C3D-RGB	X	86.3	0.15
IBL	ICME 2020	Weakly (Re-Organized Dataset)	I3D-RGB	X	82.5	0.10
GCN-Anomaly	CVPR 19	Weakly (Re-Organized Dataset)	C3D-RGB	√	76.44	–
GCN-Anomaly	CVPR 19	Weakly (Re-Organized Dataset)	TSN-Flow	√	84.13	–
GCN-Anomaly	CVPR 19	Weakly (Re-Organized Dataset)	TSN-RGB	√	84.44	–
AR-Net	ICME 20	Weakly (Re-Organized Dataset)	I3D-RGB & I3D Flow	X	91.24	0.10
CLAWS	ECCV 20	Weakly (Re-Organized Dataset)	C3D-RGB	√	89.67
MIST	CVPR 21	Weakly (Re-Organized Dataset)	I3D-RGB	√	94.83	0.05
RTFM	ICCV 21	Weakly (Re-Organized Dataset)	I3D-RGB	X	97.21	–
CRFD	TIP 21	Weakly (Re-Organized Dataset)	I3D-RGB	X	97.48	–
MSL	AAAI 22	Weakly (Re-Organized Dataset)	C3D-RGB	X	94.81	–
MSL	AAAI 22	Weakly (Re-Organized Dataset)	I3D-RGB	X	96.08	–
MSL	AAAI 22	Weakly (Re-Organized Dataset)	VideoSwin-RGB	X	97.32	–
GCL	CVPR 22	Weakly (Re-Organized Dataset)	ResNext	X	86.21	–
GCL	CVPR 22	Un	ResNext	X	78.93	–

Performance Comparison on Avenue

Model	Reported on Conference/Journal	Supervision	Feature	End2End	AUC(%)
Conv-AE	CVPR 16	Un	–	√	70.2
Conv-AE*	CVPR 18	Un	–	√	80.0
ConvLSTM-AE	ICME 17	Un	–	√	77.0
DeepAppearance	ICAIP 17	Un	–	√	84.6
Unmasking	ICCV 17	Un	3D gradients+VGG conv5	X	80.6
stacked-RNN	ICCV 17	Un	–	√	81.7
FramePred	CVPR 18	Un	–	√	85.1
Mem-AE	ICCV 19	Un	–	√	83.3
Appearance-Motion Correspondence	ICCV 19	Un	–	√	86.9
FramePred*	IJCAI 19	Un	–	√	89.2
MNAD	CVPR 20	Un	–	√	88.5
VEC	ACM MM 20	Un	–	√	90.2
ST-Graph	ACM MM 20	Un	–	√	89.6
CAC	ACM MM 20	Un	–	√	87.0
AMMC	AAAI 21	Un	–	√	86.6
SSMT	CVPR 21	Un	–	√	91.5
HF2-VAD	ICCV 21	Un	–	√	91.1
ROADMAP	TNNLS 21	Un	–	√	88.3
AEP	TNNLS 21	Un	–	√	90.2
Causal	AAAI 22	Un	I3D-RGB	X	90.3
BDPN	AAAI 22	Un	–	√	90.3
MLEP	IJCAI 19	10% test vids with Video Anno	–	√	91.3
MLEP	IJCAI 19	10% test vids with Frame Anno	–	√	92.8

Performance Comparison on XD-Violence

Model	Reported on Conference/Journal	Supervision	Feature	Encoder-based	32 Segments	AP(%)
Sultani et al.	ECCV 2020 (reported by Wu)	Weakly	I3D-RGB	X	√	73.20
Wu et al.	ECCV 2020	Weakly	C3D-RGB	X	X	67.19
Wu et al.	ECCV 2020	Weakly	I3D-RGB+Audio	X	X	78.64
RTFM	ICCV 2021	Weakly	I3D-RGB	X	√	77.81
CRFD	TIP 2021	Weakly	I3D-RGB	X	√	75.90
MSL	AAAI 2022	Weakly	C3D-RGB	X	X	75.53
MSL	AAAI 2022	Weakly	I3D-RGB	X	X	78.28
MSL	AAAI 2022	Weakly	VideoSwin-RGB	X	X	78.59