Does This Look Weird to You? An Introduction to Anomaly Detection
ABSTRACT
Discovering anomalies is a critical part of systems monitoring, financial auditing, and more. In this session, we will gain an understanding of what constitutes an anomaly, starting with a general concept of anomalies and then moving into technical definitions. We will review techniques for detecting anomalies for both time series and non-series data and implement a simple anomaly detector in .NET. Finally, we will compare these results against Microsoft's anomaly detector in Azure Cognitive Services.
ADDITIONAL MEDIA
I presented a 30-minute version of this talk for Carolina Code Conference.
SLIDES
Click here to access the slides for this presentation.
The slides are licensed under Creative Commons Attribution-ShareAlike.
DEMO CODE
Click here to access demo code for this presentation.
The source code is licensed under the terms offered by the GPL.
LINKS & FURTHER INFO
Shorter Version
A shorter version of this talk, suitable for 30-minute presentations, is available here.
Demos
- Microsoft has a demo of the Azure Cognitive Services Anomaly Detector API. Note that you do need an endpoint for this.
Links
- This post from Codesuji provided the basis for my Basic Anomaly Detector project. I made a few changes, particularly in the visualization (such as using Plotly instead of Google Charts), but this post provides the foundation.
- Microsoft has a quickstart on using their Anomaly Detection API in Azure Cognitive Services. This example is in C# and I converted it to F# for the demo.
- Microsoft has also provided best practices when using the Anomaly Detector API.
Packages
Here are links to the packages I reference during the talk.
- Anomalize (R)
- Time-series Anomaly Detection with Prophet by Facebook (R)
- AnomalyDetection by Twitter (R)
- anomaly (R)
- Anomaly Detection Toolkit (Python)
- PyOD (Python)
- ML.NET (.NET)
- Azure Cognitive Services Anomaly Detector (Multi)