ANOMALY 2 BENCHMARK MANUAL
ANOMALY 2 BENCHMARK FULL

The algorithm must run in an unsupervised, automated fashion-i.e., without data labels or manual parameter tweaking.The algorithm must learn continuously without a requirement to store the entirestream.

Detections must be made online i.e., the algorithm must identify the current state as normal or anomalous before seeing the future.

In NAB we define the ideal characteristics of a real-time anomaly detection algorithm as follows: A real-world anomaly detector has to process streaming data, learn continuously, and make accurate detections as soon as possible. Looking at the entire plot, these anomalies are easily recognized after-the-fact but identifying anomalies long after they occur isn’t acceptable. For details on these and other NAB components, please explore the NAB repository. Can you guess if the metric is blood glucose level for a diabetes patient, stock volume for a Fortune 500 company, or something else? It’s difficult to tell, yet it’s apparent the plot shows data anomalies, or deviations from the normal pattern.įigure 1: An example data file from the NAB corpus data from monitoring the temperature of an internal component of a large, expensive, industrial machine. The plot shows scalar values over time for an unknown metric. Take a look at Figure 1 below, an example of time-series data from a sensor. The system observes each data record in sequential order as they arrive and any processing or learning must be done in an online fashion.

ANOMALY 2 BENCHMARK FULL

In contrast to batch processing, the full dataset is not available. These applications involve analyzing a continuous sequence of data occurring in real time. Streaming applications impose some special constraints and challenges for machine learning models. NAB allowed us to test and improve our own algorithm, and also compare against other published algorithms. We created the open source Numenta Anomaly Benchmark (NAB) to fill this hole. There are dozens of anomaly detection algorithms in the literature but it is almost impossible to evaluate them for streaming because existing benchmarks focus on non-streaming batch data. Finding anomalies or unusual behavior in this data can be extremely valuable, but doing it reliably is quite difficult. With sensors invading our everyday lives, we are seeing an exponential increase in the availability of streaming, time-series data.

Anomaly 2 benchmark