Talk abstract: Anomaly Detection: A breakdown of Twitter’s Seasonal Hybrid ESD

In a world of deep learning statistical techniques are out of fashion but can still be very effective tools. Twitter’s open source anomaly detection project uses a statistical technique call Seasonal Hybrid ESD. This talk works through the various steps in the algorithm from data preparation and time series decomposition through to finding potentially multiple anomalies. The technique could be used to spot deviations from behavioural patterns with the benefit that it is easy to see why an anomaly is unusual.

Bio: Peter has been researching and solving leading-edge distributed computational problems for nearly 20 years. This began with intelligent agent systems; he tracked high-performance computing and their developments in both Grid and Cloud. More recently Peter has been closely following and working with Big Data, MapReduce, NoSQL and realtime streaming analysis. Peter is a Data Scientist, Trainer and Researcher who enjoys problems of scale and complexity. He combines the skill-sets of Data Engineer and Analyst and is as happy building fast real-time Kafka / Spark data pipelines as he is doing time series decomposition and building customer profiles. By fast, Peter has worked at 1 million events per second (80B events per day) with a total data warehouse size of 15PB. In his spare time Peter is a keen but talentless mountain biker, he tends to fall off a lot.