With the time-series of information technology operating analytics blog series, I wanted to introduce you to time-series and its use in information technology operating analytics. This series of blogs describe my work on my postgraduate thesis using statistical and machine learning techniques to apply anomaly detection into Information Technology Operating Analytics (ITOA) time-series data, so that the outliers of the computer system can be investigated and visualised.
Background
A time-series is a terminology that derives from statistics (1). It describes a series of data points indexed in time order. There are wide ranges of applications of time-series analysis in real life, such as; signal processing, pattern recognition, econometrics ,finance, weather forecasting, earthquake prediction, control engineering, astronomy, etc.
Time-series data contains information of different parameters on the time axis. Statistical analysis and machine learning techniques can extract this information and make it possible to alert incidents such as anomalies and predict future behaviour based on the learning and hypothesis of investigated data.
For example, in weather forecasting, the annual temperature data of London can be regarded as a whole body. The temperature data in one year will have periodic characteristics with seasonal changes, the possible annual rise caused by global warming and slight temperature fluctuations. If the researchers take a previous years’ data as the training dataset, it will be possible to predict the future temperature and alert to potential extreme climates by hypothesis and machine learning. Such a smart method can be integrated into the current weather forecasting system based on physical modelling to realise more accurate and feasible forecasting strategies.
Information Technology Operating Analytics (ITOA) is very similar. Just imagine the weather data becomes the IT operating system’s data and the predicted “temperature” becomes the computer’s parameters. By investigating the result values or ranges, researchers can see whether the system is critical or not. Obviously, this topic is playing a more significant role under the booming trend of digital applications because of its ability to detect and predicting future events.
I’ll be further investigating this area of research as my postgraduate thesis, working with T-DAB.AI and CJC. CJC is a global market data technology consulting and services firm. CJC have provided T-DAB with access to the time-series data collected from monitoring the infrastructure used to implement a tier 1 banks market data system, my ongoing project focuses on analysis of that data. The main tasks involve classification and clustering of different data/events types, real-time anomaly detection of time-series parameters, as well as some studies of time-to-event prediction. In my next blog, I would like to concentrate on anomaly detection and will provide updates on time-to-event predictions in a later post once it progresses.
