End-to-end Anomaly Detection, Correction and Prediction of Missing Values in Historical Daily Temperature Timeseries

In this paper, we present a deep learning solution to detect and correct anomalous values present in historical temperature timeseries, that are likely associated to human and weather instruments errors. Our solution consists in a joint peaks detection and end-to-end sequence prediction involving synchronous measurements of individual meteorological stations along with their neighboring peers. We designed our models in a way that the false positive rate (FPR) of the anomaly detection is minimized and the accuracy maximized, so that the historical records are corrected as less as possible. The method was applied to temperature records of 24 meteorological stations in Belgium, and allowed to automatically correct more than 80% of all errors in both max/min daily temperature records by modifying less than 15% of all the timeseries values, with an overall detection accuracy of 90%. The corrected temperature timeseries yielded a perfect match with respect to errors-free signals in several climate indicators. Our method can be potentially applied to other historical timeseries such as precipitation.