Development of a Lightweight Framework for Runtime Monitoring of Data Drift in Edge-Deployed IoT Machine Learning Pipelines

Abstract

As machine learning models increasingly enter production, especially on resource-limited edge devices in Internet of Things (IoT) systems, concerns around their long-term reliability due to data drift also grow. Data drift can lead to severe performance degradation and inaccurate predictions, wrong decisions, and even system failure without the awareness of the operators. This study designs, implements, and evaluates a lightweight runtime monitoring framework using a combination of statistical and distance-based approaches as a modular and framework-agnostic monitoring solution, measuring accuracy, latency, and memory. The proposed framework design comprises a data interceptor, preprocessor and statistical summarizer, baseline storage with fixed-size reservoir storage, drift detection engine, and alerting and reporting interface. It makes use of three statistical and distance-based techniques (the Kolmogorov-Smirnov test, the Population Stability Index, and the Wasserstein Distance) combined using the majority vote to detect drifts in a model-agnostic and label-free manner. Performance was assessed on 20 synthetic scenarios and the London Bike Sharing dataset for real-world evaluation over five independent random seeds. A mean detection accuracy of 92.63% (95% CI: 86.79%–98.48%) and a constant memory footprint of 0.039 MB and stable latency (±1.59 ms) were achieved. Competitive benchmarking against Frouros, Evidently AI, and Alibi-Detect under identical hardware conditions demonstrated higher detection accuracy than Frouros and Evidently AI, the most stable latency among all evaluated tools, and three times lower memory usage than Evidently AI. These results demonstrate that lightweight statistical ensemble monitoring can provide practical and reliable drift detection for real-time IoT deployments without heavy infrastructure overhead.