A Framework for Evaluating and Benchmarking Concept Drift Detection Methods
arXiv:2606. 07789v1 Announce Type: new Abstract: Data stream mining is fundamentally challenged by concept drift, where distributional changes can degrade model performance.
The paper introduces the Stream Cruise Control Method (SCCM), a framework for detecting and adapting to concept drift in online regression. SCCM performs early-response drift detection, quantifies drift magnitude, applies KPI-window-based thresholding to reduce false alarms, dynamically tunes hyperparameters, and recalibrates models, all within an in-memory design for real-time operation. Evaluations on synthetic and real-world datasets demonstrate that SCCM improves predictive performance compared to eight baseline detector–adaptation methods.
arXiv:2606. 07789v1 Announce Type: new Abstract: Data stream mining is fundamentally challenged by concept drift, where distributional changes can degrade model performance.
arXiv:2606. 20216v1 Announce Type: cross Abstract: Machine learning algorithms deployed for evolving streaming environments must handle the non-stationary data distributions, commonly referred to as concept drift.
arXiv:2608. 19488v1 Announce Type: new Abstract: Production machine learning systems degrade under concept drift, yet practitioners have little principled guidance on when to retrain.
arXiv:2608. 02845v1 Announce Type: new Abstract: Tabular model performance degrades when feature distributions change over time or the relationship between features and outcome variables change over time, known as data drift and concept drift, respectively.
arXiv:2606. 09430v1 Announce Type: cross Abstract: Online task-free continual learning (TFCL) requires intelligent agents to sequentially accumulate knowledge from an unbounded, non-stationary data stream under strict single-pass constraints and without any explicit task identifiers.
The paper investigates when online adaptation benefits edge time‑series forecasting under distribution drift, using a leakage‑free streaming protocol on six public multivariate datasets. It shows that the warmup budget for static baselines and the choice of learning rate can bias perceived adaptation gains, and that a validation‑only procedure selecting warmup and optimizer rates yields Adam outperforming SGD with momentum in most settings. The study also examines accuracy versus adaptation‑state memory and per‑update latency for different adaptation strategies, highlighting parameter‑efficient variants that are nondominated on the memory axis.
arXiv:2505. 04608v5 Announce Type: replace-cross Abstract: Responsibly deploying artificial intelligence (AI) / machine learning (ML) systems in high-stakes settings arguably requires not only proof of system reliability, but also continual, post-deployment monitoring to quickly detect and address any unsafe behavior.
The paper introduces the Continuous Evolution Pool (CEP), a replay‑free framework for online time series forecasting that tackles recurring concept drift. CEP maintains a dynamic pool of specialized forecasters, using lightweight statistical genes to identify concepts, spawn new models when distribution shifts occur, and prune obsolete ones under memory limits. Experiments on real‑world datasets show CEP reduces forecasting error by up to 24% compared to state‑of‑the‑art baselines, especially in scenarios with pronounced recurring drift.
arXiv:2606. 31420v1 Announce Type: new Abstract: Test-Time Adaptation (TTA) enables models trained on a source domain to adapt online to unlabeled test data under distribution shifts.
arXiv:2606. 04164v1 Announce Type: cross Abstract: Data samples used for training often differ from those encountered during fine-tuning and deployment, and while ML models show promise, their performance remains limited when only small annotated datasets are available.
arXiv:2607. 18899v1 Announce Type: new Abstract: Forecasting under real-world conditions is inherently non-stationary, as the conditional distribution of future observations evolves over time.
arXiv:2608. 11349v1 Announce Type: cross Abstract: A key obstacle to deploying reinforcement learning in real-world systems is hyperparameter selection, particularly when simulators are unavailable and online experimentation is costly.