Baier, Reimold & Kühl 2020 — Handling Concept Drift for Predictions in Business Process Mining

Empirical short paper (8 pp) from Karlsruhe Institute of Technology (KIT) presented at IEEE CBI 2020. Bridges two literatures the wiki tracks separately: PPM and concept drift in machine learning. Demonstrates on a real industrial event log that ignoring drift in deployed PPM services silently destroys predictive value within months — and that systematic retraining strategy choice matters more than detector choice.

What the paper does

Two research questions:

RQ1 — Which data should be used for retraining of a machine-learning model when a concept drift is detected?
RQ2 — What are the effects of the different retraining options in a real-life use case in process mining?

Uses a P2P (procure-to-pay) process event log from a large German company — 70,774 transactions over 2016–2018, accessed via a process-mining solution provider. Target variable: throughput time (delivery time from purchase-order creation to goods reception), discretised into three classes for tractability.

Features after one-hot encoding: material class (123), document type, plant code, supplier country, bank country, purchasing group, plus numeric purchase-order value (Box-Cox transformed).

Method

Library: scikit-multiflow (a streaming-data extension of scikit-learn).
Classifier: Naïve Bayes (chosen for incremental-learning capability + low computational cost; outperformed Decision Tree, SVM in pretest on a stationary 2,000-instance subset).
Drift detectors: Page-Hinkley test (λ=0.6) and ADWIN (δ=0.001), both grid-searched on first 10,000 instances.
Retraining strategies (Fig. 2 in the paper):
- next — retrain on the next batch after drift detection.
- mixed — retrain on data spanning before and after detection.
- last — retrain on the last batch before drift detection (intuition: drift detectors fire with delay; the most recent pre-alarm batch may already belong to the new concept).
Batch sizes: 500, 1000, 2000, 5000.

Results

Static-baseline performance (no retraining, no drift detection) over the full 70,774-instance stream: accuracy 0.5400. With incremental learning alone: 0.6717 (+24.4 %). With Page-Hinkley + retraining only: 0.6732. With both: 0.6938 (+28.5 %). Best individual configuration: Page-Hinkley + incremental + last + batch=500 → 0.7010.

Across all 16 grid cells (2 detectors × 4 batch sizes × 3 strategies × incremental on), last wins every time. Page-Hinkley slightly outperforms ADWIN. Smaller batches outperform larger.

Root-cause exploration: a step rise in the automation feature (rolling-mean window=1000) around instance 35,000 matches the prediction-quality drop in the static-baseline curve. Conclusion: process-automation rollout acted as hidden context driving drift.

Limitations explicitly acknowledged

Single use case, single classifier, single target-variable transformation (regression → 3-class).
Three retraining strategies tested; more sophisticated approaches (e.g. ensemble, transfer learning, meta-learning) untested.
Recommendation generalisability requires replication on additional logs/benchmarks.

Connections

Concepts:

concepts/concept-drift — empirical anchor for the "drift detection + retraining" subfamily in PPM.
concepts/predictive-process-monitoring — extends the corpus into the drift-aware-prediction subspace.
concepts/lstm-ppm, concepts/transformer-ppm — orthogonal direction (the paper uses Naïve Bayes precisely because it admits incremental updates).

Sources cited that exist in the wiki:

[13] sources/2013-bose-vanderaalst-concept-drift-pm — priority referenced source; not yet in raw/, the foundational drift-in-PM paper.
[16] van der Aalst et al. 2012 process-mining manifesto — referenced.
[26] sources/2018-dumas-fundamentals-of-bpm.
[28] sources/2019-verenich-survey-ppm — remaining-time survey/benchmark.
[32] Maaradji et al. 2017 detecting drifts — priority referenced source.

Syntheses:

syntheses/ppm-landscape — augments the §6 "open threads" list with a concrete drift-handling reference; complementary to sources/2022-riess-metaheuristics-concept-drift-survey which surveys adaptation via metaheuristics.
syntheses/concept-drift-in-pm — anchor source; concrete empirical case study alongside Sato survey + Riess metaheuristics review.