---
title: "Handling Concept Drift for Predictions in Business Process Mining"
type: source
tags: [ppm, concept-drift, online-learning, retraining-strategies, naive-bayes, page-hinkley, adwin, p2p-process, kit]
authors: [Baier, Lucas; Reimold, Josua; Kühl, Niklas]
year: 2020
venue: "IEEE 22nd Conference on Business Informatics (CBI) 2020, pp. 76–83. DOI: 10.1109/CBI49978.2020.00016"
kind: paper
raw_path: "raw/Predictive process monitoring/2020-baier-handling-concept-drift-bpm.pdf"
doi: "10.1109/CBI49978.2020.00016"
sources: []
key_claims:
  - "Empirical industrial case study: P2P (procure-to-pay) event log from a large German company, 70,774 purchase transactions 2016–2018, with throughput-time prediction transformed into a 3-class problem (≤6 days / 7–39 days / >39 days)."
  - "Static-baseline accuracy 0.5400 (no retraining, no drift detection) → 0.7010 with full drift handling (Page-Hinkley detection + incremental learning + 'last' data-selection strategy + batch size 500), a +30 percentage-point absolute gain or +28.5 % relative."
  - "Three retraining data-selection strategies systematically compared — next (batch after drift), mixed (around drift), last (most recent before drift). The 'last' strategy consistently outperforms 'next' and 'mixed' across all drift detectors and batch sizes."
  - "Drift root cause empirically identified in the case study: a step rise in process automation level after ~35,000 instances acts as hidden context driving prediction degradation — automation feature not available at prediction time."
  - "Page-Hinkley detection slightly outperforms ADWIN under all tested batch sizes; smaller batches (500–1000) consistently beat larger batches (2000–5000)."
  - "Managerial implication: predictive services in BPM require continuous monitoring + automated adaptation — otherwise model quality silently decays and operational value erodes."
created: 2026-05-11
updated: 2026-05-11
---

# Baier, Reimold & Kühl 2020 — Handling Concept Drift for Predictions in Business Process Mining

Empirical short paper (8 pp) from Karlsruhe Institute of Technology (KIT) presented at IEEE CBI 2020. Bridges two literatures the wiki tracks separately: [[concepts/predictive-process-monitoring|PPM]] and [[concepts/concept-drift|concept drift in machine learning]]. Demonstrates on a real industrial event log that ignoring drift in deployed PPM services silently destroys predictive value within months — and that systematic retraining strategy choice matters more than detector choice.

## What the paper does

Two research questions:

- **RQ1** — Which data should be used for retraining of a machine-learning model when a concept drift is detected?
- **RQ2** — What are the effects of the different retraining options in a real-life use case in process mining?

Uses a P2P (procure-to-pay) process event log from a large German company — 70,774 transactions over 2016–2018, accessed via a process-mining solution provider. Target variable: *throughput time* (delivery time from purchase-order creation to goods reception), discretised into three classes for tractability.

Features after one-hot encoding: material class (123), document type, plant code, supplier country, bank country, purchasing group, plus numeric purchase-order value (Box-Cox transformed).

## Method

- **Library**: `scikit-multiflow` (a streaming-data extension of scikit-learn).
- **Classifier**: Naïve Bayes (chosen for incremental-learning capability + low computational cost; outperformed Decision Tree, SVM in pretest on a stationary 2,000-instance subset).
- **Drift detectors**: Page-Hinkley test (λ=0.6) and ADWIN (δ=0.001), both grid-searched on first 10,000 instances.
- **Retraining strategies** (Fig. 2 in the paper):
  - **next** — retrain on the next batch *after* drift detection.
  - **mixed** — retrain on data spanning before *and* after detection.
  - **last** — retrain on the *last* batch *before* drift detection (intuition: drift detectors fire with delay; the most recent pre-alarm batch may already belong to the new concept).
- **Batch sizes**: 500, 1000, 2000, 5000.

## Results

Static-baseline performance (no retraining, no drift detection) over the full 70,774-instance stream: accuracy **0.5400**. With incremental learning alone: 0.6717 (+24.4 %). With Page-Hinkley + retraining only: 0.6732. With both: **0.6938** (+28.5 %). Best individual configuration: Page-Hinkley + incremental + `last` + batch=500 → **0.7010**.

Across all 16 grid cells (2 detectors × 4 batch sizes × 3 strategies × incremental on), `last` wins every time. Page-Hinkley slightly outperforms ADWIN. Smaller batches outperform larger.

Root-cause exploration: a step rise in the *automation* feature (rolling-mean window=1000) around instance 35,000 matches the prediction-quality drop in the static-baseline curve. Conclusion: process-automation rollout acted as hidden context driving drift.

## Limitations explicitly acknowledged

- Single use case, single classifier, single target-variable transformation (regression → 3-class).
- Three retraining strategies tested; more sophisticated approaches (e.g. ensemble, transfer learning, meta-learning) untested.
- Recommendation generalisability requires replication on additional logs/benchmarks.

## Connections

**Concepts:**
- [[concepts/concept-drift]] — empirical anchor for the "drift detection + retraining" subfamily in PPM.
- [[concepts/predictive-process-monitoring]] — extends the corpus into the drift-aware-prediction subspace.
- [[concepts/lstm-ppm]], [[concepts/transformer-ppm]] — orthogonal direction (the paper uses Naïve Bayes precisely because it admits incremental updates).

**Sources cited that exist in the wiki:**
- [13] [[sources/2013-bose-vanderaalst-concept-drift-pm]] — *priority referenced source; not yet in raw/*, the foundational drift-in-PM paper.
- [16] van der Aalst et al. 2012 process-mining manifesto — referenced.
- [26] [[sources/2018-dumas-fundamentals-of-bpm]].
- [28] [[sources/2019-verenich-survey-ppm]] — remaining-time survey/benchmark.
- [32] Maaradji et al. 2017 detecting drifts — *priority referenced source*.

**Syntheses:**
- [[syntheses/ppm-landscape]] — augments the §6 "open threads" list with a concrete drift-handling reference; complementary to [[sources/2022-riess-metaheuristics-concept-drift-survey]] which surveys *adaptation* via metaheuristics.
- [[syntheses/concept-drift-in-pm]] — *anchor source*; concrete empirical case study alongside Sato survey + Riess metaheuristics review.