---
title: "ProcessTransformer: Predictive Business Process Monitoring with Transformer Network"
type: source
tags: [ppm, transformer, deep-learning, foundational]
authors: [Bukhsh Zaharah A.; Saeed Aaqib; Dijkman Remco M.]
year: 2021
venue: "arXiv:2104.00721v1"
kind: paper
raw_path: "raw/Predictive process monitoring/ProcessTransformer 2021.pdf"
duplicate_of_raw: "raw/Predictive process monitoring/ProcessTransformer (2021).pdf"
created: 2026-04-13
updated: 2026-04-13
key_claims:
  - LSTM-based PPM suffers from one-hot encoding inefficiency, weak long-range dependency modelling, and no parallelisation.
  - Transformer self-attention enables global dependencies and parallel training.
  - ProcessTransformer outperforms LSTM baselines on next-activity (~80%+ accuracy on 9 logs).
  - Competitive on event-time and remaining-time tasks.
---

# Bukhsh, Saeed, Dijkman 2021 — ProcessTransformer

Landmark paper bringing **[[concepts/transformer-ppm|Transformer architectures]]** to PPM. Arguably the most cited Transformer-PPM reference.

## Contribution
- Transformer encoder stack applied directly to event sequences (no recurrent component).
- Learned activity embeddings replace one-hot encoding.
- Self-attention captures long-range dependencies.
- Evaluated on nine public logs; ~80%+ next-activity accuracy on average.

## Limitations (noted by authors)
- Remaining-time / event-time performance is "competitive" but not clearly superior to LSTM baselines — contrasted with the clear next-activity win.
- Positional encoding is standard sinusoidal; process-specific positional biases remain unexplored.

## Significance
Opened the **Transformer-PPM era** (2021+), extended by MTLFormer ([[sources/2023-wang-mtlformer-multitask-transformer]]) and countless subsequent variants. Cited as baseline in nearly every post-2021 PPM paper.

## Connections
**Concepts:** [[concepts/transformer-ppm]] · [[concepts/next-activity-prediction]] · [[concepts/remaining-time-prediction]]
**Related sources:** [[sources/2017-tax-lstm-process-prediction]] (predecessor) · [[sources/2023-wang-mtlformer-multitask-transformer]] (successor)