Explainability in APM

The second required capability of an APM agent (sources/2026-calvanese-agentic-bpm-manifesto, §3.2). Since APM systems compose AI agents, explainability is realised by endowing agents with the intrinsic capability to articulate the rationale behind their behaviour and actions. This includes agents assigned collective process-awareness responsibilities.

Each agent's specification should define:

The explanans (the explanation itself) — what must be articulated.
The explanandum (the triggering condition) — when an explanation is due.
Which XAI technique(s) are to be employed.

Why explainability is first-class in APM

Trust — stakeholders must rely on agent recommendations and decisions.
Opacity mitigation — AI agents may be black boxes; debugging requires articulated rationale.
Accountability — assigning responsibility for failures or unfair decisions.
Bias detection — APM agents can perpetuate hidden biases of underlying AI models.
Regulatory compliance — GDPR, EU AI Act, especially in finance, healthcare, HR.

Desirable properties of APM explanations

Conform to framing constraints and statements.
Capture the richness of contextual information that affects decisions.
Reflect causal execution dependencies among actions.
Be interpretable to other agents (human or digital).

Role in the APM capability stack

Explainability supports autonomy in two ways:

Enables agents to independently resolve misalignment in other agents' behaviour.
Reduces human intervention by making behaviour understandable and transparent.

Mechanisms documented in the wiki

Mechanism	Source	What it explains
White-box flow analysis	sources/2017-verenich-white-box-flow-analysis	PPM remaining-time predictions via classical flow analysis.
Explainable gated RNN (reachability graph)	sources/2023-cao-gated-rnn-explainable	Sequence-model predictions mapped onto a reachability graph.
β-learner distillation	sources/2026-padella-llm-features-ppm	LLM-PPM reasoning distilled into a catalogue of interpretable reasoning patterns (β-learners), each re-implementable as a standalone model; quantifies whether the LLM reasons higher-order or merely replicates.
Semantic-hashing probe	sources/2026-padella-llm-features-ppm	Isolates LLM-PPM reliance on embodied prior knowledge — relevant to APM benchmark-contamination challenge C3.

Limits flagged in the wider literature

Theodorakopoulos & Theodoropoulou 2026 (Section 9.2) caution that explainability ≠ effective oversight:

Cecil et al. find explainable AI advice does not significantly reduce people's tendency to follow incorrect advice.
Buçinca et al. show simple explanations are weaker than cognitive forcing mechanisms for reducing over-reliance.
Afroogh et al. distinguish trustworthiness (system property) from trust (user disposition); over-trust is as problematic as resistance.

Consequence for APM agent design: trust should be calibrated, not maximised. Explanation layers can cosmetically stabilise weak outputs without improving managerial reasoning quality.

concepts/agentic-bpm · concepts/framed-autonomy · concepts/conversational-actionability · concepts/beta-learner-distillation · concepts/semantic-hashing-probe · concepts/llm-based-ppm