---
title: Explainability in APM
type: concept
tags: [apm, xai, explainability, trust, compliance]
sources: ["[[sources/2026-calvanese-agentic-bpm-manifesto]]", "[[sources/2026-padella-llm-features-ppm]]", "[[sources/2026-theodorakopoulos-bi-bpm-genai-review]]"]
created: 2026-04-13
updated: 2026-05-11
---

# Explainability in APM

The second required capability of an APM agent ([[sources/2026-calvanese-agentic-bpm-manifesto]], §3.2). Since APM systems compose AI agents, explainability is realised by endowing agents with the intrinsic capability to **articulate the rationale behind their behaviour and actions**. This includes agents assigned collective process-awareness responsibilities.

Each agent's specification should define:
- The **explanans** (the explanation itself) — what must be articulated.
- The **explanandum** (the triggering condition) — when an explanation is due.
- Which **XAI technique(s)** are to be employed.

## Why explainability is first-class in APM
- **Trust** — stakeholders must rely on agent recommendations and decisions.
- **Opacity mitigation** — AI agents may be black boxes; debugging requires articulated rationale.
- **Accountability** — assigning responsibility for failures or unfair decisions.
- **Bias detection** — APM agents can perpetuate hidden biases of underlying AI models.
- **Regulatory compliance** — GDPR, EU AI Act, especially in finance, healthcare, HR.

## Desirable properties of APM explanations
1. Conform to framing constraints and statements.
2. Capture the richness of contextual information that affects decisions.
3. Reflect causal execution dependencies among actions.
4. Be interpretable to other agents (human or digital).

## Role in the APM capability stack
Explainability supports autonomy in two ways:
- Enables agents to independently resolve misalignment in other agents' behaviour.
- Reduces human intervention by making behaviour understandable and transparent.

## Mechanisms documented in the wiki

| Mechanism | Source | What it explains |
|---|---|---|
| White-box flow analysis | [[sources/2017-verenich-white-box-flow-analysis]] | PPM remaining-time predictions via classical flow analysis. |
| Explainable gated RNN (reachability graph) | [[sources/2023-cao-gated-rnn-explainable]] | Sequence-model predictions mapped onto a reachability graph. |
| [[concepts/beta-learner-distillation\|β-learner distillation]] | [[sources/2026-padella-llm-features-ppm]] | LLM-PPM reasoning distilled into a catalogue of interpretable reasoning patterns (β-learners), each re-implementable as a standalone model; quantifies whether the LLM reasons higher-order or merely replicates. |
| [[concepts/semantic-hashing-probe\|Semantic-hashing probe]] | [[sources/2026-padella-llm-features-ppm]] | Isolates LLM-PPM reliance on embodied prior knowledge — relevant to APM benchmark-contamination challenge C3. |

## Limits flagged in the wider literature

[[sources/2026-theodorakopoulos-bi-bpm-genai-review|Theodorakopoulos & Theodoropoulou 2026]] (Section 9.2) caution that **explainability ≠ effective oversight**:

- Cecil et al. find explainable AI advice does not significantly reduce people's tendency to follow incorrect advice.
- Buçinca et al. show simple explanations are weaker than **cognitive forcing** mechanisms for reducing over-reliance.
- Afroogh et al. distinguish *trustworthiness* (system property) from *trust* (user disposition); over-trust is as problematic as resistance.

Consequence for APM agent design: **trust should be calibrated, not maximised**. Explanation layers can cosmetically stabilise weak outputs without improving managerial reasoning quality.

## Related
[[concepts/agentic-bpm]] · [[concepts/framed-autonomy]] · [[concepts/conversational-actionability]] · [[concepts/beta-learner-distillation]] · [[concepts/semantic-hashing-probe]] · [[concepts/llm-based-ppm]]