Kitchenham & Charters 2007 — SLR Guidelines for Software Engineering

The canonical procedural reference for systematic literature reviews in software engineering. An EBSE technical report (v2.3, July 2007) that adapts medical evidence-based review methodology (Cochrane, CRD, Australian NHMR) to the realities of software engineering research, where RCTs are rare and empirical data is often proprietary.

Why it matters here

This guideline supplies the backbone procedure for any systematic literature review performed in or about BPM, predictive process monitoring, or agentic BPM. Several sources already in the wiki explicitly invoke SLR methodology — sources/2020-rama-maneiro-deep-learning-ppm-review, sources/2019-verenich-survey-ppm, the outcome-PPM benchmark cited in concepts/outcome-prediction — and are implicitly Kitchenham-shaped. See methods/systematic-literature-review for the distilled procedure.

The three-phase procedure

1. Planning — Identify the need (confirm no existing SLR answers the question); optionally commission; specify the research question(s) using PICO / PICOC framing; develop a review protocol (search strategy, inclusion/exclusion criteria, quality instrument, data extraction form, synthesis plan); evaluate the protocol (ideally by independent peers). The protocol is the main safeguard against researcher bias.

2. Conducting — Identification of research (automated + manual search, publication-bias control, bibliography management, documenting the search trail); study selection against pre-registered criteria (with inter-rater reliability checks); study quality assessment using a graded instrument (DARE's four items as a minimum: inclusion/exclusion appropriateness, search coverage, quality/validity assessed, data adequately described); data extraction via a pre-designed form; data synthesis (descriptive/narrative, quantitative/meta-analytic, qualitative, or mixed — with sensitivity analysis and explicit publication-bias treatment).

3. Reporting — Dissemination strategy (journal, tech report, poster, web); formatting the main report (abstract, background, questions, methods, included/excluded studies, results, discussion, conclusions, appendices); evaluating the report against a checklist.

Adaptations for software engineering

SE is similar to the social sciences (0.83 Budgen-similarity to education and nursing) and dissimilar to clinical medicine (0.17). Consequences: pre-specifying a narrow study-design filter (e.g. RCTs only) is rarely feasible; protocols must aggregate heterogeneous study types; quality checklists need SE-specific items; surrogate-measure risks (e.g. defects-in-testing as a proxy for quality) must be explicit.

Variants

Systematic mapping study (scoping study) — plots evidence clusters across a broad topic at low granularity; precursor to an SLR.
Tertiary review — an SLR of SLRs in a mature domain.

Connections