Method
Administrative and descriptive analysis
Chapters 7, 9, 15- What am I trying to establish?
- What happened, to whom, where, and when?
- Evidence it needs
- Administrative records, service data, registers, budgets, transaction records, geospatial measures, descriptive statistics.
- What it can establish
- Patterns, levels, distributions, trends, coverage, implementation outputs, and measurable differences under the definitions used.
- What it cannot establish
- By itself establish why a pattern occurred, what would have happened otherwise, whether a distribution is just, or who is authorized to act.
- What would invalidate or materially weaken the inference?
- Wrong denominators, changing definitions, missing populations, data-generating-process changes, linkage errors, or measures that do not represent the concept.
Method
Survey research
Chapters 7, 9, 13- What am I trying to establish?
- What attitudes, experiences, behaviors, or characteristics can be inferred about a target population?
- Evidence it needs
- Probability or carefully bounded non-probability samples, questionnaires, sampling frames, weighting, response records.
- What it can establish
- Population estimates and subgroup patterns when the sampling and measurement design justify them.
- What it cannot establish
- Automatically establish causal effects, deep meaning behind responses, or represent people excluded from the sampling frame.
- What would invalidate or materially weaken the inference?
- Coverage error, severe nonresponse, ambiguous items, mode effects, invalid weighting, social-desirability bias, or unstable constructs.
Method
Interviews and focus groups
Chapters 7, 8, 13- What am I trying to establish?
- How do people understand an institution, event, policy, identity, or lived experience?
- Evidence it needs
- Recorded or documented interviews, group discussions, sampling/recruitment logic, transcripts, field notes, coding and analytic memos.
- What it can establish
- Meaning, mechanisms, experiences, categories, interpretations, and hypotheses grounded in contextual evidence.
- What it cannot establish
- By itself estimate prevalence in a population or convert vivid testimony into representativeness.
- What would invalidate or materially weaken the inference?
- Biased recruitment, leading protocol, missing negative cases, decontextualized quotations, unexamined researcher position, or weak interpretive audit trail.
Method
Observation and ethnographic inquiry
Chapters 5, 8, 15- What am I trying to establish?
- How do institutions, people, and practices actually behave in context?
- Evidence it needs
- Field observation, participant observation, process notes, institutional artifacts, interaction records, reflexive memos.
- What it can establish
- Practice, informal rules, routines, meaning, workarounds, discretion, and gaps between formal procedure and lived implementation.
- What it cannot establish
- Automatically generalize beyond the observed setting or establish counterfactual causal effects.
- What would invalidate or materially weaken the inference?
- Selective access, observer effects, unrecorded interpretive shifts, missing contexts, or claims broader than the field setting supports.
Method
Case study and process tracing
Chapters 5, 7, 10, 15- What am I trying to establish?
- Through what sequence or mechanism did an outcome plausibly emerge?
- Evidence it needs
- Within-case chronology, documents, interviews, decisions, institutional records, mechanism evidence, alternative explanations.
- What it can establish
- Mechanism evidence and structured within-case causal inference when observations discriminate among rival explanations.
- What it cannot establish
- Turn one case into a universal causal law or replace explicit comparison when generalization requires it.
- What would invalidate or materially weaken the inference?
- Cherry-picked chronology, weak causal-process observations, ignored alternatives, post-hoc mechanism construction, or scope conditions that do not travel.
Method
Experimental designs
Chapters 7, 10- What am I trying to establish?
- What is the effect of an intervention under a controlled assignment mechanism?
- Evidence it needs
- Random assignment, treatment/control conditions, prespecified outcomes, implementation and attrition records.
- What it can establish
- Causal effects for the study population and treatment as implemented, subject to design integrity.
- What it cannot establish
- Guarantee external validity, normative desirability, long-term effects, or lawful authority to scale the intervention.
- What would invalidate or materially weaken the inference?
- Broken randomization, attrition, noncompliance, spillovers, outcome switching, interference, or a treatment that differs materially from the policy of interest.
Method
Quasi-experimental causal inference
Chapters 9, 10- What am I trying to establish?
- Can an observational setting approximate a credible counterfactual?
- Evidence it needs
- Natural experiments, discontinuities, difference-in-differences, instrumental variables, matching/weighting, panel or interrupted-time designs.
- What it can establish
- Causal estimates only under the identification assumptions specific to the design.
- What it cannot establish
- Become causal merely because a regression is sophisticated or a before/after difference is large.
- What would invalidate or materially weaken the inference?
- Failed parallel trends, manipulation around thresholds, weak instruments, unmeasured confounding, anticipation, spillovers, or violated functional assumptions.
Method
Comparative and historical analysis
Chapters 2, 3, 5, 10- What am I trying to establish?
- How do institutions, sequences, cases, or political outcomes differ across time and place?
- Evidence it needs
- Comparable case records, archival material, institutional histories, structured comparisons, sequence and critical-juncture evidence.
- What it can establish
- Institutional contrasts, path dependence, sequence effects, scope conditions, and historically grounded explanations.
- What it cannot establish
- Treat countries or periods as interchangeable observations or assume a mechanism travels unchanged across contexts.
- What would invalidate or materially weaken the inference?
- Case-selection bias, incomparable concepts, presentism, missing historical alternatives, or unexamined context dependence.
Method
Computational text, network, and machine-learning analysis
Chapters 6, 11, 12- What am I trying to establish?
- What patterns can be extracted at scale from large or complex data?
- Evidence it needs
- Versioned corpora/datasets, code, transformations, model specifications, validation sets, error analysis, subgroup checks.
- What it can establish
- Patterns, classifications, predictions, associations, or representations within the validated task and data domain.
- What it cannot establish
- Make generated categories neutral, turn prediction into causation, or make model output politically authoritative.
- What would invalidate or materially weaken the inference?
- Corpus shift, label leakage, correlated error, weak validation, subgroup failure, non-reproducible preprocessing, undocumented model/version change, or objective mismatch.
Method
Forecasting and scenario analysis
Chapters 6, 11, 14, 15- What am I trying to establish?
- What outcomes are plausible under stated assumptions and alternative futures?
- Evidence it needs
- Historical data, structural assumptions, calibrated models, expert elicitation, scenario definitions, forecast evaluation.
- What it can establish
- Conditional expectations, ranges, scenario sensitivity, and comparative implications—not certainty about the future.
- What it cannot establish
- Convert assumptions into facts or justify a value-laden policy objective simply because one forecast ranks it highly.
- What would invalidate or materially weaken the inference?
- Poor calibration, structural break, assumption failure, model instability, hidden objective changes, or repeated forecast error.
Method
Evidence synthesis
Chapters 7, 12, 14- What am I trying to establish?
- What does a heterogeneous body of evidence support when claims, methods, contexts, and uncertainties differ?
- Evidence it needs
- Systematic search, claim-level extraction, study-quality assessment, lineage/independence checks, meta-analysis where appropriate, structured qualitative synthesis.
- What it can establish
- The weight, consistency, limits, disagreement, and conditionality of evidence at the claim level.
- What it cannot establish
- Manufacture consensus from incomparable evidence or let publication count substitute for source independence and quality.
- What would invalidate or materially weaken the inference?
- Biased search, duplicated evidence, incompatible outcomes, ignored negative studies, hidden judgment in weighting, or loss of material dissent.