Quantitative inquiry is a family of approaches for describing distributions, estimating population quantities, comparing groups, measuring change, studying associations, quantifying uncertainty, evaluating measures, and summarizing structured data. It is not automatically causal analysis.
Part IV · Chapter 9
Counting Politics
Quantitative inquiry is a family of approaches for describing distributions, estimating population quantities, comparing groups, measuring change, studying associations, quantifying uncertainty, evaluating measures, and summarizing structured data. It is not automatically causal analysis.
Learning objectives
What this chapter asks you to be able to do.
- Explain the major inferential jobs of quantitative political research without treating all quantitative analysis as causal analysis.
- Trace the chain from concept to measure, population, sampling frame, selected sample, respondents or observations, analyzed cases, estimate, and uncertainty.
- Distinguish reliability from validity and explain why a stable measure can still measure the wrong construct.
- Compare probability and nonprobability sampling as families of designs with different inferential resources and assumptions.
- Distinguish sampling variance from coverage, nonresponse, measurement, processing, weighting, timing, and construct errors.
- Explain why response rate alone is not a sufficient measure of nonresponse bias and why weighting cannot recover people or phenomena that were never observed.
- Interpret counts, proportions, rates, means, medians, quantiles, and distributions with explicit attention to denominators and subgroup structure.
- Explain why missing values are not automatically zeros and why administrative records are products of institutional processes rather than neutral mirrors of populations.
- Interpret confidence intervals and statistical significance without allowing one interval or threshold to absorb every form of uncertainty.
- Evaluate graphs and maps as consequential analytical choices rather than passive displays.
- Identify the statistical, privacy, and political tensions created by disaggregation and small cells.
- Evaluate AI-assisted quantitative work as assistance or analytical influence requiring provenance, validation, and human responsibility.
- Produce Decision Dossier IX: a Quantitative Evidence and Measurement Record that carries forward Chapters 7 and 8.
Chapter summary
The argument in inspectable form.
A number can be exactly calculated and still estimate the wrong quantity. The estimand and target population must be explicit.
Concept formation precedes measurement. Qualitative evidence can improve quantitative measures by revealing dimensions, wording, categories, and meanings that an initial instrument missed.
Reliability concerns consistency; validity concerns whether the intended interpretation is supported. Reliability does not imply validity.
Target population, sampling frame, selected sample, contacted cases, respondents or observations, and analyzed cases are distinct sets.
Probability sampling supplies a design-based basis for population inference when its frame and implementation are appropriate, but it does not eliminate coverage, nonresponse, measurement, or processing errors.
Nonprobability samples can be useful, but population inference requires explicit assumptions. Large N is not representativeness.
Sampling error is only one component of total survey error. A narrow interval around a biased estimate remains a precise estimate of the wrong quantity.
Response rate is an important fieldwork statistic but not a universal measure of nonresponse bias.
Weights can correct some known selection and response imbalances under assumptions; they cannot create observations for populations with no route into the data.
Counts, rates, means, medians, quantiles, spread, and distributions answer different questions. Numerators require denominators, and means can hide polarization or concentrated harm.
Missing is not zero. Administrative data are generated by institutional rules and operational behavior and therefore require data-generating-process analysis.
Confidence intervals quantify specified statistical uncertainty under assumptions; they do not automatically contain construct, frame, nonresponse, measurement, processing, or model uncertainty.
Statistical significance is not substantive importance, the probability that a hypothesis is true, or a public-policy verdict.
Visualization choices shape salience. A numerically faithful graph can still overstate difference or certainty.
Disaggregation can reveal hidden harms while also increasing noise, multiple-comparison risk, and re-identification risk.
AI can assist quantitative work but requires task-specific validation, transformation provenance, data protection, and accountable human verification.
Current NousPolis N0 architecture contains useful evidence, assumption, provenance, metric, and ontology elements but does not by itself prove that every quantitative estimand, denominator, frame, missingness state, or dashboard salience choice is safely governed.
Association is not causation. Quantitative finding, normative judgment, legal authority, and democratic authorization remain separate.
Key terms
Concepts to carry forward.
- estimand
- The precisely defined population quantity or contrast the analysis is intended to estimate.
- parameter
- A quantity characterizing a population or statistical model, such as a population mean or proportion.
- estimate
- A value calculated from observed data and used to learn about a target parameter or estimand.
- construct
- An abstract concept, such as trust or political efficacy, that requires explicit definition before measurement.
- reliability
- The consistency or stability of a measurement procedure under relevant repeated conditions.
- validity
- The degree to which evidence and theory support interpreting a measure as representing the intended concept for the intended use.
- target population
- The set of units about which the intended claim is made.
- sampling frame
- The operational list or mechanism from which units can actually be selected or contacted.
- probability sample
- A design in which selection is governed by known random mechanisms giving eligible units known, nonzero inclusion probabilities under the design.
- nonprobability sample
- A sample whose inclusion mechanism is not fully governed by known probability selection; inference therefore depends more heavily on substantive and modeling assumptions.
- coverage error
- Error created when the frame or data system does not adequately cover the target population.
- nonresponse
- Failure to obtain a required observation from a selected or otherwise eligible unit, or failure to obtain a particular item.
- weight
- A factor used so that observations contribute unequally to an estimate, often reflecting selection probabilities or adjustments for imbalance or nonresponse.
- rate
- A quantity with an explicit denominator representing exposure, opportunity, population at risk, or another defined base.
- standard error
- A measure of how much an estimator would vary across repeated samples or comparable realizations under a specified design or model.
- confidence interval
- An interval produced by a procedure designed to achieve a stated long-run coverage rate under its assumptions.
- statistical significance
- A statement about the relation between observed data and a specified statistical model or null hypothesis; it is not a measure of political importance.
- administrative data
- Records created primarily through the operation of programs, services, regulation, taxation, registration, or other institutional processes rather than solely for the present research question.
- missingness
- The condition in which a value relevant to the analysis is unobserved, with causes that may themselves be informative.
- disaggregation
- Reporting or analyzing results for subgroups rather than only for an aggregate population.
Review & discussion
Questions for seminar, revision, or assessment.
- Why can 72 percent be a perfectly correct calculation and still be a misleading statement about Asterbridge?
- Give an example of a reliable political measure that would nevertheless be invalid for the claim made about it.
- Why did Chapter 8 qualitative evidence improve the Chapter 9 trust survey rather than merely provide an anecdotal supplement?
- Draw the difference among target population, sampling frame, selected sample, respondent sample, and analyzed cases for the Asterbridge trust survey.
- What does probability sampling give a researcher that a large volunteer sample does not automatically provide? What errors can remain in both?
- Give one research question for which a nonprobability sample would be useful without supporting a population-prevalence claim.
- Why can a survey with a 40 percent response rate have less nonresponse bias for one estimate than a survey with an 80 percent response rate for another estimate?
- Explain why weighting current riders cannot automatically recover the views of former riders who had no chance of receiving the survey.
- Construct two politically different trust distributions that have the same mean. What would be lost by reporting only the mean?
- Give a public-policy example in which changing the denominator changes the substantive claim even though the numerator is unchanged.
- Why is an absent trip record not necessarily evidence of zero travel delay? List four plausible meanings of the absence.
- Choose one administrative dataset you know. What institutional rules cause a record to enter it, change it, or disappear from it?
- Explain the frequentist interpretation of a 95 percent confidence interval without saying that the fixed parameter has a 95 percent chance of being inside the realized interval.
- A result has p = .0001 but an effect so small that no resident would notice it. What additional information should a policy analyst report?
- How can a map be numerically correct and still manufacture a sense of crisis? Identify at least four visual or denominator choices.
- When can disaggregation improve fairness analysis, and when can it make an estimate too unstable or identifiable to publish safely?
- An AI assistant writes correct code but uses the wrong denominator. Is this a coding error, a statistical error, a conceptual error, or a governance error? Defend your classification.
- Which current NousPolis objects could store parts of a quantitative evidence chain, and what additional representation would you want before trusting a public dashboard?
- Write one reopening trigger for a survey frame and one for a measurement instrument. Each must state evidence that would materially change the estimate or its interpretation.
- At what point does a Chapter 9 estimate become an authorized public decision? Explain why the correct answer must remain compatible with "AI analyses. Humans authorise."
Further reading
Continue into the literature.
For the broad logic of survey quality and total survey error, begin with Groves et al. (2009). Lohr (2022) provides an accessible but rigorous treatment of probability sampling and modern sampling design. Tourangeau, Rips, and Rasinski (2000) remains a foundational account of how respondents understand, retrieve, judge, map, and report answers to survey questions. AAPOR's Standard Definitions (2023) is essential for transparent disposition and response-rate reporting, while Baker et al. (2013) provides a careful treatment of nonprobability sampling and the assumptions required for inference. Groves and Peytcheva (2008) is a useful corrective to mechanical interpretations of response rates. Meng (2018) shows why very large datasets can become highly precise and still badly misleading when data defect is correlated with the variable of interest. Little and Rubin (2019) is the standard advanced reference on missing data. For statistical communication, read Wasserstein and Lazar (2016) alongside Wasserstein, Schirm, and Lazar (2019). Cleveland and McGill (1984) supplies a classic empirical foundation for thinking about graphical perception. For current AI use in survey research, Rothschild et al. (2026) provides a field-specific framework organized around validity, performance, sensitivity, reliability, transparency, and human-subject responsibilities.
Companion, not replacement
The textbook carries the complete argument.
This page reproduces the chapter's study and navigation layer from the current living manuscript. The Asterbridge narrative, historical and methodological argument, figures, Political Science Lens boxes, NousPolis Canon crosswalks, labs, red-team exercises, and N5 Practice boundaries remain in the canonical textbook rather than being republished wholesale here.