Normative Choices

It may be asked whether choices underlying measurement design are normative and, if so, in what sense? If data are constrained and exactly one educational variable exists, in what sense is its selection normative? Similarly, if an indicator is redundant or invalid according to statistical assessments, how is its deselection normative? And if nutritional experts judge that an indicator of stunting is more accurate than wasting, in what way is a choice in its favour normative?

Normative considerations operate at different levels.

Symbol 0 refers to a d-dimensional vector of zeros. is more likely to improve welfare than not releasing it.^[181] This assessment may be made after consideration of what Sen (2009) terms a ‘comprehensive' description of the situation. At a lower level, in each part of measurement design, value judgements are used to justify particular choices—like dimensions, weights, and poverty cutoffs. The value judgements may pertain to the content directly, or they may address the methodologies or processes by which to justify design choices, as later sections will illustrate.

At this higher meta-level, the comprehensive description and its normative assessment will draw upon different kinds of analyses—statistical, axiomatic, deliberative, practical, and policy-oriented, for example—to authorize the use of measures that fulfil a set of plural purposes reasonably well.

These higher-level reasoned judgements that draw on a comprehensive description of the options often include the following types of assessments:

Expert (including qualitative) assessments of indicator accuracy—for example, in showing the level and changes of a key functioning like nutrition (Svedberg 2000) or the quality and legitimacy of a participatory process.

Empirical assessments, which could include analyses of measurement error, data quality, redundancy, robustness, statistical validity and reliability, or triangulation with other analyses and data sources.

Deliberative insights on people’s values from participatory discussions, social movements, consultations, and from documentation of similar recent processes.

Theoretical assessments, which could consider properties and principles, sets of dimensions, standards or conventions on indicators, or legal and policy frameworks.

Practical constraints such as constraints of data, time, human resources, authority, political will, and political feasibility given the processes and authorities involved.

Policy relevance—for example, how the timing and content of the measure could dovetail with resource allocation decisions or how a measure might support and monitor a set of planned interventions as set out in a national plan or a current campaign.

This section introduces this meta-coordination role of normative reasoning; later sections describe how particular kinds of assessment mentioned already may inform particular design choices.

The higher normative function is inextricably linked to the purpose(s) of the measure, which are often multiple and normally motivate policy and public action. As Foster and Sen put it, ‘The general conclusion that seems irresistible is that the choice of a poverty measure must, to a great extent, depend on the nature of the problem at hand' (1997:187).

A relevant example is Mexico's move towards a multidimensional poverty measure. In his book Numbers that Move the World, Miguel Szdkely points out that:

Just like there are ideas that move the world, so too there are numbers and statistics that move the world. A number can awaken consciences; it can mobilize the reluctant, it can ignite action, it can generate debate; it can even, in the best of circumstances, lay to rest a pressing problem (2005: 13).^[182]

In describing Mexico's steps towards new options of poverty measurement, Szdkely describes how a committee was formed whose mandate was ‘to propose to the Secretary of Social Development a methodology that could be officially adopted as an instrument of the Mexican government to measure the magnitude of poverty, its intensity, and its characteristics' (Szdkely 2005: 17).

- It must be understandable and easy to describe.

- It must reflect ‘common-sense' notions of poverty.

- It must fit the purpose for which it is being developed.

- It must be technically solid.

- It must be operationally viable—e.g. in terms of data requirements.

- It must be easily replicable (Szdkely 2005: 10 and 19).^[183]

As these criteria suggest, there are usually plural desiderata for a measure, and these must be taken into account within a coordinating normative framework.^[184] Consider the first purpose: a measure should be simple and easy to communicate. Earlier we observed that the widely used headcount ratio of income poverty lacks some very desirable proper­ties. Indeed, because the headcount ratio ‘ignores the “depth” as well as the “distribution” of poverty', Foster and Sen found it ‘remarkable that most empirical studies of poverty tend, still, to stop at the head-count ratio' (1997: 168 and 169). On the other hand, when formulated as a criterion, it becomes evident that this characteristic—that a measure not only be axiomatically sound and empirically solid, but also easy to understand—is actually essential if the measure is to inform and engage public debate and policy.

Returning to the income poverty headcount ratio, it seems that the desirability of certain properties is balanced against ease of communication. For measures whose purpose is to incite public action, the choice to favour communication is comprehensible. Indeed, the development of the Human Development Index, as Sen describes it, was largely driven by this need of communicability. Sen recounts how Mahbub ul Haq, the director of the then newly created Human Development Report Office of the United Nations Development Programme, called for an index ‘of the same level of vulgarity as the GNP—just one number—but a measure that is not as blind to social aspects of human lives as the GNP is' (Sen 1999b).

The ‘higher' or coordinating normative reasoning creates a ‘comprehensive' description of possible measures according to the criteria, rules out options that are strictly worse than others, and identifies their relative strengths and challenges.¹⁶ Even if, as is likely, the final parameters used for a multidimensional poverty measure are but one subset of multiple plausible measures, each of which is defensible and cannot be further ranked, the criteria will still have worked to eliminate measures that may have been less comprehensible and violated more key properties—or had higher measurement error, lower robustness, and less policy salience than the ones that remain. They will also have identified the strengths and challenges of each candidate, and so the selection among them is essentially also a selection of which criteria to prioritize—a choice that will have been simplified by a clear analysis. For example, a society may wish to prioritize a measure that has legitimacy because it transparently draws on public consultations, which are important because recent history had discredited poverty statistics (so prioritizing criteria 2), or a measure that will incentivize policies because it is closely tied to a popular national plan (criteria 3), and so on.

In sum, multidimensional poverty measurement can seem rather bewildering at first because its justification may draw on axiomatic, statistical, ethical, data-related, deliberative/participatory, policy-oriented, political, and historical features. But in practice, poverty measurement is considerably more concrete (Anand and Sen 1997; Alkire 2002a). The available resources and actual constraints—from timing to data to funding to political demand—for a given exercise often provide considerable structure and limit the degrees of freedom considerably. Thus, although normative engagement is required, ‘there is no general impossibility here of making reasoned choices over combinations of diverse objects' (Sen 2009: 241).

¹⁶ This use of plural principles to identify that set of options ‘than which nothing is better' is described in Sen (1985, 1997b, 2009) and Alkire (2002a: ch. 3.4).

6.3