Issues in Indicator Design

All data sources, however rich, impose constraints on a poverty measurement and analysis exercise. These are navigated via a number of important decisions that are made when designing a multidimensional poverty measure and choosing its component indicators.

7.2.1 UNIT-LEVEL INDICATOR ACCURACY

A first essential and distinctive criterion for counting-based multidimensional poverty measures is that each indicator must be a relatively accurate reflection of the achieve­ments enjoyed by each unit of identification across the relevant period and not simply the achievements enjoyed ‘on average' by some group, as we will clarify in this section.

The unit of identification refers to the entity who is identified as poor or non-poor (determination of the poverty status)—usually the individual or the household. In the literature of poverty measurement, this is typically referred to as the unit of analysis. However, the term unit of identification is more accurate because the identification may take place at a more aggregated level, but the analysis can still be performed at a more disaggregated level. For example, if the unit of identification is the household, analyses can still generate the percentage of the population who are poor.

Household surveys are usually designed to create indicators that are representative of the achievements and/or distributions of some population subgroups, such as states or ethnic groups, at the time of the survey. For example, there are indicators typically collected which have short reference periods in order to increase indicator precision and are judged to be accurate ‘on average' (Deaton and Kozel 2005).

Multidimensional measures require the joint distribution of deprivations to be accurate on average. This is straightforward to achieve if, as in the case of targeting, each indicator accurately identifies each person's deprivations, and so the deprivation scores, which reflect joint deprivations and will determine whether or not the person is identified as poor, are similarly accurate at the individual level. Selected indicators ideally balance indicator precision and unit-level accuracy in order to justify the assumption that responses reflect individual or household achievement or deprivation status during the relevant period. A related requirement, which occurs when multidimensional poverty measures (and, incidentally, other measures) are used to track trends in poverty over time, is that the indicators should ideally reflect individual achievement levels across the relevant period so that the comparisons are not unduly distorted by seasonal effects or short-term shocks.

Many indicators are relatively unproblematic in this regard.

That being said, in the case of other indicators, particularly those with short reference periods, the ideal may be more difficult to obtain. In practice, when collecting primary data, the unit-level accuracy issue should be considered when creating the questionnaire. When using secondary data, unit-level accuracy may guide the choice between different indicators, when such an option exists.

7.2.2 INDICATOR TRANSFORMATION TO MATCH UNIT

OF IDENTIFICATION

Alkire-Foster multidimensional poverty measures reflect the joint distribution of deprivations for a given unit of identification. While the advantages of this and related approaches have been much discussed in Chapter 3, its empirical implementation actually requires novel techniques. In particular, relevant data may be available for individuals, for the household, and, perhaps, also at the community level. But it is necessary to transform all indicators such that they reflect deprivations of just one (the chosen) unit of identification. Consideration of the unit-level profile of joint deprivations leads to its identification as poor or non-poor (based on the poverty cutoff and deprivation score) and hence to the inclusion or censoring of its deprivations in the resultant multidimensional poverty measure.

To begin with an elementary case, consider a child poverty measure covering children aged 6-12. Suppose that there are data on children's education, health, and nutrition; household-level data on income and housing; and village-level data on the quality of primary school facilities. In the n ? d achievement matrix, each child will naturally have their own achievement levels in health, education, and nutrition.

7.2.3 THE UNIT OF IDENTIFICATION AND APPLICABLE POPULATION

We begin with an important definition. The applicable population of a certain achieve­ment refers to the group of people for which such an achievement is relevant; namely, it can be measured and has been effectively measured, in this case, to inform poverty measurement. Note that both conditions need to hold for the population group to be applicable.^[208] In some cases, the achievement is conceptually applicable to the whole population (with appropriate adjustments in the levels considered adequate by age group or gender), but, despite this, data is typically not collected at the individual level. An example of this is the case of anthropometric indicators (nutritional indicators). While there are anthropometric indicators for all age groups and genders, these are often collected only for children under 5 years of age and for women of reproductive age. Thus for those anthropometric indicators, the remaining population groups are non-applicable because of data constraints.

The choice of the unit of identification and the treatment of non-applicable populations are often constrained by data availability. In many cases, the ideal unit of identification would be the individual; another commonly used unit is the household. In this case, the household is judged to be poor or not, and all its members are then identified as poor. ‘The standard apparatus of welfare economics and welfare measurement concerns the wellbeing of individuals. Nevertheless, a good deal of data have to be gathered from households..1 (Deaton 1997:23). Both units have advantages, as discussed in Chapter 6.

If the unit of identification of the poor is the individual, then all the considered achievements need to be available at the individual level, and thus all the achievements need to be applicable to the whole population for which the poverty measure is defined. If the unit of identification is the household, then one data point on common achievements like housing and heating may be applied to all household members. But not all the considered achievements are equally applicable to the whole population—income being a case in point (see below, this section). In these cases, certain (explicit) assumptions need to be made regarding the sharing and impact of the achievements of certain household members with respect to the others. Other achievements are individual—like health status or educational level—although they may affect household members also.

Strictly speaking, few household-level indicators are applicable to each household member. Most vary by age and some by gender. Housing conditions are an example of indicators that satisfy the ‘universality' of applicable population.

Another universally relevant and applicable indicator for all people, now at the individual level, is nutritional status. In this case, one will need to combine at least two indicators: one for the nutritional status of children under 5 years of age (which can be weight-for-age, weight-for-height, or height-for-weight) and another for the nutritional status of adults, typically the Body Mass Index (BMI).^[209]

Food consumption is another universal indicator. Although nutritional requirements vary significantly by age and gender, it maybe possible to define the relevant consumption range for each type of individual.^[210] However, data availability usually poses a limitation: ‘Household surveys nearly always collect data on household consumption (or purchases), not on individual consumption, and so cannot give us direct information about who gets what' (Deaton 1997: 205). Thus, even though conceptually food is a private consumption good, applicable to each member of the population, in practice, it is often assumed that total consumption is distributed within the household in proportion to the nutritional requirements of each member. This assumption may or may not be accurate.

Income is not a universal indicator, as it is inapplicable to people who lack income from non-labour sources like financial assets and do not earn an income (for example, babies and children, housewives, non-working students, and some elderly members) or are unemployed. Standard income poverty measures aggregate all forms of household members' income (labour or capital) and divide by the total household size (including the members who do not earn an income) to obtain the household per capita income. Equivalence scales may be applied to obtain the household adult equivalent income. The per capita or adult equivalent income is compared to the poverty line, and the household is identified as poor or not and thus all its members. Here, as in the case of consumption, assumptions regarding equal or proportional sharing have been made.

Some other relevant indicators for poverty measurement which are inapplicable for certain groups include child vaccination (adults and children older than the relevant age group do not qualify), child mortality (people who have not had children do not qualify), and employment status. Even education as measured, for example, by years of schooling is not universally applicable, as children below the official mandatory age for starting school do not qualify.

Given that some achievements relevant for poverty measurement are either concep­tually or empirically applicable only for certain population groups, the selection and definitions of indicators may take one of the following three options, depending on the purpose of the measure. They may also work as complement measures. The options are:^[211]

(a) to restrict consideration to universally applicable achievements;

(b) to construct group-specific poverty measures;

(c) to combine achievements that are not universally applicable and test assumptions regarding intra-household distribution and/or impact.

(a) Universal measures

One option is to include only ‘universal' achievements, that is, achievements which are applicable to the whole population. This approach narrows the set of possible indicators, although it will include housing, consumption, nutrition, and access to services, if data permit.

(b) Group-specific measures

A second option is to construct different poverty measures for each relevant group, such as child poverty measures or measures of female or elder poverty. This is an attractive option and suited to group-based policies, such as for children or women or elders.

However, three issues need to be considered. First, discriminating by groups may lessen but not eliminate applicability issues. For example, the relevant nutritional indicators change at the age of 5, whereas schooling becomes mandatory typically at the age of 6 or 7, and vaccination ages vary. Thus, no group is perfectly homogeneous and indicators may need to be adjusted within population subgroups. Second, it may be critically important to have detailed poverty analyses by population subgroups and these may inform group-based policies profoundly. Yet ‘general' measures may also be required to track national poverty or to target households. Finally, using unlinked poverty measures for different population subgroups may miss the overlaps of disadvantaged groups and fail to fully exploit possible synergies in policy design.

(c) Combined measures

The third option is to use achievements drawn from a subset of household members (or other unit of identification) and make explicit assumptions about the distribution of such achievements and potential positive or negative intra-household externalities.

This has been the route taken when constructing the global Multidimensional Poverty Index (MPI) (Alkire and Santos 2010, 2014). Assume, for example, that one wants to include nutrition but information was only collected for children under 5 and women of reproductive age. Given these data constraints, a household nutrition deprivation indicator might be defined as ‘having at least one child or woman undernourished’. Hence, any household member is considered deprived in nutrition if—despite being well nourished herself—any child or woman is undernourished in her household. Similarly, if one wants to include an indicator of child mortality in the poverty measure, one way to proceed is to consider all household members as deprived if even one child in the household has died. Analogously, one can consider all household members as deprived if there is at least a child of school age who is not attending school. In all of the three examples above there is an obvious assumption of a negative intra-household externality produced by the presence of an undernourished person, the experience of a child’s death, or a child being out of school. Indicators based on the assumption of a positive intra-household externality are also possible. For example, in the global MPI all household members are considered non-deprived if at least one person has five years of schooling, assuming that there is interaction and mutual sharing of cognitive skills within the household as suggested by Basu and Foster (1998).

Two practical situations must be addressed in order to compute combined measures. First, there are households where not even one person qualifies for the achievement under consideration. For example, a household may not have any children in the relevant age bracket. What indicator is used to define deprivation in these households? There is no perfect procedure. Dropping these households from the sample would bias the estimates, as the inapplicability of a certain achievement to a particular population subgroup is not a random issue and households without that population subgroup would be systematically excluded.^[212] Dropping any indicator for which no applicable person is present in a household and reweighting the remaining indicators would violate dimensional breakdown and compromise comparability across people. A viable option is to consider all households lacking the relevant data as non-deprived (or deprived) in that particular indicator, then scrutinize this assumption. For example, a household with no children of school age cannot be deprived in child school attendance, so they could be considered non-deprived in this indicator.

A second situation occurs when the survey has not collected information from all applicable members. The example of nutrition above illustrated this case. Nutrition is applicable to all household members, yet some surveys only collect information on children and women of reproductive age, for example. So households that do not have children or women of reproductive age lack data altogether. As above, all such households may be considered as non-deprived (or deprived) in that indicator. Clearly, considering them as non-deprived is a heroic assumption, as we simply do not know because information is not available. But it could be seen as a ‘conservative' approach (in the same spirit of the ‘presumption of innocence' principle in legal arenas), and it will lead to a ‘lower bound' poverty estimate. That is, it will offer the minimum possible estimate of the proportion of people in households with an undernourished member, which could be improved later.

Naturally the assumptions used in measurement design should be examined empir­ically as well as debated normatively. Special studies of omitted populations (such as the elderly), including qualitative studies, should be considered to enrich measurement design and analysis, and results should be compared with group-based results to cross-check conclusions.

7.2.4 ASSESSING COMBINED MEASURES

Most multidimensional poverty measures are combined measures, hence some assump­tions are made to permit analysis. This section describes how to assess key assumptions. Two particular kinds of analysis may be especially relevant. The first is related to the household composition effect; the second is related to the prevalence of the indicators.

First, by including indicators referring to achievements that are specific to certain groups (such as school attendance), the composition of the household will affect the probability that the household will be identified as poor. Taken to the extreme, if all the indicators in the poverty measure refer to a deprivation that can only occur among children, then clearly households without children would never be identified as poor. Obviously, a balance regarding the relevant populations for the considered achievements is required. The potential effect of household composition need not prevent the inclusion of an indicator if its importance is normatively clear. For example, it may be the case that there is a national concern about child nutrition. Inclusion of such indicators can be made, provided (a) not all of the indicators in the poverty measure refer only to a particular specific group (if so, then a group-specific poverty measure is a better alternative); (b) the specific group for which the achievement is relevant is big enough so that a known and significant proportion of households have at least one member for whom the achievement is relevant; and (c) an empirical assessment of the impact of household composition is performed to test the sensitivity of the measure to its specification.

For example, Alkire and Santos (2014) present two assessments of the influence of household composition on the probability of being poor. The first are hypothesis tests of differences in means. In each country they test whether MPI-poor households have a significantly different average size, average number of children under 5, number of females, number of members 50 years and older, prevalence of female-headed households, and proportion of school-aged children, compared to non-poor households. The second analysis decomposes each country's MPI by age and gender and compares the rankings, correlations, and the proportion of robust pairwise country comparisons across subgroup MPIs.

Regarding requirement (b), some deprivations are intrinsically important yet are either rare events or pertain to population subgroups that are very small, and thus the presence of a household member for whom the deprivation is relevant is quite unlikely. Under such circumstances, it is best to keep this indicator separate from the multidimensional poverty measure. Examples of such indicators vary by context but might include prevalence of a rare condition that affects pregnant women.

7.2.5 HANDLING MISSING VALUES

The construction of household variables, for example, from data pertaining to a subset of household members described above is completely separate from the need to address missing values, to which we now turn.

Missing values are particular cases for which a variable that is collected by the survey is not available. For example, if there is a woman of reproductive age for whom the information on BMI should have been collected, given the survey design, but for whom this information is incidentally not available, that is a case of a missing value. Missing values require attention in all poverty measures; multidimensional poverty measures have the advantage of using fewer variables than many monetary measures, but the treatment of missing values also differs slightly.

There are essentially two ways of dealing with missing values. One is to drop that observation from the entire sample. That is, if the unit of identification is the household, households with a missing value in any indicator for a multidimensional measure are dropped from the sample.

The other option is to create a rule that may assign a value for the missing data, particularly in a combined measure in which, for example, data are missing only for some individuals for whom the indicator is applicable. For example, in the case of the global MPI, if at least one member has five or more years of education (although other members have missing values), the household was classified as non-deprived. If there was information on at least two-thirds of household members, each having less than five years of education, the household was classified as deprived; otherwise it was dropped from the sample (Alkire and Santos 2014).

If the observations with missing values systematically differ from those with observed values, the reduction in the sample leads to biased estimates. To assess whether the sample reduction creates biased estimates, the group with missing values is compared to the rest, using the indicators for which values are present for both groups. If the two groups are not (statistically) significantly different, then one may proceed with an estimation using the reduced sample. If they are (statistically) significantly different, then one may still use the reduced sample but should explicitly signal whether the poverty estimate is likely to be a ‘lower' or an ‘upper' bound, based on the results of the bias analysis (Alkire and Santos 2014).^[213]

One might consider whether to use imputation to assign the observation with a missing value an estimated value of the indicator under consideration. This is commonly done in income poverty measurement, but further research is required before it is applied multidimensionally. Imputation techniques entail using the observations for which there is information to estimate a model with the achievement under consideration as the de­pendent variable against a set of explanatory variables. The estimated parameters are then used to predict the achievement for the cases with missing values, given their values in the explanatory variables. However, imputation techniques are not problem-free. First, the estimated model needs to be accurate. Second, in the case of multidimensional poverty measurement, the issue of missing values is multiplied because what is of interest is each person’s joint distribution of deprivations. This poses a significant challenge for imputation. One could specify a different model for each indicator. However, when that option is taken, it is likely to incur endogeneity problems. Moreover, this option is blind to varying profiles of joint deprivations. The more accurate route to take would be to specify a model that could predict a vector of deprivations. This imputation would also have to be per­formed such that it was accurate for each unit of identification, not merely on average. But this has not been done yet. Third, and connected to the previous section, it is also worth noting that imputation techniques cannot solve the problem of non-applicable popula­tions as there are no observations that can be used to estimate a model. For example, one cannot impute the BMI of elderly people using a survey that collects BMI information only for women of reproductive age. As the field of multidimensional poverty measure­ment progresses, appropriate imputation techniques may be developed for this context.

7.3