Immunize Thy Children—Paving the Way for Their Success: Micro-level Evidence from India

3.2.1 Measuring the Treatment Variable: ‘Childhood Immunization Status’

We calculate immunization status from a baseline survey (IHDS-I) based on information on doses received by children. The age cohort in this study consists of children between one and less than two years old. Because as per international and Government of India guidelines, children should be fully vaccinated within the first year of birth. IHDS-I has collected data on immunization from the immunization card for each child born since January 2000 or on the mother’s report in case of non-availability of the card.

Immunization status is calculated following the completion of six vaccine-preventable diseases, viz., tuberculosis, diphtheria, pertussis (whooping cough), tetanus, poliomyelitis and measles are free of cost to all.

Children who received BCG, measles, three doses of DPT and three doses of polio (excluding Polio 0, which is administered at the time of birth) are fully vaccinated or fully immunized (GoI, 2007). ‘A child is said to be fully immunised if child receives all due vaccine as per national immunisation schedule within 1st year age of child, mentioned at National Health Mission Website’. ² If the outcome variable is immunized, we coded it as ‘1’. The outcome is coded as ‘0’ if a child is not fully immunized. We have clubbed the ‘partially immunized’ category and ‘never immunized’ category together because both cases are vulnerable to the disease for which the dose is missed. Moreover, as per the UIP, Government of India, the effect of routine immunization cannot be achieved unless the child is fully immunized.

Immunization information is only available in the survey for the last two births during the five years preceding the interview. We used children’s basic information (sex, approximate age and date of birth) from the birth history file. If a single immunization date is missing, we have excluded that case. IHDS-I has first collected the immunization information from the immunization card if available. If the card is not seen, IHDS-I has collected the information from mothers’ verbal communication on each immunization uptake with frequency.

3.2.2 Measuring Outcome Variable: Cognitive Levels

The IHDS takes cognitive achievement tests for children of age eight to eleven years. In this analysis, we have utilized the information on the level of cognitive development indicated in IHDS-II. They assess reading, mathematics and writing skills through these three tests. These tests were developed by Pratham, an Indian NGO, and are included in their nationwide survey reported in the Annual Status of Education Report (ASER) (ASER Centre, 2011). The reading, math and writing tests categorized the children into five, four and three levels depending on their performances. Table 1 describes the levels for all tests.

OPEN IN VIEWER

Table 1.

Levels of Reading, Mathematical and Writing Tests.

Reading Level	Math Level	Writing Level
Cannot read letters = 0	Cannot recognize numbers = 0	Cannot write = 0
Can read letters = 1	Recognizes numbers = 1	Can write with 1–2 mistakes = 1
Can read words = 2	Can subtract = 2	No mistake = 2
Can read paragraph = 3	Can divide = 3
Can read story = 4

In this paper, we have used levels of reading, writing and math tests, as given in IHDS-II, as the indicators of cognitive development. Therefore, we consider three test results for each child for her cognitive performance. We assess the impact of immunization on these cognitive skills—reading comprehension, math and writing separately.

Based on IHDS-I, there is a significant variation across the Indian states regarding uptake of complete immunization. Figure 1 depicts that wide variation. There is a good body of literature (Erchick et al., 2022; Francis et al., 2021; Gurnani et al., 2021; Heidi et al., 2014; Priya et al., 2020; Sissoko et al., 2014) which talks about the different reasons for large-scale variation of completion of routine immunization in India under UIP. Among major reasons, first is the vaccine hesitancy, which varies to a large extent across regions and different demography; second, heterogeneous state capacity and institutional functioning; third, vaccine supply chain disruption across different regions; fourth, access and transport to the remote habitations; fifth differential state focus to the topic of child and public health.

OPEN IN VIEWER

Figure 1.

Rate of Full Immunization Across Indian States.

In Appendix 1, we present the pan-India comparison between the state-wise rate of immunization completion status and the status of performance of different cognitive achievements. It would be interesting to check whether such variation in immunization completion rate can explain the variation in cognitive ability. More precisely, it would be worth investigating how the variation in immunization status can explain far variation in cognitive achievements. To do that, we need to undertake multivariate or regression-based confirmatory analysis. One proposed baseline specification of our multivariate analysis could as follows:

In Equation (1), y i v t 1 is the dependent variable which is the cognitive score of child i, in village v , in 2011–2012 (considered as period t 1 ), x i v t 0 is the primary independent variable which is the immunization status of child i, in village v , in 2004–2005 (considered as period t 0 ), S e x i is the sex of the child i, M E i t is the maternal endowment (age and education of the mother) of child i, at time t and hh_it is the household-specific variables: caste, religion and number of rooms in the household, water source for drinking, type of toilet, access to electricity, income, ' v i l l ' is the village-level covariates to control for village-level heterogeneity ³ of infrastructure that will affect both the health (immunization) and education (cognitive scores) of the children, ' s t a t e ' is the state fixed effect to control for within state heterogeneity, and ε i is the unobserved heterogeneity or idiosyncratic error term.

One potential econometric challenge would be this unobserved heterogeneity, which could simultaneously influence y i v t 1 and x i v t 0 and thereby making C o v x i v t 0 , ε i ≠ 0 which would typically cause the endogeneity problem. Factors such as parental motivation or awareness can influence the decision of immunization ( x i v t 0 ) of the child as well as other factors (e.g., the choice of nutritional food, home-based learning facility, choice of schools, etc.) which would have a direct impact on cognitive test score ( y i v t 1 ) of the child. In the said specification, we try to address this issue by controlling maternal endowments and household-specific controls, but unobserved heterogeneity remains. As discussed later, we try to address this unobserved heterogeneity using the propensity score matching (PSM) exercise. However, first, we start with fixed-effects ordered logistic regression. Such methods of confirmatory analysis are outlined in the following sections.

First, we analysed the effect of childhood immunization on cognitive skills using fixed-effects ordered logistic regression (Baetschmann et al., 2020; Cyrenne & Chan, 2022). The outcome variables in our study are reading comprehension level, math level and writing level. These variables have more than two categories and have a meaningful sequential order. The treatment (that is, independent) variable may impact three aptitudes differently. Keeping the above logic in mind, village fixed-effects ordinal or ordered logistic regressions are run using the panel dimension of the data, separately for three tests: reading, maths and writing. Here, the outcome variable is the level of reading, math or writing skill. The leading independent variable of interest is a dummy (0 or 1) which captures the child’s immunization status. Other control variables are the child’s sex, maternal endowment (age, education), household-specific variables: caste, religion, number of rooms in the household, the water source for drinking, type of toilet, access to electricity and income. In this model, the panel identifier has to be a group viable and in our case that group is the village (Baetschmann et al., 2020). However, this village fixed-effects ordered logistic regression is susceptible to those said econometrics issues, namely, unobserved heterogeneity at the parental level and endogeneity bias. Hence, we propose PSM (as used by Bloom et al., 2012) as an alternative regression technique approach to address those said econometric challenges. PSM is outlined next.

3.2.4 Propensity Score Matching

To trace the impact of immunization on the kids’ test scores, one must ensure the validity of the following three identification assumptions related to treatment effect estimation. Otherwise, in the absence of the validity of these assumptions, one could face unobserved heterogeneity and possible endogeneity problems, and, hence, simple regression would result in a biased estimate of the impact estimation of immunization on cognitive ability.

We can find individuals belonging to the treatment and the control group for all values of the covariate Xi. This assumption is important for all methods which rely on conditional un-confoundedness. It can be checked in practice by computing the probability of being in the treatment given X, which we call the ‘propensity score’.

The perfect validity of assumption 1 is possible only in a controlled experimental setup, which is possible under a randomized control trial set-up. However, in our context, as we are trying to estimate the impact of immunization on cognitive achievement in terms of a test score (unlike the health outcome), contamination of effect from the treated to the control group is not possible. Essentially, we are ruling out the possibility of any external effect or equilibrium effect from the control to the treated child.

Un-confoundedness assumes that selection into treatment conditional on some characteristics X is independent of potential outcomes. That is, the factors or covariates that could influence the treatment assignment (i.e., the decision to take immunization) will not affect the treatment outcome (i.e., the test score). However, factors such as parents’ education, family background of the child, urban proximity, etc., can affect the decision to take immunization and can directly affect school performance, including test scores. This un-confoundedness is a necessary assumption if we use the difference-in-differences, regression discontinuity or instrumental variables techniques for impact evaluation. As our principal method would be PSM (which we discuss below), un-confoundedness would not be an issue unless we rely on OLS.

Assumption 3 is subject to propensity score estimation, essentially the ex-ante calculation of the probability of getting treated for the actual treated and control group based on the observed covariates. We use data on household characteristics taken during IHDS-I as the baseline in 2004–2005. Data on test scores come from tests administered as part of the 2011–2012 follow-up survey of the children.

As a child’s basic immunization results from conscious decisions taken by parents, it is a bit complicated to estimate the effect of immunization on cognitive performance. The difference in cognitive abilities between an immunized and non-immunized child may not be purely due to the child’s vaccination history. It may be partially due to family characteristics associated with immunization decisions. Therefore, simply estimating the impact of immunization on the cognitive outcome may violate the assumption of un-confoundedness. This causal effect can only be established when children are randomly assigned to the control group (here, non-immunized) and treatment group (here, immunized children). In this observational study, children were not randomly immunized. Immunization was the parent’s choice. In this case, if we use simple OLS, which cannot tackle the violation of un-confoundedness, it would not be easy to separate the effect of immunization and the effect of other family attributes (Bloom et al., 2012).

In this case, if there is a difference in cognitive outcomes between fully immunized and not immunized children, we cannot separate the causal effect of immunization and other family attributes. Estimation of the effect of immunization on cognitive development may be biased owing to the existence of other confounding factors. Because children who were fully immunized may belong to totally different social strata compared to non-immunized children. In other words, there can be some differences between the control and treatment groups in terms of other covariates that may affect the outcome. So, this selection bias can overestimate the effect of immunization on children’s developmental outcomes.

Difference-in-difference (DID) is a popular method in this scenario. It takes care of the selection bias stemming from unobserved time-invariant factors. DID estimates the treatment’s average effect by comparing the programme’s ex-ante and ex-post outcomes between treated and control groups. However, in our analysis, the post-programme (read post-Immunization) outcome variable (read cognitive outcome here) is not available for the pre-programme time. So, standard DID is not possible in our study.

To resolve this problem, we can employ PSM, which matches a vaccinated child with a non-vaccinated child with similar observed attributes. This approach was first used by Rosenbaum and Rubin (1983). In this method, we will compare treatment and control groups that are almost similar in all visible characteristics. So, the difference in outcome in the control and treatment groups can be attributed to a causal effect.

Now, this is not feasible to match both groups based on all sorts of observable dimensions. So, this method prepares an index for everyone based on as many dimensions as possible. The matching method does not eliminate the bias caused by confounding factors. This technique only reduces the bias. DIDs can be argued as a better matching technique. However, we cannot use that method as baseline data for outcome variables (read ‘cognitive scores’ here) is unavailable.

Under PSM, each immunized child is matched with another non-immunized child who shares commonly observed characteristics. So, this match creates a control group (non-immunized children) vis- à -vis a treatment group (immunized children) with similar background features. When a pair of matched immunized and non-immunized children has similar visible features, the difference in cognitive outcome can be attributed to the causal effect of the difference in immunization (Bloom et al., 2012). Jalan and Ravallion (2003) have used this method to analyse if piped water connection at home has a causal effect on the occurrence of diarrhoea in children. For example, this approach has been used to determine whether piped water to homes reduces childhood diarrhoea. We assume that once we match all observable characteristics’ the assignment to the treatment or control group is random. Therefore, we can compare average outcomes for treated children with matched controls.

It is not easy to get an exact matching of each immunized child for each control child. Matching based on all possible observable dimensions is not feasible. According to Rosenbaum and Rubin (1983), it is sufficient to match on ‘propensity score’ rather than finding a match on each characteristic. ‘Propensity score’ is the estimated probability of success given the observable characteristics. The propensity score here refers to the probability of getting fully immunized. It is derived from probit analysis using variables related to child and household characteristics during IHDS-I. Children are matched based on the propensity score rather than individual features. As matched children have a similar background, the treatment and control groups can be treated as if randomly selected.

If matched children do not have a common background, then causality between immunization and the cognitive outcome cannot be established. If the characteristics do not overlap and matches are not found, then the causality cannot be identified. To avoid this problem, we take a common support zone where all immunized and non-immunized are matched based on the propensity score. We dropped all the children who were not matched and remained outside the common support zone. So, this method will only be applicable when ‘reasonably close matching children’ can be found (Bloom et al., 2012). The average treatment effect is calculated in Stata as described in Becker and Ichino (2002).

This is further to note that PSM is very much based on the observed characteristics of both treated and control households from where the children are coming. Using PSM, the problem of un-confoundedness can be addressed to a large extent but certainly not to the full extent. Because there can be some unobserved (time-varying and time-in-varying) covariates which we commonly call the unobserved heterogeneity, can influence the treatment status that is the decision to immunize. If unobservable factors are critical for the model, the bias will persist, which would be a potential caveat of this estimation. However, our village fixed-effects ordered logistic model tries to control for time-in-varying unobserved heterogeneity at the village level.

3.2.5 Matching Variables

Based on existing literature and studies, we have selected several pre-intervention characteristics that significantly impact immunization. Our PSM design is motivated by Bloom et al. (2012), but we did not use exact covariates to derive our propensity score. Bloom et al. (2012) found a positive relationship between socioeconomic status, access to a toilet, number of rooms in the household, fewer children at home and more educated and healthy mothers having a higher probability of being vaccinated in the Philippines. In our case, the variables used ⁴ for matching are state of residence, area of residence (urban/rural), sex of the child, highest education of the household head, religion and caste of the household, child’s mother’s age, family income, birth order, whether the household has drinking water within the house if the house has electricity connection.

In the propensity score model, while finding determinants of immunization uptake, we have classified all the states into seven categories: (i) Southern zone: Andhra Pradesh, Telangana, Tamil Nadu, Kerala, Karnataka; (ii) Upper North zone: J&K, Himachal Pradesh, Uttarakhand; (iii) North zone: Delhi, Punjab, Haryana, UP, Bihar; (iv) North-East; (v) Eastern zone: West Bengal, Orissa, Jharkhand; (vi) Central zone: Madhya Pradesh and Chhattisgarh; (vii) Western zone: Maharashtra, Gujarat and Rajasthan.

4.2.1 Results and Discussion from Fixed-effects Ordered Logistic Regression

In this sub-section, we present the results from fixed-effects ordered logistic regression and then present the results from PSM. The outcome variable in ordered logistic regression, shown in Table 5, is the level of cognitive ability, and the variable of interest is immunization status. We have tested the relationship between immunization and reading, math and writing level in three separate regressions.

OPEN IN VIEWER

Table 5.

Impact of Immunization on Cognitive Levels: Village FE-ordered Logistic Regression.

Factors	Marginal Effect: Reading Level	Marginal Effect: Math Level	Marginal Effect: Writing Level
Fully immunized: (Not/partially ref)	0.654***	0.513***	0.264*
	[3.54]	[5.25]	[1.68]
Child’s sex: Girl (boyref)	−0.019	−0.116	0.120
	[−0.14]	[−1.14]	[1.18]
Mother’s age	−0.006	0.001	−0.012
	[−0.39]	[0.06]	[−0.88]
Mother’s education (in years)	0.093***	0.075***	0.053***
	[6.88]	[3.96]	[3.23]
Father’s education (in years)	0.047**	0.058***	0.074***
	[2.57]	[3.12]	[4.35]
No. of children in the household	−0.103*	−0.108	−0.075
	[−1.72]	[−1.04]	[−1.40]
Per capita income	0.000	−0.000	−0.000
	[1.03]	[−1.64]	[−0.07]
Number of rooms in the house	0.013	0.099**	0.123***
	[0.30]	[2.20]	[2.94]
Urban locality: (Rural ref)	0.064	0.012	−0.042
	[0.51]	[0.07]	[−0.16]
Muslim (Hindu ref)	−0.381**	−0.457**	−0.096
	[−1.97]	[-2.01]	[−0.52]
Others	0.063	0.463*	0.279
	[0.17]	[1.88]	[0.88]
Other Backward Class (OBC)(Brahmin and upper casteref)	−0.106	−0.272*	−0.458***
	[−0.79]	[−1.66]	[−4.43]
Schedule Caste (SC)(Brahmin and upper casteref)	0.201	−0.150	−0.315
	[0.69]	[−0.49]	[−1.23]
Schedule Tribe (ST)(Brahmin and upper casteref)	0.002	−0.573*	−0.871***
	[0.00]	[−1.78]	[−2.62]
traditional latrine (Open toiletref)	0.245	0.053	0.080
	[1.17]	[0.22]	[0.40]
Flush toilet (Open toiletref)	0.161	0.353	−0.021
Other improved drinking water sources (Piped waterref)	0.092	−0.239	−0.383*
	[1.05]	[−1.58]	[−1.86]
Unimproved drinking water sources (Piped waterref)	−0.094	−0.544	−0.227
	[−0.37]	[−1.57]	[−0.94]
State fixed effects	Yes	Yes	Yes
Number of observations	1,304	1,299	1,283

Note: t statistics in brackets; * p < 0.10, ** p < 0.05, *** p < 0.01.

The immunization status as fully immunized as against not or partially immunized in FE-ordered logistic regression is significant for all three cognitive categories. This suggests that fully immunized (as against not or partially immunized) children are exhibiting higher cognitive scores than not fully immunized peer groups in a probabilistic sense. In other words, exposure to full immunization is associated with a higher chance of better cognitive ability.

Table 5 reports the marginal effects for all outcome levels of reading, mathematical and writing skills, but we have reported the results only for the highest scores of each test. Each level of reading, writing and math proficiency has a marginal effect derived from FE-ordered logistic regression. With binary explanatory variables, marginal effect measures how the predicted probability of outcome changes as the binary independent variable changes from 0 to 1. So, for the binary independent variable, it measures the discrete changes. Referring to Table 5, we can say that if a child is fully immunized as against partially or non-immunized, then the likelihood (or probability) of achieving the highest reading level (i.e., reading stories) will increase by 0.65, the likelihood (or probability) of achieving highest maths level (can do the divisions) will increase by 0.51 and the likelihood (or probability) of achieving highest writing level (that is writing with no mistake) will increase by 0.26. If explanatory categorical variables have more than two values, then the marginal effect shows the difference in predicted probabilities of the specific outcome for one category relative to the reference category. For example, in our study, religion has five categories: Hindu (reference category), Muslim, Christian, Sikh and Others. Marginal effects of, say, Christians, would measure how much more (or less) likely Christians are to obtain a particular outcome compared to the reference category Hindu. Mother’s and father’s education are consistently a positively significant factor for all cognitive categories. The number of children in a family is negatively related, and household income is positively related to only reading test scores.

Under PSM, we first calculate the ex-ante probability of getting treated (meaning getting fully immunized) based on the observed characteristics or variable, as stated as matching variables in Section 3. To calculate this probability, we run a probit regression with immunization status (0 or 1) as the dependent variable. The probit result (in Appendix 6) shows the household head’s education, birth order, electricity, Muslim religion and, in some zones, antenatal check-ups, are strongly associated with immunization. Interestingly, the northern and western zones are more likely to get fully immunized compared to the southern zone in India. Figure 2 reports the propensity score or the distribution of the ex-ante probability of being treated (that immunized) for the group of treated (i.e., immunized) and non-treated (i.e., partially or non-immunized). We eventually excluded the observation outside the common support zone (as shown in green in Figure 2). Now arguably, after excluding the observations outside of the common support zone, revised treatment (immunized) and revised control (partially or non-immunized) are comparable.

OPEN IN VIEWER

Figure 2.

Distribution of Propensity Score Across Treatment and Control Groups.

Average treatment effect on treated (ATT) results after matching is reported in Table 6. Our PSM-based ATT results suggest that children who were fully immunized perform significantly better in reading level (i.e., 0.30 percentage points higher), math level (that is 0.17 percentage points higher) and writing level (i.e., 0.10 percentage point higher) test.

OPEN IN VIEWER

Table 6.

Effect of Immunization on Child’s Cognitive Skills Using PSM: ATT.

Variable	Sample	Treated	Controls	Difference	SE	T-stat
Reading	ATT	2.54	2.24	0.30***	0.09	3.17
Math	ATT	1.54	1.37	0.17***	0.06	2.76
Writing	ATT	1.19	1.09	0.10**	0.05	1.96

Source: Authors’ calculation using IHDS-1 (2004−2005) and IHDS-2 (2011−2012) unit-level data.

Note: (1) ATT = average treatment effect on treated.

We have also conducted PSM with other sets of household-level variables and performed the robustness check for the same. However, the main results with all signs and significance remain the same.