Journal of Development EffectivenessISSN: (Print) (Online) Journal homepage: https://www.tandfonline.com/loi/rjde20Improving learning by improving vision: evidencefrom two randomized controlled trials of providingvision care in ChinaXiaochen Ma, Huan Wang, Yaojiang Shi, Sean Sylvia, Lei Wang, Yiwei Qian &Scott RozelleTo cite this article: Xiaochen Ma, Huan Wang, Yaojiang Shi, Sean Sylvia, Lei Wang, Yiwei Qian& Scott Rozelle (2021): Improving learning by improving vision: evidence from two randomizedcontrolled trials of providing vision care in China, Journal of Development Effectiveness, DOI:10.1080/19439342.2021.1876139To link to this article: shed online: 26 Feb 2021.Submit your article to this journalView related articlesView Crossmark dataFull Terms & Conditions of access and use can be found ation?journalCode rjde20
JOURNAL OF DEVELOPMENT .1876139ARTICLEImproving learning by improving vision: evidence from tworandomized controlled trials of providing vision care in ChinaXiaochen Maa, Huan Wangand Scott Rozellebb, Yaojiang Shic, Sean Sylviad, Lei Wangc, Yiwei QianeaChina Center for Health Development Studies, Peking University, Beijing, China; bStanford Center on China’sEconomy and Institutions, Stanford University, Palo Alto, CA, USA; cCenter for Experimental Economics inEducation, Shaanxi Normal University, Xi’an, Shaanxi, China; dDepartment of Health Policy and Management,University of North Carolina at Chapel Hill, Chapel Hill, NC, USA; eDepartment of Economics, University of SouthernCalifornia, Los Angeles, CA, USAABSTRACTARTICLE HISTORYThis paper examines the external validity of health intervention by com paring the impacts of providing free eyeglasses on the educational per formance of nearsighted children in two settings: rural public schools inWestern China and urban private migrant schools in Eastern China. Theintervention significantly improves educational outcomes by 0.14 stan dard deviations in math in rural public schools but not in private migrantschools. The difference in measured impacts is due in part to lower qualityschooling in migrant schools in Eastern China. Our findings show that onlywhen school is providing a quality education, health interventions mightincrease student learnings.Received 17 August 2017Accepted 10 January 2021KEYWORDSField Experiment; healthIntervention; educationalEconomicsJEL CODES:C93; I12; I21; O15IntroductionResearchers have been increasingly interested in understanding demand-side constraints to studentlearning (e.g., Ganimian and Murnane 2016). In this regard, economists have begun to evaluate, as oneof the most high-profile approaches to alleviate demand-side constraints, the impacts of certain healthand nutrition interventions, such as deworming programmes (Miguel and Kremer 2004), the provisionof micronutrient supplements (Maluccio et al. 2009), and nutritious school meals (Vermeersch andKremer 2005). Despite the frequently cited positive impacts of the interventions on children’s healthand nutrition, the evidence is mixed when examining the impacts of these interventions on thelearning outcomes of students (Evans and Popova 2016; Krishnaratne et al. 2013; McEwan 2015).Mixed evidence from impact evaluations of similar interventions in differing contexts raisesa challenge for policymakers. When studies are run in a single venue or subset of schools,a common criticism is that these studies lack external validity and that the sample is not representa tive of a nation, region, or sector of an economy (Barrett and Carter 2010; Deaton 2010). As a result,many authors argue for the specificity of the study context. Some argue that the same type ofintervention can elicit different results in different contexts due to the strong role of complementa rities in specific education and health systems (Culter and Lleras-Muney 2014; Glewwe 2013) and theneed for entire systems to improve. Interventions that had large effects in one place may end upproducing no or, at the most, small improvements in others if the overall system in the setting of thecurrent intervention is broken (Acemoglu 2010).CONTACT Yaojiang [email protected], Xi’an, Shaanxi, China 2021 Informa UK Limited, trading as Taylor & Francis GroupCenter for Experimental Economics in Education, Shaanxi Normal
2X. MA ET AL.In response to the criticism that impact evaluations lack external validity (Pritchett and Sandefur2015a, 2015b), new studies have begun to focus on addressing the issue of how the results of theinterventions that are being tested are applicable to populations in other contexts. As a part of thismovement, there is a small and growing empirical literature that seeks to replicate similar interven tions in different contexts. Bauchet et al. (2015) study a replication of an anti-poverty program inSouth India and find no net impact even though significant effects were found in Bangladesh andother sites due to differences in the nature of local labour markets. In contrast, Banerjee et al. (2015)reviewed six randomised controlled trials (RCTs) to examine the impact of a similar anti-povertyprogram and found that impacts were not sensitive to context.Nevertheless, studies that reference the external context of the intervention are more theexception than the rule. After reviewing 92 RCTs published between 2009 and 2014 in leadingeconomics journals, Peters et al. (2016) concluded that most papers that used RCT methodologiesdid not discuss the issue of external validity and did not provide the information that was necessaryto assess potential problems in upscaling to other localities.The goals of this paper are twofold. First, we measure whether there is a causal impact of visioncare of nearsighted students on their educational performance in two different primary schoolsettings. Second, we explain why such a program might have different impacts in different schoolcontexts. In other words, we present evidence about whether a vision care program works as well asevidence about where or when such a vision care program might or might not work.To achieve our goals, the paper has four main sections. In the first section, we describe the twoRCT research settings – one setting in which the public primary schools in a rural area of westernprovinces generally deliver quality education (Western China program) and the other setting inwhich the private migrant primary schools in migrant communities of China’s coastal urban areadeliver poor, low-quality education (Eastern China program). In the second section, we describe thedata and analytical approach in which we intervene with a single, standardised vision care treatment.Briefly, in 2012, in a set of 84 rural primary public schools, we implemented an RCT in which we gavestudents in the 42 treatment schools high-quality vision screenings and refraction, produced glasses,and dispensed them without charge. In a randomly assigned set of 42 control schools, we did notgive the students free glasses. In 2013, the same treatment that used the same set of protocols wasreplicated in a second setting, a set of 94 private migrant primary schools (47 treatment and 47control schools) by the same research team.In the third section of the paper, we report the results of these two RCTs, which are implementedin the two different contexts. We found that 20% of the students in the Western China program and17% in the Eastern China program were nearsighted. Among those with poor vision at baseline,however, only 18% in both study regions were using eyeglasses to correct their vision. In theintention-to-treat (ITT) analysis in the Western China program, providing free eyeglasses and trainingimproved maths test scores by 0.14 standard deviations relative to students in control schools. Incontrast, there was no significant impact of the treatment in the Eastern China program.In the fourth section, we examine three hypotheses that concern why the impact of our provisionof free eyeglasses differed between the Western China program and the Eastern China programschools. Differences in compliance rates (Hypothesis 1) cannot account for the differences observedin treatment effects. In fact, the share of students who actually wore the glasses they received in theEastern China program was significantly higher than that in the Western China program. In addition,we used our data to show that there are no systematic differences in the (at least) observablecharacteristics of individuals and their families who favour educational performance (Hypothesis 2).After examining differences in the schools in the two settings (that is, the differences in public ruralschools and private migrant schools), we show that the differences in measured impacts across thetwo regions are mainly a matter of the difference in the characteristics of the teachers and schools inthe two samples (Hypothesis 3). Overall, our results suggest that vision care programs that areimplemented in schools in which teachers are in an environment that is committed to providingquality education, the programs can have a significant impact on learning. Even more generally, our
JOURNAL OF DEVELOPMENT EFFECTIVENESS3study, in which we replicate an RCT in two regions of China with two different schooling environ ments, clearly demonstrates that context matters.Research settingAverage levels of income in the sample regionsThe first program (Western China) was conducted in rural areas of two provinces in Western China:Gansu and Shaanxi. Gansu’s GDP per capita of USD 3,976 was ranked the third poorest amongChina’s 31 provincial administrative regions, while Shaanxi’s GDP per capita of USD 6,108 was ranked13th and was similar to that for the country as a whole (USD 6,969) (China National Bureau ofStatistics 2008–2014). Despite the higher ranking, Shaanxi still houses a sizeable number of house holds that live in poverty. Shaanxi has one of the highest numbers of nationally designated poorcounties.1The second program (Eastern China) was conducted in migrant communities in Jiangsu Provinceand Shanghai Municipality, two of the richest areas in China. In 2013, Shanghai’s GDP per capita wasUSD 15,696, and Jiangsu’s GDP per capita was USD 12,155.Provincial yearbooks, however, provide only average income per capita for an entire province. Inaddition, most of the information in provincial yearbooks concerns the measurement of attributes ofthose individuals who are formal residents (and not temporary migrants) of the province. Hence,average income figures at the provincial level can disguise the relative poverty of the families in oursample.Indeed, most of the areas in our sample are dominated by poor households. In Gansu andShaanxi, the students in most of our schools are from poor rural families. Of the 18 counties in theWestern China program sample, 15 are nationally designated poor counties. All of our sampleschools are in rural communities. Thus, the average income of most families in our sample is nearthe poverty line.Despite living in one of the wealthiest regions in China, the households in the Eastern Chinaprogram are in many ways more similar to the surveyed families in our Western China sample areas.All students in the Eastern China program region are from families who live in migrant communities.These migrant families are part of a large movement of people from rural to urban areas that hasincreased dramatically over the past three decades (China National Development and ReformCommission 2015).Although labour has flowed relatively freely from agriculture to industry, the process of shiftinglives, homes, and families has been more difficult. A core aspect of the challenges that migrants facestems from China’s hukou household registration system, which classifies China’s citizens as eitherrural or urban residents. Without an urban hukou, migrants and their families have limited access tourban public services, including housing, health care, social security, and, above all, education fortheir children. As a result, as Li et al. (2014) argue, although individuals in these migrant communitieslive in some of the wealthiest regions of China, the families and the schools that serve them moreclosely resemble communities, families, and schools in underdeveloped rural areas. As discussed inthe subsection below, the data show that children and families who live in migrant communities aresimilar in many observed characteristics to those who live in rural communities.Schools in the sample regionsIn China today, public schools in both rural and urban areas are required to provide free education tochildren, according to the compulsory education law. This free education, however, is guaranteedonly for children whose hukou matches the school’s location. In the case of migrants, if there are notenough slots in urban public schools, the children of migrant families have no choice but to attendprivate migrant schools.
4X. MA ET AL.In rural areas of Western China, nearly all children attend the nearest public primary school, whichis typically located in their township’s urban centre (or a nearby large village). Although rural schoolsare located in the parts of China that are poorer and more remote, they are supported by publicfunds and are subject to monitoring by national and provincial government/education officials. Allrural public schools also are under the direct management and supervision of county-level bureausof education.Due to the combined input of upper-level governments (province, prefecture, and county) andthe national government, the basic ingredients of a sound school system are present in most ruralpublic schools. The salaries of teachers have been paid by the national government since 2005 andwere raised to levels of the incomes of civil servants in 2009 (An, 2018). The curriculum for rural