News from AMERU
AMERU seminar series: There’s No Escape from External Validity – Reporting Habits ofRandomized Controlled Trials (Upcoming)
Presented by Dr Joerg Peters on the 30th of April
Abstract: Randomized Controlled Trials (RCT) are considered the gold standard in empiricalsocial sciences and have been increasingly used in recent years. While their internalvalidity is in most cases beyond discussion, RCTs still need to establish externalvalidity. External validity is the crucial determinant for a study’s policy relevanceand might be at stake because of potential general equilibrium effects, Hawthorneeffects, or representativeness problems that compromise generalizing results beyondthe studied population. For this paper, we reviewed all RCTs published in leadingeconomic journals between 2009 and 2012 and scrutinized them for the way in whichthey treat external validity. Based on a set of objective indicators, we find that theRCT literature does not adequately account for potential hazards to external validity.A large part of published RCTs does not discuss potential limitations to externalvalidity or provide the information that is necessary to assess potential problems. Weconclude by calling for a more systematic approach to designing RCTs and toreporting the results.
AMERU seminar series: The Power of Tests for Pareto Efficiency Within the Family (Upcoming)
Presented by Jesse Naidoo on the 23rd of April
Abstract: Empirical studies of the allocation of consumption within the family have often failed to reject the null hypothesis of Pareto efficiency. To study the power of these tests, I construct two models of public goods allocation. One model is cooperative and leads to efficient allocations, and the other is noncooperative and leads to inefficient allocations. In the noncooperative models, variations in the economic or social circumstances outside of the household (“distribution factors”) cause variations in implicit property rights within the household. If these variations go unobserved by the econometrician, then noncooperative models imply the same restrictions on household demand as do the cooperative ones. Therefore, many tests in the literature cannot identify inefficiency when it is present. I then show that without restrictions on preferences or data on individual con- sumption, the hypothesis of Pareto efficiency is not well-separated from the noncooperative alternative. So in a nonparametric setting, there can be no tests that both reject all inefficient models and do not reject any efficient models. However, if the effective wealth of each partner is observed without error and satisfies a large support condition, and if the consumption of a public good and at least one assignable private good can be observed, then it is possible to reject efficiency.
AMERU seminar series: Assessing the impact of a Reading Catch-up Programme in primary schools. (Upcoming)
Presented by Volker Schöer on the 17th of April
Abstract: A substantial proportion of schoolchildren in South Africa are one or more years below the acceptable achievement levels, particularly in key subjects, such as English First Additional Language and Mathematics (Taylor, 2014; NEEDU, 2014; Spaull, forthcoming). Schoolchildren that are academically behind the acceptable levels of performance in the Foundation Phase are likely to fall further behind their counterparts as they progress up the school system. This study evaluates the impact of the Reading Catch-Up Programme (RCUP), an eleven-week programme, which focused on re-teaching Foundation Phase English First Additional Language skills and content to learners in underachieving primary schools to ensure that learners in these schools had an opportunity to master the basics of English-language literacy. Using a Randomized Control Trial with 40 treated and 60 control schools in the Pinetown district of Durban, Kwazulu-Natal, the study shows that even though the learners’ spelling and language scores in the treatment schools increased and their ANA scores show statistically significant relative gains compared to the control schools, the gains may have limited educational significance.
AMERU seminar series: Do young South Africans think that a targeted hiring voucher benefits them and do their perceptions align with the evidence suggested by a Randomized Control Trial?
Presented by Ms. Allegra Cockburn.
Abstract: This paper provides an analysis of the 2011 retrospective evaluations of those treated with a hiring voucher in the randomized control trial (RCT) conducted by the African Microeconomic Research Unit at the University of the Witwatersrand. It will compare these evaluations to the estimated impact of the hiring voucher in order to ascertain whether or not subjective evaluations align with experimental evidence in this instance. Overall, young, targeted South Africans viewed the hiring voucher positively but, when we disaggregate the effect of the voucher into subgroups, it becomes apparent that those most likely to benefit from the voucher were actually less likely to think of it positively. This will be established by making use of average effect of the treatment on the treated (ATT), intention to treat (ITT) and proportions within particular subgroups and by comparing the treatment effect to the probability of a positive evaluation using logit analysis. The paper discusses the reasons for and implications of the inconsistency between qualitative retrospective evaluations and RCT evidence.
Impact evaluation workshop for industrial policy
CLEAR-AA in collaboration with the United Nations Industrial Development Organisation (UNIDO) and members/ affiliates of AMERU ran the first pilot workshop in impact evaluation for industrial policy, held in Pretoria from the 23-26 March 2015. The course was attended by public officials, technical staff and participants from the South African Department of Trade and Industry (the dti), the Namibian Ministry of Industry and Trade (MIT), the South African National Treasury (NT), Department of Planning, Monitoring and Evaluation (DPME), the Gauteng Provincial Department of Economic Development (GPGDED), Training and Industrial Policy Strategies (TIPS), the German Development Agency (GIZ) and the Wits School of Economics and Business Studies.
The course aimed at introducing policy-makers to the concepts, techniques and methods of impact evaluation within the field of industrial policy. Facilitators included local and international experts such as Prof. Neil Rankin (Stellenbosch University - AMERU Affiliate), Prof. Tendai Gwatidzo (Wits University - AMERU Affiliate), Dr. Antonio Andreoni (SOAS University), Adnan Seric and Philipp Neurerburg from UNIDO, and Wayde Flowerday of AMERU. CLEAR/AMERU and UNIDO plan to continue working together in promoting the use of M&E for industrial development and will be delivering similar courses to other clients in the the SADC region.
AMERU seminar series: productivity signals and job search outcomes
Mr Martin Abel, PhD student at Harvard University and former research manager at the African JPAL Centre at SALDRU/ UCT, presents on a Randomized Control Trial (RCT) study of productivity signals and job search outcomes. Mr Abel together with Dr. Rulof Burger (RESeP, Stellenbosch University) and Dr. Patrizio Piraino (SALDRU, University of Cape Town) investigates the impact of providing young job seekers with trustworthy productivity signals in form of numeracy and English language ability test score certificates as well as employer references on their job search outcomes. The study is implement in Gauteng and JPAL research associates who are helping with the implementation of the study are housed in the AMERU during the duration of the study.
Dr. Peters joins SEBS and AMERU as a visiting professor
Dr Joerg Peters of the Rheinisch-Westfälisches Institut für Wirtschaftsforschung (RWI) in Essen (Germany) officially joins the School of Economic and Business Sciences and the AMERU as a visiting professor for a three year period. Dr Peters is a specialist in access to modern energy in developing countries and the evaluation of development interventions. His research focus is on the economics of poverty and on the effectiveness of development policy.
First IMF Country Office Internship in collaboration with the AMERU
Maria Ngarachu is the first graduate student to participate in an IMF Country Office Internship. Dr. Axel Schimmelpfennig, the Senior Resident Representitive of the IMF in South Africa, initiated the very first internship for a graduate student in an IMF country office worldwide. The three months internship is structured as a collaboration with the AMERU. As part of the internship, Maria Ngarachu investigates the relationship between transport costs and labour market outcomes in South Africa.
AMERU wins prestigious PEGnet Best Practice award
A joint research project on Youth Wage Subsidies in South Africa undertaken by researchers from the AMERU at the School of Economic and Business Sciences (WITS), University of Stellenbosch, and Yale University (Professor James Levinsohn) recently won the Poverty Reduction Equity and Growth Network (PEGNet) Best Practice award for 2014 at a conference in Lusaka, Zambia. A committee consisting of Ms Sonja Kurz from the Deutsche Gesellschaft fuer Internationale Zusammenarbeit (GIZ), Mr Shebo Nalishebo, from our local host, the Zambia Institute for Policy Analysis and Research (ZIPAR), Ms. Natascha Weissert from the German Federal Ministry for Economic Cooperation and Development (BMZ) and Mr. Manfred Wiebelt from the Poverty Reduction Equity and Growth Network (PEGNet) of the Institute for the World Economy in Kiel chose this project over all other submissions because of its effective interaction between research and practice and its success in translating research into action. The selection committee was particularly impressed by how the research undertaken as part of this project helped inform the design and implementation of South Africa’s Youth Employment Tax Incentive which was introduced in January 2014. The AMERU research team include Dr Neil Rankin (formerly Wits, now Stellenbosch University), Volker Schöer (Director of AMERU, SEBS) and Gareth Roberts (AMERU, SEBS).
Appointment to the Umalusi Research Forum
Volker Schöer, Director of the AMERU, has been appointed as a member of the Research Forum of Umalusi. Members of the Umalusi Research Forum provide guidance and support to the Umalusi Statistical Information and Research Unit with regards to the: conceptualization of the research agenda; design of specific research projects; strengthening of the research function of the Research Unit; and to evaluate outcomes of research conducted. The appointment is for the duration of one year.
CLEAR/ AMERU present Technical Course on Impact Evaluation : How to Manage, Design, and Conduct Impact Evaluations in Johannesburg, South Africa
The technical course introduces impact evaluation as a key instrument for determining project/program effectiveness, informing policy development and improving program designs. It covers commonly used econometric and statistical methods to evaluate the impacts of social and other programs in developing countries. The course includes both randomized and non-randomized methods and covers other practical aspects, such as managing an impact evaluation team, planning and implementing data collection, and disseminating the results of the evaluation. Together with Professor Paul Glewwe (University of Minnesota) and Dr. Claire Vermaak (University of Kwazulu-Natal), several AMERU researchers and affiliates (Volker Schöer, Dr. Tendai Gwatidzo, Tasha Naughtin, Muneera Domingo, Wayde Flowerday and Amy Thornton) presented and facilitated the course to government officials and evaluation specialists.
AMERU is mentioned by Finance Minister Pravin Gordhan in the 2013 Budget speech for its involvement in the Youth Wage Subsidy study:
"Dynamics in the youth labour market: Since 2009, the National Treasury has worked with the African Micro-Economic Research Unit at the University of Witwatersrand to study the youth labour market. The study tracked 4 000 non-economically active and unemployed young people aged 20-24 years, and more than 600 firms, over a period of three years. Part of this research included testing the impact of a R5 000 hiring voucher on employment outcomes. Half the sample received a hiring voucher that could be used to reduce the initial cost of labour for their prospective employer, over a minimum of six months, while leaving their wage unaffected."
For the full Budget Speech see: