In 2019, we set out to develop a tool to measure actual impact on individuals as a result of the Make India Capable program. We had London Business School and Project Aasha, a pro-bono consulting initiative, partnering with us in this endeavour. Through its Social Impact Club, London Business School nominated its post graduate business management students to intern with Head Held High from Feb-May in 2019. The engagement had two key outcomes;
2. Pilot the framework for at least two batches to define its application methodology and limitations. Gather feedback from trainers and trainees in the design process and finalize the tool.
Our aim was to develop an impact tracking framework that would allow us to measure tangible benefits that accrued to trainees as a result the program. We could then define success in terms of how effective our interventions were in enabling transformation in the lives of youth. We conducted surveys of youth at different stages in the process and then analysed the data generated to gauge shifts in specific indicators of individual growth.
The process of data gathering and analysis is outlined below:
Connect with key stakeholder in the program – the youth. Conduct a survey at the baseline and end line to enter responses to questions to assess their personal, social and economic status.
Develop methodology for assessing real and tangible change in the youth based on 11 key indicators.
Gather data through baseline and end line assessment covering at least 70% of trainees in a batch.
Data from baseline/end line surveys analyzed to obtain scores for each trainee and batch. The analysis gives visibility into key drivers of and barriers to transformation through the program.
Upstream and downstream sharing of impact assessment report. Impact assessment process broad based with each new batch.
MIC measures youth transformation on three dimensions: economic, social and personal. Each dimension is defined by multiple indicators. This allows for a more comprehensive assessment of the trainees based on all the factors that can potentially affect their life and work readiness.
The economic dimension covers the three major areas of:
The social dimension of the framework attempts to understand the trainee’s perception of social institutions and culture and ways in which these affect them. This comprises four metrics as below:
The personal dimension of the framework measures the personal development of the trainee. Not only does the MIC program aim to improve physical skills, but it also helps to develop positive personality traits such as grit, motivation, drive, confidence and vision. The key metrics included here are:
Trainee assessments are conducted as per the following timeline:
| # | Assessment | Timeline | Remarks |
|---|---|---|---|
1. | Baseline | Induction Day 1 | Day 1 |
2. | Midline | Sprint 6 - 3rd Week | Day 173 |
3. | End line | 6 months Post-graduation | Day 360 |
All metrics are measured on a 5-point scale as below:
| 1 | 2 | 3 | 4 | 5 |
|---|---|---|---|---|
Strongly disagree | Somewhat disagree | Neutral | Somewhat agree | Strongly agree |
Post measurement we derive a transformation index – both at the trainee and batch level – which ranges from 1 to 5. This is indicated both at the dimension level as well as an overall transformation index which is an average summation of all the dimensions.
The transformation index is a measure of change that occurs in the personal, social and economic dimensions of a disconnected young person. The higher the score on the 5-point scale, the greater the degree of transformation. The impact framework is aligned to measure the incremental effect on multi-dimensional poverty that our program aspirants belong to. The transformation index plays out like a predictive tool to indicate how long a program aspirant will take to move out of the cycle of poverty.
Using a questionnaire tool at the baseline, midline and end line, the program can observe shifts in all three dimensions. These shifts are enabled by in-class curriculum, sector-based exposure, extensive guidance/counselling and context-based learning.
Post the baseline survey, the index provides evidence to the program trainers to create focused interventions for those who have a score below 3.5. At the midline (end of sprint 6) a score of 3.5 and above indicates the success of the training. At the end line (12 months post training), a transformation score higher than 4.1 indicates upward mobility of the program aspirant on the transformation trajectory.
The program gathers gender disaggregated data for personal, social and economic transformation. Such data lends the program its unique ability to design market access solutions that suit the needs of male and female program aspirants.
In an effort to validate the impact framework, we conducted a pilot study involving about 30 youth who had been in the program in the decade or so since it was initiated.
This exercise helped us uncover some insights into how transformation occurs in youth. Below are some of the main shifts that we observed in the lives of the youth as a result of the program.
Youth from poor socio-economic backgrounds lose at least 5-6 years to poverty before they can break the cycle. This can be accelerated by focusing on place-based solutions and also adopting a long term multi-dimensional approach that includes upskilling, mentoring, financial literacy and social security support.
The program has been able to change their social status – they have earned respect and admiration from their communities and are looked upon as role models.
As youth start earning, their first priority is to improve quality of life for their families. They then shift their focus towards asset creation. This shift usually takes place after being employed for 203 years following the training.
Most of the program alumni we talked to indicated that educating their children and ensuring their well-being is their topmost priority
A key outcome of the program is the development of the spirit of changemaking in alumni. Many of them said that, once they moved out of poverty, they were more aware of their ability to help others and were motivated to do so. From time to time, many of them make small social contributions.
The graphic below details the mindset and behavioural changes that we observed in youth in the study over the decade following the program:
