Select Page

Original Discussion Paper

The Metrics Project

“Measuring a Systems Approach to Youth Job & Livelihood Creation”

At the Commonwealth Heads of Government Meeting held in London in April 2018, Heads  – “….stressed the importance of investing in a systems approach to create meaningful employment opportunities for the Commonwealth’s growing youth populations. They also stressed the need for better data to target interventions effectively. ”        Clause 16, Final Communiqué, London CHOGM, April 2018

Purpose of the Metrics Project:

To secure better data for, and attract more funding to, the field of Youth Job and Livelihood Creation;

Theory of Change:

There is currently no single, agreed metric by which to measure “What works?” in the field of Youth Job and Livelihood Creation.  This means that ODA funders have no mechanism by which to judge and compare the impacts of different interventions. More important, they do not have the means by which to compare impacts within different contexts, nor the cumulative impacts of a concatenation of different interventions combined in a ‘Systems Approach.’

By reviewing what different actors in the field of Youth Job and Livelihood Creation are measuring at the moment, the Coalition for Youth Employment will develop a methodology that delivers comparable metrics across similar interventions.  Armed with this methodology, we can compare, and learn from, each other’s approaches and reinforce each other’s successes by highlighting the added value(s) that each of us brings to the field.

The Process:

  • Share current evidence: Coalition members have been invited to share & review the M & E methodologies they use at the moment along with a digest of the evidence they have drawn from it. [See Appendix “B” Spreadsheet & Links.] Much of this information is publicly available on websites already, but we have been privileged to discuss current approaches with Coalition Partners. This process is in its early stages and will expand into in-country studies. Though we have talked to the Institute for Development Studies and find their insights are invaluable, this Metrics Project is designed principally to share the approaches of practitioners – not academics or theoreticians.
  • Share current approaches: For many young people, Monitoring & Evaluation is perceived to be a chore to be endured, like end-of-term exams. Many of us in the field are exploring novel approaches by which M & E can be perceived to be a celebration and a learning process in itself. In the same way that creative HR people have transformed young people’s competitive interview procedures into a constructive shared learning experience, we shall encourage and promote Coalition members’ methodologies which are esteem-building experiences for young people and the teachers / trainers who serve them
  • Identify Supply Side Measures: Most coalition members deliver supply side interventions so a rich seam of study is to explore how best to measure the Soft/Life Skills they teach + the Hard/Technical, Skills, Basic Skills, Entrepreneurial Skills, creative skills & confidence/self esteem engendered..
  • Identify Demand Side Measures: The Coalition questionnaire at Appendix “A” and at: gcye.co.uk [the Trinidad & Tobago National version]) – includes bench-marking questions on Demand Side measures as well as the supply side. These deliver important information on the comprehensiveness of a government’s, and other stakeholders,’ systemic approaches.
  • Host Brainstorming meeting: The House of Commons meeting on Thursday April 25th will:
  1. Discuss different approaches and how to handle verification;
  2. Agree a Schedule of Work with agreed dates for the deliverables – and
  3. Set up a working group to implement the Schedule and organise –
  4. Deliverable ONE – by 31/12/19: a Working Paper to be peer-reviewed by Coalition members & others (including, we hope, a 2nd Wilton Park meeting on “Turbo-Charging youth employment” +
  5. Deliverable TWO – by 1/6/20: a detailed analysis of, and Proposal on “How to measure What Works in a Systems Approach to Youth Job & Livelihood Creation” to be delivered at the Kigali CHOGM;

Metrics Project Imperatives:  The Coalition leadership adheres to the following priorities:

  • To Sectoralise: Though our goal is to draw together the cumulative impacts of all components of a systems approach into a single measure, using a combination, or innovative, left-of-field approach, we will not compare apples and oranges: the same measures used for a school-based entrepreneurship programme are unlikely to work for a large-scale industrial start-up park.
  • To Talk the language of the ODA Agencies: The Coalition will use the language of DFID’s 4 x Es approach: Economy, Efficiency, Effectiveness & Equity. It was DFID which initially asked the question: “What works in the field of Youth Employment?” – and we feel that the language / vocabulary that works for them will work for other ODA agencies as well. We shall also adhere to DFID’s approach to value for money in programme and portfolio management.
  • Contextualisation: In our search for comparability, we will not prescribe rigidity or that ‘one-size fits all.’ Rather, the measures we employ will incorporate flexibility to allow for the geographic, cultural, gender, economic, urban/rural, political and social contexts in which they are found.
  • Results-based Measures: We will measure the results achieved – the Jobs & Livelihoods created – the incomes raised, the health, nutrition, education and housing measures improved. We will not just measure numbers registered for, or completing, a training programme or their test scores;
  • Social and Financial Indicators: It is imperative that our metrics measure both Job Creation and decent Livelihood Though the major measures of job creation can be numeric – salaries earned, hours worked, length of employment etc. – Livelihoods need to be measured in social and qualitative measures. Our metrics should embrace both because, as Stefan Dercon and others have established, security and growth are as much dependent upon the contentedness of a work-force as it is on their incomes. Or, as the 2019 “Getting By” Conference put it: “Getting a job is as much about getting a life as getting a living….”
  • Verification: It is vital that we find ways independently to verify data collected by Coalition partners. This is a key question to be discussed at our House of Commons Meeting;
  • Sharing, transparency and openness: though we are well aware that some results from some Coalition partners is proprietary and not to be shared – the Coalition and, we would argue, the whole field, can only develop and improve if it is open and willing to share what has worked well – and what has not worked so well. There is no compulsion: participants can engage, or cease to engage, at any point in the process. Also – before anything is released for peer review, all participants and members of the Coalition will be invited to sign off on – or insert written reservations to – the Deliverables before they are published.

Cost-per-job-created?

This would be an obvious comparator:  if an economist could reckon that, per $1,000 invested, 5 x jobs would be created – this would provide an easy, accessible metric by which to measure the impact of job creation investments. However, a moment’s thought would remind the economist that it costs several times more to train and create a job for a neuro-surgeon than it would to train and create one for a street retailer. SPARK once announced on their website home page: “17,000 jobs created! Cost-per-job-created: between $79 to $1,897” – which subtly hints at the ridiculousness of a simplistic approach to the measure.

PCI did a considerable amount of work on the C-P-J-C idea and came up with a questionnaire for the young women on our courses in Low Income Countries that asked them questions on:

1) their Employment Status [• salary? • hours worked? • time in the job?];

2) Their Basic & Business Skills [• literacy & numeracy? • able to write a business plan? • able to do market research? • able to run a marketing campaign?];

3) Their Health;

4) The Situation of their Children [• children at school? – and / or • Children working?];

5) Their Power status in the family [• How much of your salary do you give to your husband or father?];

6) the amount of unpaid domestic work they were expected to do; – and –

7) Their general sense of well-being, self-confidence and self-esteem.

It was delivered pre-training, at training end, then at +6-month, +1-year and +2-year intervals. PCI’s staff reaction to this approach was that it was extremely complex and would require funding far in excess of that which any donor agency would be prepared to give. However, any systems approach metric must, perforce, embrace complexity – so an approach along these lines could be explored.

HAVs & HAV-nots?

In the above example, we gave each answer a numeric ‘Value’: 0 thru’ 5, 5 being the highest value, 0 the least. This ushered in our concept of HAVs and HAV-nots: HAVs are youth with High Asset Value in terms of their basic & business skills, health, well-being  etc.; HAV-nots lack this value to the Labour Market.  This may points to a valuable way to measure the quality of a government and private sector’s systems for preparing young people to have value to employers and in the self-employment / household enterprise situation of the 21st Century job market. A supply system which gives young people a wide range of ‘Assets’ to the labour market contributes to a nation’s prosperity; one that doesn’t fails it. Married to a rigorous analysis of demand side measures, this could deliver the comparable methodology of systems approaches that we seek.

Job-Creating Governments – A League Table?

The country-specific Questionnaire attached at Appendix “A” was answered by a variety of stakeholders from different Caribbean countries at a recent Youth Employment seminar. Taken to scale, and answered by representatives of the 5 x stakeholder groups identified in our Systems Approach:

1) government;

2) the private sector;

3) NGOs and academics working on Youth Job Creation;

4) Donors & Investors; – and –

5) youth themselves

– answers to the questionnaires allow stakeholders in different countries to see how much, or how little, each is doing to create jobs for their youth populations and how effective each stakeholder feels those efforts are?  In contrast to other League Tables, this one is all about gap identification, and garnering support for governments and stakeholders which seek to fill those gaps.

In Trinidad & Tobago, the first country to adopt the Coalition’s approach, the government has convened a partnership of Stakeholders called a National Coalition for Youth Employment, managed by a Board comprised of representatives of representatives of each of the 5 x Stake-holders. With the gaps identified by over 1,000 responses to the questionnaire, each stakeholder group is now developing its own National Action Plan(NAPs) to fill those gaps. The government, and Coalition consultants, will review the NAPs, and draw them together into a single, comprehensive National Systems approach for enabling the Demand & Supply sides in their country to create more jobs.

These are just some ideas PCI and other Coalition members have been discussing as potential solutions to this timeless challenge. The House of Commons meeting will, we hope, generate more. Should you wish to discuss these, or other ideas that you would like to raise at the meeting, please write to Acting Coordinator, David Woollcombe at: david@woollcombe.org or call:  07798 665 202

Share This