N O T E S from CYE Metrics Meeting
Thursday 25th April 2019 – 14.30 to 16.00, House of Commons, Room W-2
Present:
- Jeremy Lefroy MP – Chair, CYE
- James Ingram – Prince’s Trust International
- Israel Coomer – Prince’s Trust International
- Julian Baker – School Enterprise Challenge
- Diletta Morinello – Y-Care International
- Philip Mader – Institute for Development Studies
- Justin Flynn – Institute for Development Studies
- David Woollcombe – Peace Child International
- David Kitt – Raleigh International
- Alexander Woollcombe – United for Mental Health
Key Outcomes:
- The meeting encouraged Philip Mader & Justin Flynn of the Institute for Development Studies to prepare & submit a proposal for a Systematic Review of the field of measuring youth job creation outcomes;
- The meeting encouraged Jeremy Lefroy (Chair) to set up a meeting with HE Yamina Karitanyi of the High Commission of Rwanda in London to discuss conducting a pilot metrics project in Rwanda
- The meeting supported Jeremy Lefroy’s suggestion of setting up a website where members of this Coalition of the Willing could share metrics methodologies, approaches and results;
Introductions & Expressions of Interest: the chair invited each member present to explain why they were interested in this Metrics Field:
Philip Mader: – “I’m here to listen and learn – and to scope the breadth of what is being measured now. I feel it’s important systematically to assess the whole field – to find out what’s failing young people? I have a sense that programme-driven approaches do often fail young people.”
Justin Flynn: – “Philip and I come from the same place. I have been researching youth employment for six years so know that metrics are really valuable. To find out what works – you have to know how to measure it? And it’s not just programmes: you have to think about the systems approach – a systems approach enables you to figure out how you get people in rural Nepal employed? Nobody’s done that – and I think we can do that.”
Diletta Morella: – “This is really relevant to what I do. I am in the process of re-framing our results framework and currently systems are a missing piece. It will be ready at the end of June and be more inclusive. Vulnerable people are our major target. So my interest goes beyond standard metrics and do things differently: standard metrics don’t go deep enough and are not always relevant in rural areas.”
David Kitt: – “Raleigh has great access to networks of young people on the ground; it’s very important NOT to have a bunch of older people sitting in a room in Westminster deciding these questions for young people. It is vital that youth themselves are involved. We’re very aware that our measurements of programmes to date have been business-as-usual. We’re in the process of change – so this meeting is very timely for us: we have woken up to the importance of Social Capital and need to learn how to measure that…”
James Ingrams: – “PT International is recruiting a new Data Manager! Please put it out to your networks and if you know any good candidates, please put them my way. I drafted the 1st Intl. Strategy for PTI. Entitled: “My Journey”: it follows a young entrepreneur from baseline entry to finish. My background is as an anthropologist – so I’m interested in longitudinal studies and the wider family aspirations / implications of decent jobs – about “how to get a life, not just a living” – in the words of David’s briefing paper. Accountability drives results – but its not just about results and reporting to donors: it’s about young people flourishing. We know what we need to know but we have to learn what is verifiable and what is possible within the current constraints of data protection and portability. These are issues that we have to deal with.”
Julian Baker: “I came to Teach a Man to Fish five months ago and am here to represent their 2 x staff people, Christine & Amelie, who do their M & E; 70,000 went through their Schools Enterprise Challenge(SEC) last year (“We want it to be 7 million!”) But we know it’s not just about numbers – it’s about the quality of programmes. We know the SEC has a wonderful impact on students and teachers but we need better evidence that we are changing people’s lives. So we are developing a programme to measure qualitative indicators.”
Izzy Coomer: “I don’t have much to add to what James said but we are concerned about the verifiability, and the quality of the data; how it links to the SDG 8 and ILO ‘decent work’ indicators – and how our indicators can link into the government VNRs that are being drawn up at the moment. Anything we do as a coalition should be cognizant of the VNR process and feed into it as far as possible.”
Jeremy Lefroy: “My interest is coming from the Private Sector in Tanzania where I ran a coffee and cocoa business for ten years. And set up Equity for Africa – an equipment leasing bank which has helped to create sustainable jobs for thousands of young Africans. As an MP, I have served on the International Development Committee in the House of Commons so, cutting to the chase, I see 2 x lines of work here:
- Data Side – “I’m not in favour of data collection for the sake of data collection. I come from a business background – coffee and cocoa. My leasing bank created the most jobs – it was our main focus;
- Job Creation Side – as Louise Fox said, a lot of training programmes don’t lead to any jobs. So he supports the idea of delivering a Working Paper on what works to be presented to the Kigali CHOGM next year.
David Woollcombe(DW) introduced his interest in the field as an eagerness to secure better data for, and attract more funding to, the field of Youth Job and Livelihood Creation which he feels should be at the very top of the International Development Agenda. He also explained that his interest in ensuring that the language we fought hard to get into the CHOGM Final Communiqué last year, namely: “… the importance of investing in a systems approach to create meaningful employment opportunities for the Commonwealth’s growing youth populations. They also stressed the need for better data to target interventions effectively.”
David is determined that these should not just remain words on paper: we must come to the next CHOGM in Kigali in June 2020 with a serious response to this mandate. He added that there are several other interested parties who could not be with us today, but who wanted to be a part of the process going forward:
- Hur Hasnain, formerly of Y-Care, now moving to the British Council; a founder director of IDEAS;
- Christy Olenik, Vice President of Technical Services at Making Cents International – with whom DW had discussed the idea of holding a Round Table on the systems approach to Youth Employment Metrics at their Global Youth Economic Opportunities Summit on October 3-5 2019. (In a follow-up call with Christy and Sarah Sladen this morning, they confirmed their interest in having a progress report in October this year, with a view to having a full-dress event following the CHOGM at the 2021 Summit. They asked us to work this into our calendar of events – and keep them updated regularly with our progress.)
- Jessica Greenhalf and Anita Thiessen of Youth Business International and Shedron Collins of the Youth Business Trust of Trinidad & Tobago (YBTT) – who have offe. red to share their immensely valuable tried and tested approaches and methodologies;
- Linsey Wallace, Director of Strategy and Learning at the Mastercard Foundation; DW has a call with her on 9th May to see how we can draw on the considerable learning and leadership they have in the field;
- Susan Reichle, the new CEO of the International Youth Foundation, is currently overhauling their strategic approach which involves “deepening our monitoring, evaluation, research and learning.”
- Stephen Hunt of the Challenges Group, a Business Development organisation based in Edinburgh
- Alison Hilliard, the Wilton Park executive master-minding the Powering the Future Youth Dialogues
Alexander Woollcombe apologised for being late and introduced his interest in the field as part of his work for United for Mental Health – a global movement designed to create a world where everyone, everywhere, has someone to turn to in support of their mental health: poor mental health is a massive constraint to successful youth job creation, and should therefore be considered a central component of a Systems Approach.
Discussion Notes:
DW defined a systems approach as one that “evaluates all the elements which affect the viability of a policy initiative. It’s a line of thought which focusses on the interdependence of all external and internal factors in an organizational strategy or management policy.” It is a combination approach of every component of the social, financial and political machinery that impacts the outcome: it looks at everything – and seeks measures to find out if each component is working optimally. It is immensely complex – which is why the measurement of a Systems Approach has not really been tried before. DW supports the idea of a Systematic Review because a systematic approach can be defined as one that can be used to determine the viability of a project or a procedure based on the experiential application of clearly defined and repeatable steps and an evaluation of the outcomes leading to the generation of consistent, optimum results. “The reason I am interested in doing Pilot Metrics Study projects in Rwanda and South Africa is that they have both been successful in generating new jobs for young people: so I would like to be able to capture HOW they have been successful, put it in a bottle, and share their procedures and methodologies with other members of the Commonwealth at the Kigali CHOGM in June 2020.
Philip: Let’s be honest – we don’t know what works; the caveat on a systematic review – is that it takes 12 months; they make sense where there is a wealth of evidence; no single person can do the methodology alone; you don’t always end up with evidence on “What Works” – what you do get is a sense of what’s being found out; I don’t think you will ever find a common metric – but you might get a menu of alternatives;
DW: I don’t think we should lose sight of the idea of the National Rankings which I mentioned in the Briefing Note. I road-tested these at the ComSec’s Caricom Workshop – comparing policies and stakeholder impressions of their effectiveness across 11 x different Caribbean nations using the “Draft Bench-marking questionnaire” that I sent you all: you can see the results here. Getting more detailed responses from all 5 x stakeholder groups across successful job-creating countries like Rwanda and South Africa to feed into a systematic review and, thereby, into the CHOGM in Kigali would allow governments, and the CHOGM Youth Forum, to self-assess how they are doing in their own countries – and start to see where the gaps are in every Commonwealth Country.
Julian(?) mentioned McKinsey’s Generation Project and their recommended CPED (cost per employee day) which is a useful way to measure the cost-effectiveness of different approaches to job creation training interventions in 5 x Countries: India, Kenya, Mexico, Spain & the USA. Several of the data collection tools and strategies may have wider application. Read the article by Mona Mourshed and Ali Jaffer, of the Generation Project;
David K asked: Should we narrow down?? – and focus on a particular sector: government / private; urban / rural; formal / informal. It was agreed that the USP of the CYE approach is the effort to measure systems approaches – which, by definition, embraces all sectors. However, several mentioned at different times through the meeting the importance of establishing context: measuring apples with apples, not apples with oranges. Also central to a systems approach are the emergence of dangerous indicators – on climate, demographics etc.;
Jeremy was asked: “Where do you see the momentum in the UK government coming from for this issue?” Jeremy answered that everything is up in the air at the moment (because of Brexit – Eg. Lord Bates suddenly resigned to walk from Belfast to Brussels, to be replaced by Baroness Liz Sugg – who is great!) But he understands that DfID staff are happily getting on with their work unencumbered by the usual Ministerial oversight. He added that he is very keen to highlight what UK-based organisations are doing at the moment; drawing out the toplines of what you are all doing! He suggested creating, or expanding, the Metrics Section of the Coalition website to gather and shine a spotlight on that information. For example, he had just had a meeting with Farm Africa – a £16m per year organisation which has helped create tens of thousands of jobs in rural Kenya, Uganda, Tanzania and Ethiopia;
He also mentioned that we should talk to James Whiting, the CEO who has made such a success of Malaria No More. Today is World Malaria Day – and he is going to see John – and Harriet Baldwin – later tonight.
Julian pointed out that SEC has learned a lot from programmes that didn’t work in Uganda & Rwanda. That learning needs to be captured as well.
James mentioned that, in a recent conversation he’d had with HE Yamina Karitanyi, the word that came up in many of her answers were: “Announcables” and “Deliverables.” That is what her government wants to come out of the Kigali CHOGM. We need to create something to announce – a result to deliver.