The Education Select Committee has launched an inquiry into CEIAG – Careers Education, Information, Advice and Guidance – and issued a call for evidence to which I submitted a few thoughts.

Among the many points I made, there were two that I thought might be worth blogging about. Firstly what do we actually mean by CEIAG? Secondly, what does that tell us about professional careers practitioners?

What is CEIAG?

It’s worth drawing a distinction between the components of CEIAG and why it is necessary to consider them separately as well as together. 

Careers education is education about careers, ie. learning about the different ways people make a living, what those different careers involve and some of the pathways that people take into and through careers. Ideally, careers education also involves learning about employability (those attributes that mean an employee can add value to an employer), how to acquire it and how to demonstrate it to a potential employer. 

Careers information is factual. It may be data (for example, labour market information) or it may be other factual information, but generally, it is largely uncontentious (if soundly derived) and lacks context. One analogy I often use is to say that if I say “beer in this pub is £2 a pint”, I am providing you with information. In isolation, information is not very helpful to the person at the receiving end.

Careers advice puts information into context, making it potentially useful to any person who happens to receive it. To use the same analogy, it would be advice to say that “the average price of beer is £2.40/pint, so this pub is relatively cheap”. Good advice is true in a general sense, even though it is insensitive to any individual’s perspective. 

Careers guidance, however, is personalised and starts with the individual and their hopes, opportunities and needs. For example, it is guidance to ask, “Are you thirsty? Do you like beer? How much can you afford? What are your alternatives?” 

I find this a useful distinction because it helps us understand how best to deliver the component parts of CEIAG. Careers support should not stop at CEIAG though. Beyond those components we should not overlook the potential role of mentoring, behavioural/mindset support and practical help (such as funding for trips and open days or clothes appropriate for work experience, etc).

Careers physicians

Of the four components of CEIAG, guidance is the most useful to the individual, but the hardest to deliver and largely redundant without the other three. Guidance requires knowledge, skills and contact (albeit sometimes virtually) with the person being guided. A careers guidance practitioner bears enormous responsibility because it is their role to draw aspirations out of their client and frame them in the context of opportunities.

I often find myself drawing comparisons between professional careers practitioners and doctors.

A doctor uses their training, experience and expertise to diagnose someone’s needs and prescribe treatment with the selfless aim of relieving suffering and improving the patient’s quality of life.

A careers practitioner uses their training, experience and expertise to diagnose someone’s needs and provide guidance with the selfless aim of giving their client self-agency and improving their quality of life.

Like medicine, careers guidance has become an evidence-based, theory-driven profession equipped with sophisticated tools and, given the almost Hippocratic responsibility, careers guidance should never be entrusted to anyone who is not adequately trained to do it responsibly, knowledgeably and professionally. The Government should require anyone working in a publicly funded role as a careers practitioner to be on the CDI Professional Register.

Speaking plainly

Another parallel with medicine is that careers policy seems to get bogged down in jargon easily. Many professions do this – law, academia, the armed forces – develop a jargon to signal to those on the outside that there is a guarded gateway through which only the cognoscenti may pass. 

In the case of careers practitioners, it may be something to do with a defensiveness against the kind of dismissive attitude that they often face and which I’ve written about before.

So, in my submission to the Select Committee, I made a point of writing plainly.

The problem of inadequate CEIAG contributes to huge policy issues: low productivity, social and regional inequality, and the opportunities of individuals to live fulfilled lives. Yet the solutions – or at least the principles behind them are not that complex and don’t need to be wrapped in gate-keeping language.

After all, the MPs on the Committee – boundless in their wisdom though I’m sure they are – are not inside the gateway.

Nor am I. I have no qualifications in careers practice and, by my own strictures, I should definitely not be allowed to deliver careers guidance. However, I have worked in awe alongside careers professionals; I have delivered careers education, information and advice for many years; and I have read and researched widely.

Some of the most useful research – for me – has not been the research on careers itself, but the wealth of behavioural science research that has been published in recent decades. This developing understanding gives us a fresh perspective on how humans do that difficult thing of making decisions. By understanding that, we get a whole new window on how to improve CEIAG to promote informed choices.

If anything I’ve written here has piqued your interest, I do hope you’ll feel it’s worth reading my submission in full – I will post it here when the Committee has reviewed it. (Until then, I am not supposed to put it in the public domain.)

UPDATE 30/6/2022

The Select Committee has now published the evidence it received and so, I can now share my submission.

The post Signs and wonders: Better CEIAG appeared first on Johnny Rich.

The Higher Education Policy Institute has kindly published an article I wrote on the interim plans for reform of the National Student Survey. 

The proposed changes are contained in the OfS’s Phase 1 Report of its NSS Review which was sparked by a somewhat untoward statement by the DfE last year that the NSS was responsible for “dumbing down standards“. No evidence for this claim was offered and it was exactly the opposite of… well, everything that they and predecessor governments had ever previously said about NSS’s role in enhancing the quality of higher education.

Indeed, the credibility afforded to the NSS previously meant that it was a key metric used in the TEF (the Teaching Excellence Framework, as it then was, now called ‘the Teaching Excellence and Student Outcomes Framework’).  Its weighing as part of TEF was downgraded, however, when student opposition to the exercise led to widespread boycotting of the survey.

The main reason the government has gone sour on the NSS though seems to me to be that it doesn’t endorse their political narrative about higher education – or, even if it does, the signal is too noisy, too nuancy. For example, NSS doesn’t say that the only good education is one that results in a job. It doesn’t say that our universities are all world-beating while at the same time managing also to say that they’re full of woke academics and snowflake students. And it fails woefully to confirm that traditional redbrick and Russell Group unis are better than jumped-up polys. 

Indeed, the university with the strongest record of performance in the NSS since its inception is – wait for it – the Open University. What should we make of that? There are multiple explanations for its NSS success, not least the fact that the survey is taken as students approach graduation and for OU students, that’s likely to have been a long, hard slog of many years, involving considerable commitment and sacrifice. Anyone who wasn’t going to give a good report will probably have fallen by the wayside by that point or at the very least will be convincing themselves that it was all worth it after all. Another explanation is that the OU does an amazing job for its students far exceeding their expectations and therefore yielding high satisfaction. 

What it doesn’t tell us is anything absolute. No wonder the government has lost interest in the NSS – it doesn’t tell them anything clearly or that’s politically helpful and even what it does tell them is not what they wanted to hear.

By chance, the DfE does happen to be right that the NSS needs reforming. It’s just it’s not for the reasons they imagine. As my HEPI article explains, the problems lie in (i) imagining that NSS can ever be about informing prospective students helpfully, (ii) the snapshot data dip process of a survey and (iii) the over-emphasis on satisfaction as a measure of quality when it is in fact a function of expectation compared to delivery. 

The reform needed is to shift to a longitudinal national survey of student engagement that tracks shifting patterns throughout a student’s time at university. Engagement has been shown to be an indicative precursor of positive learning outcomes. If you can show that a student has been effectively engaged throughout their studies, you’ve got a good indicator of effective education.

Satisfaction measures are poor proxies that will never tell you much and will always be too easily gamed or misinterpreted. They do not, however, dumb down anything that wasn’t dumb already. 

The post What’s really wrong with the NSS? appeared first on Johnny Rich.

This slide, presented by UCAS’s Chief Executive Clare Marchant at a recent event hosted by Wonkhe on higher education admissions, shows starkly the correlation between clearing and drop-out that I have been banging on about since the 1990s.

The post University admissions: what’s the real problem? appeared first on Johnny Rich.

Back in August, a teacher drew my attention to the following tweet and asked if I might be able to answer it:

Can anybody point me to research regarding outcomes in later life for disadvantaged students who go to university vs. disadvantaged students who don’t? https://t.co/BiYatOdKMh

— Mr Pink (@Positivteacha) August 1, 2019

The tweet was from Matt Pinkett (@PositivTeacha), teacher, blogger and author of Boys don’t try? Rethinking masculinity in schools.

My thread of tweets in response sparked a correspondence between us and, in the end, Matt was kind enough to say I had challenged his whole perspective. He suggested others might be interested too and I should publish some of my thoughts on the topic. 

So, in the first of two blogs (here’s the second), here’s how I responded to his initial question… 

There is plenty of research showing a significant earnings premium on average for graduates regardless of background. Probably the most comprehensive work is the paper by the IFS ‘How English domiciled graduate earnings vary with gender, institution attended, subject and socio-economic background’. The Sutton Trust has also done many excellent studies on different aspects of this question which is actually a lot more complex than it sounds.

Policy Press | The End of Aspiration? – Social Mobility and Our Children’s Fading Prospects, By Duncan Exley

 

The post Is it worth going to uni if you’re from a poor family? appeared first on Johnny Rich.

This article first appeared on the Wonkhe website (8th April 2018) under the heading ‘TEF won’t sweeten my rankings rancour’.

Which is the best university? It’s a seductive question to ask, but that doesn’t mean there’s a sensible answer. League tables, aka rankings, is the nonsensical answer you’re likely to get.

They weigh the wrong factors – a very narrow idea of best, based on counting what’s measured rather than measuring what counts. Traditionally, this has led to a dominance of rankings by research-led institutions.

But even if the factors weighed were the right ones, the rankings use poor proxies to measure them – as if research citations, for example, were an unambiguous marker of quality, rather than being hugely dependent on publication in English, in the right journals and in the right disciplines.

But even if they were the right proxies, the data is often of poor quality: out of date, non-comparative, self-reported.

But even if the data were good, what rankers do with it isn’t: aggregating and weighting arbitrarily.

But even if the methodology were sound, the way the results are presented suggests an equal distance between say, first and thirty-first place as between fortieth and seventieth. Anyone who has ever seen a bell curve knows that is misrepresentation.

But even if league tables didn’t make all these mistakes and more, their worst crime is to imagine that there is such a thing as a single best university, rather than many different ways in which universities can be good at different things. Indeed, it is the very diversity of the higher education sector that is its strength. It means the sector as a whole can paint a rainbow of objectives catering to the divergent needs of particular students, communities, employers, economies and societies.

No platform for rankings

You can’t ban league tables, sadly. If we want information about higher education to be transparent, then there are those who will put it in a pop chart. That will attract attention, because offering an answer to that “best university” question is sexy.

The answer might not be to have fewer league tables, but instead to have more: an infinity of rankings so that each person can pick the one that combines just the factors they want, weighted perfectly to their needs. No ranking would be authoritative, because the array would reflect the personal and diverse nature of the question.

THE’s latest rankings product (its Global Impact Ranking) is a step in the direction of infinity in that it adds another league table to the shop window, incrementally diminishing the value of the ever-increasing heap.

However, perhaps we should welcome the desire to rate universities according to criteria such as recycling, fair labour practice and admissions policies, even if the process is as flawed as all the others? After all, the sexiness of rankings does shine a light on issues that might get overlooked (especially when the desire to do well in other rankings distracts universities from considering what else matters).

TEF: just another ranking?

That was explicitly the government’s intention when it introduced its own form of ranking – the Teaching Excellence Framework (TEF), which the then-minister Jo Johnson said would “introduce new incentives for universities to focus on teaching”. The idea was to rank universities’ teaching quality to get them to improve it and to drive student choice based on quality.

The problem is that TEF repeats the mistakes of other rankings. It weighs the wrong factors: the metrics (as was later acknowledged with the change of the name to include student outcomes) have little to do with teaching. It uses poor proxies, such as measuring employment not employability. The data is poor: the NSS component was downgraded after an NUS boycott undermined it. The methodology is arbitrary: for example, benchmarking by disciplines, but not regions.

The list goes on, but TEF is unlike other rankings in at least three respects. First, being the government’s own ranking, TEF bears more responsibility than most. It purports to be a truer truth – an authority that it hasn’t earned.

Second, most league tables – even though they are rarely entirely open about their methodology – do tend to stick to it. TEF, however, recognises the failings of its metric methodology and adds a subjective element: the review panel.  It may be the best part of TEF, but it’s the least transparent and most susceptible to inconsistency.

Third, most league tables’ misrepresentation is a single hierarchical list. TEF retains the hierarchy, but shrinks distinctions to three categories: good (bronze), better (silver) and best (gold). This, of course, creates a cliff edge where a fine judgement between silver and bronze, say, translates into a presentational gulf.

Informing student choice

Interestingly, there is no “mediocre” or “bad” in this hierarchy, but that’s not how students see it. Bronze is no one’s idea of an endorsement. This highlights an absolutely critical issue about rankings – TEF included – which would be the case even if they were more rigorous in their approach: how do they inform student choice?

Human choices are rarely rational. They emerge from a soup of feelings and preconceptions, sprinkled with croutons of information fried in confirmation bias. When it comes to a complex decisions, such as which university to choose, we don’t devise a personal list of criteria, sourcing objective data on each, and then coolly and fairly appraising the options relatively. Instead we latch on to something that provides a basis for beliefs we already hold.

In other words, we use heuristics: rules of thumb that often bear little resemblance to nuanced realities, but which hurt our brains less. This is precisely the quality about league tables that makes them so sexy. They say, don’t you worry your head about the real differences between two institutions that are both good in their own way, we’ve made the whole process simpler. Misleading, but simpler.

The same is true of TEF. Rather than providing information that disrupts misplaced beliefs and encouraging students to examine what kind of educational experience will support their own learning, TEF short-circuits the thinking and provides a yes/no/maybe checklist.

The Government was right to shine a light on teaching (well, on learning), but not the seedy neon beam of TEF. There are other approaches and, as Dame Shirley Pearce proceeds with her review of TEF, I hope she will think boldly about options that promote diversity and innovation rather than aping league tables that suppose there is a single model of “good” and which play blind darts to see who gets closest.

The post Rancour, rankings and the rankness of TEF appeared first on Johnny Rich.