Open Data for Higher Education: The Road to Democratizing University Metrics

1 Comment

shutterstock_source_code_McIek

When institutions are gaming national and global rankings with rampant internal citation, indulging in “manipulation around the edges” of experience, or spending huge amounts on crazy golf courses to increase applications by “pander[ing] to the fantasies of 18 year olds”, does it really benefit the students whose applications the institution is seeking? Or the faculty it hopes to allure? How free (and how creative) is research when it’s directed towards institutional citation? How much does the steady increase in tuition fees to support the rankings arms race benefit the students who bear the brunt of the costs? The answer, which should be quite obvious, is very little.

Like every good problem, we need to start with a “why”. What are rankings trying to offer, and how we might go about doing that a little better in this, “The Age of Information”?

Why Rank?

There’s a simple answer: consumerism. Higher education, far from offering students ‘a clear, conscious view of their own opinions and judgements, a truth in developing them, an eloquence in expressing them, and a force in urging them’ (the wish of John Henry Newman), is now a commodity in the labour force—the lowest common denominator for entering many professions. As the demand for graduates increased, the premium on the degree itself increased — and with it, the price for participation.

When an 18 year old is essentially taking out a mini-mortgage to pay for their education, the need to make the right decision weighs heavier. Choosing well and aiming high is the mark of a great investment.

It’s not that data – or its friendlier relative that I like to term “information” – isn’t valuable in this context: it’s more valuable than ever. But if higher education is going to play the consumer game (and whether it should is another question entirely), the data provided to students needs to be transparent, and reliable in answering a range of questions driven by individual needs.

A forced order ranking is by no means awful, but it does homogenize the higher education landscape, forcing colleges and universities to compete on the same metrics (and to compete using the same tactics). In reality, institutions should be focusing on their key differentiators — the things that will actually appeal to students. And that’s rarely a like-for-like comparison.

“Compare and Contrast…”: Or, A Humanities Approach to College Rankings

In literary studies there is, more or less, a Canon. And there are lists and lists and lists of the “100 Best Novels” – no two completely alike, of course, and most provoking indignant disbelief from critics. But we don’t simply tell our students to read those works — or those authors — to understand their machinations and grammatical habits, their quirks and idiosyncrasies. And we certainly don’t try to formalize and model something so qualitative with quantitative measures. Well, very few of us do.

But in the study of literature, one of the first things students are taught is to “compare and contrast”. We bring our own questions — our own preferences and pet-peeves — to our explorations, and determine not only what’s similar and comparable between greats, but also what’s different and differentiating. It’s this key second aspect that’s missing from the presentations of rankings – and one that would better serve institutions and their stakeholders when it comes to analyzing data.

But that kind of analysis – qualitative consideration over quantitative forced-order ranking – demands a much different approach to the underlying data that’s driving decision making.

The Road Ahead: Open Education Data and Increased Data Fluency

But the data isn’t always there for the diving. Sure, it’s compiled in bits and pieces by various studies (with a helpful “Best Of” available here). Institutions taking part in certain ranking systems and professional services can take advantage of some individual benchmarking. Would-be analysts variously have access to increasing portions of data: the U.S. Government’s Obama Administration made headway into affording students more personalized analysis options with their Open Data initiative; and South Africa, too, is making headway, with the Centre for Higher Education Trust’s Open Data publishing 26 key indicator metrics for the country’s public institutions along with in-depth guides for use. These are great beginnings, for a data commons, but they are by no means as textured and far-reaching as we might hope to accomplish.

And the journey doesn’t stop there. Not only are we missing a data commons, but access is far from the lowest hurdle; meaningful analysis demands adept statisticians and mathematical modellers moving through the available data to provide insight into its trends, both at large and with regards to the situation and questions at hand. Whether for students looking for the right-fit institution, administrators and faculty members looking to compare spending, or deans looking to promote a more diverse approach to hiring, way-stations are needed on the journey to digital empowerment.

Complicated privacy laws, siloed data, and institutional lethargy all contribute to stalling the start. But, luckily for institutions, Universities UK has gone some of the way to illuminate the promise and perils of Open Data for higher education with An Introductory Guide (2015). And for the digital natives filling the pipeline to University, there are a wealth of data manipulation courses seeping into pre-college curricula and enrichment activities.

With Open Higher Education Data, all of education’s many stakeholders will increasingly be able to answer a range of different questions and access reams of relevant data on a daily basis. But for this to come to pass, Universities need to stop thinking about data-driven rankings, and instead pool their data not in competition, but for mutual edification — thinking along the way about how best to bring their faculty, staff, and students along for the ride.

—

*Image: Mclek/Shutterstock.

Advertisement

How Rankings are Ruining Higher Education

Leave a comment

In Weapons of Math Destruction (2016), blogger, professor, algorithmic goddess, and self-proclaimed “math babe” Cathy O’Neil draws readers through a an eye-opening journey of life in the Digital Age. She documents our life in a sea of Big Data, where so many corners of society are controlled by algorithms: everything from our credit score, to our justice system, to the cost of our insurance, and even to our participation in our own democracies is increasingly micromanaged by mathematical formulae that are all but impenetrable to us mere mortals just down here swimming.

O’Neil is an unflinching advocate for the power of mathematics, but her unerring moral bent leads her to expose the dangerous rationale trapped away in some of those automated black boxes that help to govern our lives: the algorithms she terms “Weapons of Math Destruction” (WMDs). To fit O’Neil’s taxonomy — to be an algorithmic evil — mathematical models must satisfy three criteria: they must be opaque, they must have achieved scale, and they must be causing damage to the people or processes they impact. While the book is fascinating throughout, and a must-read for any whose incredulity at modern data madness is growing (see: Facebook’s role in the demise of what we somewhat optimistically term Western Democracy), O’Neil raises some important questions for Higher Education — a sector awash with data, with a strong ethical and societal imperative — through her exploration of the U.S. News College and University Rankings.

The U.S. News College and University Rankings system exemplifies maleficent modelling — a series of hunches formalised into a taxonomy — that led all institutions to start shooting for improvement on the same squishy variables. It led to schools sending false data to the reporters, and spending ungodly sums on improving the specific metrics that the U.S. News journalists and statisticians deigned to consider.

In O’Neil’s words,

If you look at this development from the perspective of a University President, it’s actually quite sad. Most of these people no doubt cherished their own college experience — that’s part of what motivated them to climb the academic ladder. Yet here they were at the summit of their careers dedicating enormous energy toward boosting performance in fifteen areas defined by a group of journalists at a second-tier newsmagazine. They were almost like students again, angling for good grades from a taskmaster. In fact, they were trapped by a rigid model, a WMD.

In its 30+ year history, the U.S News model has gathered a whole host of detractors. From an exposé in the San Francisco Chronicle in 2001 to O’Neil’s book, academics and the public have become increasingly skeptic. But the academy should be skeptical, too, about all kinds of rankings we’re leaning on — particularly those that come from outside of industry, or that seek to turn rudimentary proxies into a real-world analysis of the effectiveness of higher education. The web and our publications are littered with well-regarded rankings considered far less egregious than the U.S. News model.

When you cast the net wider, and with a skeptical eye, the fickleness of rankings comes into sharp focus. As an Oxford Alum, there’s an odd thrill of vindication seeing Oxford sitting pretty once again this year atop the Times Higher Education World Rankings (especially, perhaps, as we’re standing on the shoulders of Cambridge). Yet while Oxford is undoubtedly home to some of the smartest, most globally influential people I know, its climb to the top of the THE world rankings occurred in 2016, in part due to the unfortunate murder of a lion on a Zimbabwean nature preserve. When the donations flooded in to support the researchers who cared for Cecil — some £750k (or $1.1m)  — Oxford’s already substantial research income (a critical data point in many rankings) saw an unexpected boost, and the institution over-leapt some of its American rivals. If something so trivial, so tangential, and so temporary can help land an institution on top of the world, is the ranking really telling us anything at all? Herein lies a more vital, a more fundamental question: what are such rankings for?

Oxford

As the rankings spread into the institutions themselves—into their own self-regulation processes, their hiring and enrolment efforts, their spending habits—it behoves academics, institutional leaders, policy makers, and other educational stakeholders to remain cognisant of what we’re trying to achieve. Educational excellence, of course, as the U.K. has been not-so-quietly demonstrating, is not a readily reducible or quantifiable goal.

In my experience tutoring and teaching, giving students grades—numbers on the tops of their essays—does little to foster their ambition and intellectual creativity. Instead it leads to questions about improving scores, and competing with classmates, not about furthering individual understanding. (Never mind that it’s also completely arbitrary: give a student at Oxford a 75%, and they might call their parents to celebrate—unless, of course, it’s a visiting American student, who might be on the phone home to talk about how to proceed with their lawyers to have that changed to a more respectable and expected 95%). While I’m all for assessment and evaluative thinking, quantifying the qualitative seems more often than not to fuel reductiveness. Moreover, as O’Neil demonstrates persuasively, algorithms merely codify the status quo.

With Universities trapped in a cycle of bending and bowing to journalistic whim, and at the mercy of arbitrary events far outside their control, what can be done to break the homogeneity imposed by formalised quantifications of their core missions in teaching and research? To really revolutionize teaching and learning at the postsecondary level, it’s a question the academy needs to confront.

That’s something I’m going to think about tomorrow. [Update: it’s here.]

The Trouble with Lammy’s “Social Apartheid”

Leave a comment

David Lammy’s recent explosion across U.K. headlines with accusations of “social apartheid” aimed at the nation’s oldest institutions of higher education is an incendiary misnomer that does more harm than good, in the grand scheme of promoting racial diversity in Oxbridge. The MP for Tottenham unleashed his fury against newly-released data on the ethnic diversity of Oxford and Cambridge, along with some disingenuous analyses that add fuel to a flame already burning bright for those entrenched the the institutions’ outreach and diversity efforts. Lammy’s numbers are correct, and they are dismal. But his analysis is impoverished, painting an incomplete picture of the admissions landscape. Add to that the charged tone of his tweetable whistle-blowing, and you have a situation that’s likely to do more damage than good to Oxbridge diversity.

As Lammy points out, on average some 378 black British students achieve the highest possible grades in their A-levels, usually a precondition for acceptance for most courses at these institutions. His calls for granularity in data related to racial diversity might well be countered with queries of subject-breakdown in these numbers. The barrier to entry might be 3As, but at Oxbridge as in other top-tier institutions in the U.K., subject choice matters. A student looking to study Economics and Management at university might take 2 of their A*s in Economics and in Management — both offered as A Levels in many state schools across the country. These courses, however, will do little to secure a place at a top university — the applications for which will focus on “traditional” A Levels in subjects like Maths, English, and the sciences. Similarly, a Psychology A Level — increasingly popular among pupils — is not a gold ticket to a university course of the same name. For many institutions, it’s considered a soft choice. When some straight-A students make their A Level choices, they all but write themselves out of consideration.

When Lammy sets up his dichotomy — between the pupils of Eton and the BME students underrepresented in Oxbridge — it’s a false one. Pupils at the country’s most elite independent schools are rounding out their studies ready-made for the pipeline that they’ve been funnelled into for centuries. Yet there is enough truth to the tracing of socio-economic lines over racial ones to make the comparison worth tracking. As I’ve written before, 7% of the U.K. population is privately educated in institutions such as Eton, with that minority taking up an alarming proportion of places in these top-tier institutions (in 2016, that amounted to 42% of the U.K. students attending the institution). Equally telling is the fact that the number of places taken up by students from the top two socio-economic groups hovers at around 80%. To continue with the subject of subjects, there’s substantial evidence that state school applicants make far less of their applications—at least to Oxford.

Despite the institution’s best efforts to the contrary, securing a place at Oxford goes far beyond measures of “academic excellence”; undoubtedly, there is a high academic bar to entry. However, the acceptance rate for applications varies dramatically across subjects. The comparison of private-schooled applicants with their minority counterparts is therefor not like-for-like. As Oxford points out in its own data crunching, state students’ application success rate is significantly affected by their subject choice: they apply disproportionately for the most oversubscribed subjects. Between 2014 and 2016, an average of 34% of state school applications were for the top five most oversubscribed subjects, compared to just 28% of independent school applications (subjects: Economics and Management, Medicine, PPE, Law, and Mathematics). Conversely, their applications for the lower-subscribed subjects (Classics, Music, Modern Languages, Chemistry, and History) were disproportionately low—24% of independent school applicants targeted these places, compared to just 17% of state applicants (all stats here; incidentally, if this is all University administrators are doing with their Tableau licenses, it’s a substantial purchase for some sub-par data visualisation). If you’re rising through a state system that (at least in my experience) earmarks your active and inquisitive mind for the prestigious practice of Medicine or Law, you’re already at a disadvantage.

The problem, then, is made stark by the numbers. But its constituent parts are more nuanced than Lammy cares to entertain. Moreover, as with many things that data helps to highlight, the solutions are not quick fixes. And what happens when the words “social apartheid” splash across the internet is a torrent of articles, and a resurgence of the old, articulating the traumas these elite institutions have heaved upon their minority attendees. Liked reading about Oxford’s social apartheid? Check out more from the Guardian’s “Opinion > Oxford and elitism” category: here’s how one black woman feels isolated and representative of her race; here’s how another found it alienating and elitist. Branching out to other sites? Check out this Times Higher Education piece on one black student who was mistaken for a construction worker. As the news bot algorithms chug away to promote like-material to their readers, would-be applicants are faced with a barrage of information that amounts to “why even bother?” If Oxbridge is not diverse, and I won’t belong, and I won’t be happy. It won’t be worth it—not for me. Unwittingly, Lammy is stoking the anxiety that contributes to many non-elites’ well-documented imposter syndrome, though adding to it the more targeted tinge of racial prejudice. Basically, the argument echoes, it doesn’t get any better.

If Lammy cared thoughtfully about the students he is setting out to lift up, he might focus his energies on directing their attention to the wealth of opportunities made available by Oxbridge colleges, or by his (disingenuously) championed Ivy League examples. He might encourage headlines and Twitterstorms around Oxplore, Oxford’s new platform to encourage critical thinking among aspirant applicants; he might direct underserved schools to seek out The Brilliant Club, or Oxford’s own Early Career Academic Outreach Network, or even the specific college offering outreach to their area. He could laud the students who have made it, and are attempting to extend the ladder beneath them.

Screen Shot 2017-10-23 at 16.07.08

He might even, if he were being honest with himself, connect the South East’s Oxbridge prosperity to their relative national privilege; he might, that is to say, start looking at the government’s own education system, which fails so many bright students every single year.

Lammy is undoubtedly right — “The underprivileged kid from a state school in Sunderland who gets straight As is almost certainly more talented than their contemporary with the same grades at a top public school – they far outshine their peers and would benefit most from an Oxbridge education.” But screaming about it in racially-charged soundbites moves us backward, not forward, in helping them to get there.

The Keble College Tutorial Enrichment Project: A Review

Leave a comment

The following piece is given as it appeared in The Keble Review (2014).

In October 2012, Oxford received over 17,000 undergraduate applications, and subsequently made more than 3,000 offers for entry. Of the offers made to UK residents, 43.2% went to pupils at independent schools—a sector that educates just 7% of the UK school population but accounted for 37.2% of all applications to Oxford that year. Meanwhile, 35% of applications from state schools were for the University’s five most oversubscribed subjects, and just 13% for the five least popular (stats here). Not enough state school pupils are applying to Oxford, and those that are might not be making the most of their application.

Recognizing this, the University as a whole has committed ‘to help bright students make competitive applications, regardless of background’, and as part of this commitment each of the colleges is paired with a geographical region of the UK upon which to focus their own outreach initiatives. For Keble, this is Birmingham, Coventry, and their surrounding areas. That’s why, for the past nine months, I’ve been travelling up to Bishop Challoner Catholic College in King’s Heath. The school is one of a number in the Midlands to have been involved in the pilot year of Keble’s Tutorial Enrichment Project. The project sends early career academics and DPhil students (like myself) into state schools within our region to hold humanities and science tutorials with gifted students during their first year of A-levels. Rather than a fleeting visit to give talks, answer questions and deliver prospectuses, the project aims at a longer-term investment in the pupils’ road to higher education. By working with pupils to foster their academic interests, and with individual schools over time, it hopes to encourage more Oxford applications from the brightest pupils in the region.

At Bishop Challoner I’ve been working with a group of four pupils all hoping to study either the humanities or social sciences at university, though without any firm ideas of a particular university, or even a specific course. At the end of our first tutorial (a whistle-stop tour of reading lists and essay writing followed by a discussion of extracts from Coleridge’s Rime of the Ancient Mariner) I asked for their interests: history, English, French, politics. Faced with such breadth it was challenging to pick subjects for discussion, and to keep all pupils engaged in each session—particularly on areas they knew little about. Across the remaining tutorials we looked at topics as disparate as ‘the Western Canon’ and early modern witchcraft, following the recognizable Oxbridge undergraduate pattern: reading list, essay submission, discussion. Though the group began quietly each week, with prodding and goading each debate was lively, and overran our time.

In July, the pupils visited the Oxford Open Day with a larger group of their classmates. After attending subject talks and different colleges, they came for a tutorial in Keble, including what to expect in the subject aptitude tests and a mock interviews. In the final tutorial, later this month, I will be helping to finalize personal statements—undoubtedly excising exclamation marks and the many synonyms of ‘passionate’. Whether any will apply to Oxford, I’m not sure; but should they decide to have a go—which I certainly hope some do—the Keble Enrichment Project has undoubtedly allayed some of their fears.

“Investigating Options”; or, a demonstration of the relevance of all disciplines

Leave a comment

This blog post originally appeared on the Oxford Early Career Outreach Network blog, and can be found here.

On 4 November, Year 11 pupils from around the country came to Oxford as part of the Pathways Program. The day involved talks, tours, and take-home pamphlets. At Keble, 95 students and their teachers gathered to take taster-sessions in the humanities, and the physical, life, and social sciences. DPhils in English, Statistics, Ecology, and International Relations took to the floor to show what their subjects had to offer.

In honour of Halloween—and to put forward the case for both English and history—my talk investigated Elizabeth Sawyer, historical ‘witch’ and protagonist of the 1621 tragicomedy The Witch of Edmonton. I asked students to reflect on the nuance that literary analysis can bring to historical study, and how it opens up the subtleties of a past we might want to homogenize. Statistician and schools liaison officer Mareli Augustyn discussed chance and randomness. She warned against the lottery, and showed how it really is quite likely that an iPod set to shuffle will play songs by the same artist back-to-back. Harriet Downey then wooed pupils to Ecology with her research into tropical rainforests before James Shires discussed issues of ‘security’. International politics is performative, he argued, and school lunches could easily be ‘a security issue’, if only someone instated the sandwich police to prevent the rioting of hungry pupils.

As a presenter, it was the Q&A that proved most rewarding by offering a great insight into what the next generation is looking for in a degree. One topic dominated discussion, posed by both pupils and their teachers: relevance. What exactly are you going to do with that degree? What comes next? [The theatre scholar in me wishes to note: it started to rain.]

Sitting in humanities corner, I was a little nervous—but, as it turned out, without cause. We discussed the broad career options for the different degrees as well as the relationships between disciplines. Presenters had come from different undergraduate courses and institutions, with time out of education and a variety of career experiences, demonstrating that all degrees open doors. Uniformly, the panel presented a successful degree as one undertaken with genuine interest.

Verdict: if you’re having a crisis of relevance, try telling Y11s why your subject is great (rather than your parents).