When the sorting function is broken

April 26, 2019 (1817 words) :: Our current 'meritocratic' sorting function is a terrible way to sort people into a system with highly variable economic rewards.
Tags: meritocracy, the-left, working-in-tech

This post is day 116 of a personal challenge to write every day in 2019. See the other fragments, or sign up for my weekly newsletter.

This came out a little more than a day late, and the next day’s fragment is also late. Please forgive me. I will be caught up soon.

Imagine a world where no one gets to choose their job/role/industry. Instead, this choice is outsourced to some sorting function that places people wherever they’re considered most useful, taking into account all the work that needs to be divvied up for society to function (like, say, the sorting hat in Harry Potter). Only, unlike J. K. Rowling’s supernaturally omniscient hat, this function isn’t clairvoyant. It doesn’t know exactly what you’re best suited for. It tries to guess, based on your performance on various tests, but the tests are always imperfect because they’re designed through trial and error. There’s always still a stochastic element, some uncertainty or possibility of error that can never go away. Still, this system does optimise for efficiency of job allocation as best as it can, and society hums along reasonably well, because people understand what the sorting function is trying to do.

Now imagine that the job you get also determines your economic standing. And not just little bit, either: your quality of life is hugely different depending on whether you land a role in the professional-managerial class, or get trapped in the ever-expanding group beneath whose role is to serve that class. It could mean the difference between being perpetually on the verge of homelessness vs owning multiple homes.

To make things worse, imagine that the economic rewards for a particular job are disconnected from its social value. There’s some correlation in terms of “skill”, at least when measured in a very particular way (time spent in educational institutions, preferably elite), but overall the link between economic value and social value appears fairly tenuous. People still want to do jobs that provide social value, either because they personally love it, or because they recognise the importance of such jobs in maintaining a functioning society, but they know that it might mean resigning themselves to a precarious existence. Consequently, some of these people end up in jobs they don’t really want to do, and which aren’t what society needs, solely because they want economic stability.

Adding to that mix, imagine that the tests for allocating people into jobs are highly varied and poorly managed. Some people are never given the chance to take some tests, or are graded more harshly; some people cheat without consequences or get to bypass them entirely.

What’s the expected result? Well, a sub-optimal allocation of people into jobs, for one. You would no longer have people doing the job they’re most suited for, which would impede the overall functioning of the system. And some people would unnecessarily suffer, because 1) the punishment for not being “sorted” into the top tiers of the system is way too high, and 2) the sorting function isn’t very good, anyway - too easily gamed, and not congruent with what society truly needs.

I’ve written previously about the sham that is meritocracy (both in practice and in theory). My main problems with it come down to the two factors mentioned above: 1) it is used to justify the existence of massive inequality on a scale that is fundamentally unjustifiable; and 2) it is a bad sorting mechanism, partly because of 1), and partly because there is no universal conception of “merit” independent of power. Meritocracy might be a step up from the system it was created to replace, but only a small one, and if you rely on it too long it ends up entrenching the interests of those who are powerful enough to shape the definition of “merit” in their own image.

When I try to interrogate my own political views before I turned drastically leftward, one thing that stands out is that I used to think that the sorting function of meritocracy worked. It’s not that I didn’t notice that sorting existed - how could I have ignored it, when I had daily interactions with low-wage service workers who clearly had much less freedom on the job than I did? Or when I walked past homeless people nearly every day on my short walk home? Clearly there was sorting, and it resulted in some suffering, which was unfortunate. But still, I mostly believed that the sorting function was the best we could do.

So I could feel guilty about the fact that I, as a 21-year-old college student, could make more in one summer (without even working that hard) than a full-time minimum-wage worker would in a year, in most US states. I could feel bad about getting to bring guests to my workplace’s free cafeteria while the workers serving me food were forbidden from doing the same, even though they could have used it more. I could sympathise with long-time residents of San Francisco being displaced because techies like myself were able to spend $3,000+ a month on a one-bedroom luxury apartment.

Or I could view it as my just desserts - my prize for having worked hard, and just being like innately capable, I don’t know. I had a profound faith in the ability of the system to sort people who had “merit” (i.e., people like me) into their rightful place within the system, where they would be justly rewarded. Those who weren’t as talented would end up having to settle for the jobs that paid poorly and offered little freedom or possibility of advancement, possibly for the rest of their lives. I considered this unfortunate, but a reasonable price to pay for a well-functioning society.

One of the most profound personal realisations I’ve had in the last few years is that just because something worked for you, doesn’t mean it works for everyone. Just because a system feels fair to you, doesn’t mean it’s actually fair in general.

This is both incredibly obvious and startlingly difficult to fully absorb. It’s really hard to absorb in the case of an all-encompassing system, a system as invisible and as naturalised as the air we breathe, whose alibi for entrenched hierarchy is an appeal to intelligence as a sorting mechanism. It’s especially hard to absorb if, growing up, you were constantly told how intelligent you are and presented with a barrage of scientific-looking evidence to confirm this assertion.

As a result of this, you’ll probably end up believing that the system is good. Is it really, though? All you really know is that it’ll be good for you, because you’ve been given a comfortable place within it - whether it’s good universally is hard to judge. But enough people seem to believe that the system is good that it seems like a reasonable thing to believe. And anyway, by this point, your whole identity is tied up in believing that you deserve your fortunate place in the system.

I suspect this sort of view is fairly common among meritocracy-worshipping liberals (like I used to be). When confronted with a clearly unequal system, where one’s place in the system is determined by some sorting function, there are really only two reactions. The more reassuring one is to assume that the sorting function is good, or at least good enough, because otherwise why would it be the function in use? Surely if it weren’t that good, then the people in charge would have gotten rid of it already.

Sometimes it’s useful to look at the past for comparisons, in order to see how the present might hold up in the future. History gives us many examples of unequal, hierarchical systems for determining where people should allocate their time. Even the worst system could be defended on seemingly rational grounds: people are put into the position they are most suited for, and even if it seems unfair because some are denied freedom, it’s still the best way to optimise for overall productive efficiency. You could say this about historical expectations for women to perform reproductive labour, or slaves picking cotton to enrich plantation owners, or - to use a more contemporary example - gig economy workers delivering food or driving cars to make life easier for the professional-managerial class.

That’s not to say that these jobs were/are not worth doing. It’s to say that the circumstances under which these jobs were/are being done are cruel and unjust, because of the lack of choice of those doing the work.

Now, the equivalent sorting function under modern-day capitalism is less rigid, for sure. But it’s still noticeably bad. Consider how much of your prospects in life are rooted in characteristics entirely outside your control, like race, place of birth, family wealth, etc. And even the things that are somewhat in your control (like whether you get into an elite college) are still highly determined by things you have little control over, especially when you’re young. The elite college admissions scandal should serve as a wake-up call for those who still believe that the system is worth defending: if the system is this easily manipulated, then it is clearly not good enough. It is intolerable, and we must do better.

As competition for the few decent jobs remaining continues to rise, I think we’ll see a corresponding rise in attempts to improve the sorting functions that determine who gets access to those jobs. We’re already seeing automated coding interviews for software developers, and various algorithmic attempts to screen/rank job applicants, with all the biases you’d imagine.

In my view, this is the wrong approach. We don’t need more quantification of “merit” as a way to gatekeep the coveted roles in a system characterised by stark inequality. Instead, we should take a step back and consider what the ultimate goal is of this sorting function in the first place. Is the goal to wring every last bit of work out of people, even if it’s the sort of work the world does not need? How important is it to match the most appropriate candidates with job openings if the jobs aren’t all that well-designed to begin with? Why is the endless pursuit of capital accumulation a worthy justification for a system where some people are forced to do menial labour they hate just so they don’t starve, while others sit in front of computers, doing their best to make their already-wealthy boss even wealthier?

My ideal is a world where economic equality has virtually disappeared, and people choose the work they want to do based on a combination of 1) what they enjoy doing; 2) what they’re naturally good at; and 3) what society needs. This is obviously not something that can be achieved overnight, not least because it’ll require radically overhauling the way people think about something as culturally ingrained as work. Still, it seems to me like the only system for allocating work that’s worth striving for.

« See the full list of fragments