When it's time to refactor

March 18, 2019 (1574 words) :: A programming analogy for how we could see the failures of modern-day capitalism.
Tags: working-in-tech

This post is day 77 of a personal challenge to write every day in 2019. See the other fragments, or sign up for my weekly newsletter.

(I’m wary of relying on programming analogies too much, because I want my fragments to be accessible to those without tech backgrounds, but in this case I think it might work anyway.)

Imagine you’re a programmer at a typical successful tech startup: started small, grew quickly. Because growth came quickly, the codebase has a lot of technical debt, the result of architectural decisions that prioritised short-term metrics over long-term sustainability. At the time, that made sense; but now the company is big, and the product is used by millions of people, and so the cracks are starting to show.

Users are reporting bugs all over the place. Things crash every now and then and rebooting is a painful, manual process. Certain user actions seem to be triggering memory leaks and data loss.

The worst part is that fixing things is getting harder and harder. The design of the codebase is simply not suited for the scale of the product as it exists now. That’s not to say that the original decisions were wrong, per se - it’s just that the relevant technology has improved a lot since then, and in any case the priorities were different back then, when you were running experiments and adding features to keep up with demand. Now, though, you just need a stable product.

You know why things are so buggy: you can pinpoint the exact architectural decisions that need to be undone, and you’ve come up with alternatives. There will still be bugs, of course, but the new system you’ve designed is much more suitable given the current conditions, taking into account advances in technology as well as the fact that so many people now rely on the functioning of your product. You’re no longer in “move fast and break things” mode - a mode that you never supported in the first place, but there’s no point going into that now, since you can’t change the past. But you can change the future.

You tell the company about your plan to refactor the codebase. The CTO sounds almost offended when he hears your plan, because he was the one who made most of the architectural decisions back in the day. He tells you that the codebase worked well enough to grow the company from nothing to millions of users, and it would be a mistake to change things now - it would be a waste of resources, and anyway you have no proof that your system would be better. He recites anecdotes about other companies that have tried to refactor their codebases at a similar stage and failed catastrophically as a result.

You protest, surprised at the response. You know you’re right, but the CTO is so adamant that you’re not that you start to doubt yourself. So you spend some time talking to others, to see what they think. You tell them about the architectural changes you’d like to make, and how that would change the codebase, the environment that runs it, and workflows for fixing bugs. You acknowledge that it would be a huge, maybe even revolutionary, transformation, and would require some getting used to it, and like the CTO said, there isn’t a guarantee that the new system would be better (but you think it will).

Not everyone cares, or fully agrees with your analysis, but the people who work the most closely with the codebase support you, because they’ve been begrudgingly implementing hacky fixes and they’re sick of accruing all this technical debt. They don’t enjoy constantly being paged at 3 in the morning to deal with site outages that are an incredible pain to debug, much less recover from. They murmur that the CTO hasn’t checked in a single commit in the last 6 months, and probably doesn’t even know how janky the patches have been, lately. He never gets paged at 3am, either.

You try talking to the CTO again, but you quickly realise that you’re not going to get anywhere. He’s happy with the existing system, partly because it makes him feel good about himself, and partly because he never has to directly bear the consequences of the system’s increasingly numerous failings. No matter how rational your argument seems to you, and most of your co-workers, it’s never going to work on him, because he has different values than you and so operates under a different mode of rationality. He doesn’t really care if the employees are overworked or unhappy, because he’ll still get his millions anyway.

What can you do, at this point? You could quit and find another job, but this is the only employer in your town that you’re qualified for, and you’re too settled in this town to move elsewhere. You can give up your refactoring crusade and resign yourself to things steadily getting worse, in the hopes that it’ll get better down the line.

Or you can stay and fight: build the power you need in order to make the changes that need to be made. Things won’t get better by themselves, and knowing that “things could be better” is only the first step toward actually make that a reality.

The imperfect analogy I’m trying to draw here is between a codebase and the rules of our socioeconomic system. Think about all the historical architectural decisions that have led to modern-day capitalism being the way it is: the profit motive, the primacy of markets, publicly-traded companies, easy credit, digital advertising, intellectual property, wage-labour … these are all contributors to how the system works today, and many of these features have an element of path dependency which accounts for their resilience.

These decisions were often reasonable at the time, and some of them are still reasonable now. But some of these features have long outlived their usefulness, at least in the sense of providing utility to the majority, and are purely kept around because the minority who benefit from them are blocking the avenues for potential change.

Probably the most well-known proponent of “refactoring” our socioeconomic system is Marx. And the thing about plans for refactoring is that you can’t really judge them a priori - that something requires a revolutionary and maybe even disruptive transformation isn’t necessarily an argument against it, if the alternative is worse. On the other hand, it could be a transformation that leaves us worse off than when we started. We can’t really know in advance, is the problem; we can only theorise and hypothesise based on our understanding of the system, which everyone sees in a slightly different way.

If you see the system as mostly working, with maybe a few failures that would be inevitable anyway, then refactoring is likely not high up on your list. You might acknowledge that it will make things better, but you also recognise that time’s arrow only goes in one direction & we all live finite lives, so just because something “could be better” doesn’t mean it’s necessary to spend a lot of time on it. But if you see the system as utterly failing, to the point where you think you’ve identified the structures that need to be changed to make it fully work again, then refactoring begins to feel like more than an option - it feels like a necessity.

To use an overly personal example: every day, for breakfast, I eat a bowl of plain oatmeal with a dash of table salt. It’s mostly tasteless and definitely not the most enjoyable thing. I’m sure that it could be better if I added like peanut butter or milk or bananas or something. I could. But it’s a lot of effort, and mildly salty oatmeal isn’t that bad, in the grand scheme of things.

To continue the breakfast example: I also tend to have a cup of tea in the mornings, too. Usually it’s black tea (English breakfast) with a dash of milk. I’ve tried having it without milk on several occasions, for reasons that always turn out to be misguided: saving money, saving space in my grocery haul, saving calories. But I always end up adding milk in the end because black tea without milk is unbearable. I just hate drinking it. I mean, it wouldn’t kill me, but I would be very unhappy. In this case, the initial situation (plain black tea) is so bad that finding an alternative (adding milk) becomes more pressing. It’s not simply that it “could be better”; it’s that it has to be better.

Of course, there are even worse food-related fates than drinking tea without milk: if you don’t cook your chicken properly, you could die from salmonella, for instance. It’s a spectrum.

Hopefully you see where I’m going with this analogy. Where does capitalism (defined as the “institutionalised social order” in which we live, following critical theorist Nancy Fraser) fit on this spectrum? A lot of it depends on personal circumstances as well as values: how does it affect you personally? Are you suffering, or are you mostly shielded from its consequences because of your place in the system? And how does it affect the people you care about, either in an abstract sense or in a personal sense?

Making the argument for refactoring isn’t easy. It’s a leap of faith, in the end. Still, better to do it when it’s necessary than to let the whole company (i.e., our planet, not to mention the people who live on it) collapse.

« See the full list of fragments