Public goods problems are everywhere in software

I claim that many puzzles and failures of the software world can be explained as public goods problems. Judging from Hacker News, software engineers and Silicon Valley people don’t always know this standard piece of economics. So!

Voluntary economic transactions make all parties better off. But not all transactions are voluntary on all sides. If I take the car to work, saving $10 of time, I may cause 10 cents worth of pollution to 1000 people in my neighbourhood. Even though society would be $90 better off if I didn’t drive, those people can’t stop me. My driving is a public bad. Another classic example is overfishing. If everyone can fish from a lake, they will not take account of the cost they impose on others by leaving less fish for them. The lake is a commons. Elinor Ostrom won her Nobel Prize for studying how communities solve these common pool dilemmas.

Software is bloated because your computer is a commons

Computers get faster and have more memory every year. Your computer is many times more powerful than one from 10 years ago. Yet these improvements don’t always seem to translate into faster performance. Many common programs still take seconds to start, or they use gigabytes of memory, seemingly to do the same job they always did.

Your computer’s memory and processing power is a commons. Many different programs run simultaneously on it. Like a fisher on the lake, each software maker faces a trade-off: they can add features, which take up processing power and memory. The cost of this will be shared among all the programs on your computer. The software maker cares about making their program slower, but not about making other programs slower. (Most users are not tech-savvy enough to work out which programs slow the computer down.) The maker faces another trade-off: they can write faster, leaner software, but at a cost in development time and highly-paid engineers. They pay that cost alone, but the faster software benefits all the other programs on your computer. Again, lean software is a public good. The result is that programs are featureful, but bloated.

Software is also bloated because large organizations have public goods problems

Firms are collections of people cooperating to produce something. Often, the interest of different divisions is not aligned. Division X works on feature Y. Feature Y may not actually be very useful: the product would be better, and bring in more revenue, without it. But getting rid of Y would mean shrinking division X. If the firm’s leadership is not perfectly effective, division X may get its way, keeping the useless feature. The industry phrase for this is “shipping your org chart”.

Open source has problems because open source software is a public good

Open source software licenses allow users to view and modify the source code for software, and to share their modifications. Open source software has some notable successes, most famously the Linux kernel which powers much of the internet and every Android smartphone. It also has problems. Open source authors suffer burnout from demanding users. Key pieces of software infrastructure are underfunded, putting security at risk.

Companies can profit from selling open source software. But they know that a rival can take their code and resell it, effectively expropriating the development time they have put in. (This happens.) As a result, investment in open source software is like a public good. There are various imperfect workarounds for this, like selling extra closed-source features, or making money from consulting. In other cases, open source development is done by volunteers, out of altruism or personal passion, or so as to demonstrate their skills to possible employers. Again, this can work, but not perfectly, because altruism and personal passion do not perfectly track consumer demand. Open source hackers write many projects which are useful to hackers, but relatively few which are directly useful to Grandma.

My point isn’t to attack open source: all software markets are imperfect, as we’ve just seen. Thinking of open source as a public good can clarify the problems it faces.

Open standards are public goods too

Much modern software is delivered via the web. The frontend is written in languages like HTML, Javascript and CSS. These aren’t developed by a company but by standards bodies. Anybody may write a web browser which uses them.

These standards are big and complex. They include a way to display a repeating elliptical gradient, for a web page styled like a tie-dye t-shirt; an HTML tag for acronym, plus a completely different tag for abbreviation. Yet they have notable gaps. On a modern smartphone, you can make a payment with a single finger press. After 25 years of the web, most payments still involve typing in a 16 digit number plus several other details — typically, yet mysteriously, stored on a separate piece of plastic. It took decades to develop a standard method to show a video, or a decent way to display things in rows and columns. And while nerds can write and host their own websites, there is no usable standard way for an ordinary person to author content.

What explains these failings? The historical details are complex and fought over, but the fundamental issue is simple: open standards are a public good. Making video or payments easy in HTML benefits all the browser makers, and every website owner. By contrast, Apple takes a 30% cut whenever you pay for an iPhone app. Of course it’ll make payment easy.

Monopoly platforms can fix public goods

Indeed, a public goods problem can be fixed if a single agent has the power to enforce contributions. Classically, the government might tax your driving or enforce limits on overfishing. In the software world, monopolists can solve public goods problems. When Apple writes software for the iPhone, it has an incentive to minimize bloat, because it makes money from sales of the whole platform, not each individual program. For software that Apple won’t write itself, it can provide a “walled garden” App store. It takes a cut of the profits; it can also enforce some quality guarantees. Customers don’t mind this, because Apple mitigates (some of) the public goods problems of bad software.

Facebook lets ordinary people write content on the web and show it to their friends. It provides another platform. Instead of charging users, it extracts information from them and sells it to advertisers. Billions of people are prepared to pay this price.

One web browser, Chrome, has an increasing share of the web browser market, and that browser’s owner, Google, increasingly dominates the HTML standards-setting process. Effectively, the web might become Google’s platform. Like Facebook, it profits from advertising. Google’s monopoly is not, basically, the result of nefarious behaviour. As the biggest player, it has the most incentive to drive the standards forward. This in turn reinforces its dominance.

Policy-makers are right to worry about big tech. But big tech exists — partly — because it solves public goods problems. You cannot address the issue successfully if you don’t get that. You cannot wish these problems away by appealing to the web’s founding ideals of freedom and openness… unless those ideals will help Joe buy his dog biscuits online.