Software is the new Hardware

Plus! Diff Jobs; Vendor Risk; Hardware and Software; Compounding; AI as an Interface; Rotation

Byrne Hobart Byrne Hobart

In this issue:

audio-thumbnail
The Diff July 22nd 2024
0:00
/810.684082

Software is the new Hardware

If you spend much time trying to make the case for deep tech investments, you start to get a real appreciation for what a great business software companies have. There are nominally high fixed costs, but most of those costs become fixed well after the company is established. Meanwhile, the marginal cost of delivering the product rounds down to zero, and that means that companies have enormous flexibility in how low they can price—but interacting with any kind of software product means offering a steady flow of data to whoever created it, so they also end up with lots of pricing latitude on the upside.

All of that is changing. The marginal cost of writing code is dropping, while the marginal cost of running it—to the extent that much of this new code is frequently using inference, or doing other computationally-intensive tasks—is rising. That's the reverse of the economic tailwind that's done so much for software engineers' incomes and software investors' returns. There have been some essays on this that are a bit more apocalyptic, calling for "the end of software," for example. But that kind of bear case goes too far; even industries that die tend to die gradually—the number of full-time travel agents in the US has declined by more than half since the year 2000, but annualize that and it's 3.1% per year. If the average person switches careers at least once in their working life (including switches from individual contributor to manager, or from designer who knows a little code to frontend engineer, for example), then the average pace at which people change careers already lines up with the required pace of forced career changes.

More importantly, it misses something essential about software economics: the complementary good to any given software product is a lot more software! This is part of the mystery of Docusign, i.e. why do they have so many employees? And why doesn't someone else just make a cheaper replacement? You can build a cheaper replacement pretty quickly—the service I use, Dropbox Sign, was launched as a side product by a company whose original business was sending faxes. It didn't seem to require serious engineering headcount to build. What does require that headcount is making sure that the product is integrated with any of X CRMs, automatically archived in Y storage providers, integrated with Z business communications tools. The biggest product typically has the most integrations, and as the cost of launching a new SaaS product declines, the investment required to have the right integrations for 99% of customer use cases actually increases.

But those economics start to look different. They look a lot less like a high-fixed/low-marginal cost business and a lot more like a lower fixed-cost, but higher marginal-cost business. It's easier than ever to build version 0.1, and takes more time to connect with everything that's going to get it to 1.0 status.

That's a case where, paradoxically, a labor-saving tool increases the labor cost associated with any given increase in revenue. But as anyone in AI knows, just as people in video knew a decade ago, there is a difference between costs that are roughly zero and costs that are low in an absolute sense, but high enough that it's possible to have upside-down economics—it's entirely possible to run a video site where the cost of storing and transmitting video exceeds the ad revenue captured (it would be interesting if anyone at YouTube or Netflix periodically went back and looked at how far in the past the video-delivery cost structure would have rendered their current business uneconomic. Netflix and YouTube may in some ways be the same business they were ten years ago, but you're getting a lot more pixel-seconds per dollar than you used to).

Meanwhile, a lower cost to creating new programs also means a shorter half-life for a given piece of code. If APIs are being updated more frequently, there will be more times where it makes sense to deprecate them, and API providers will also assume that for their users, the burden of an update is lighter.

There are some businesses that are nominally a fixed-asset business that depreciates something over time while its use has a fairly low or at least predictable marginal cost, but when the depreciation curve gets fast enough, as with GPUs today, Bitcoin ASICs in the mid-2010s hashing speed race, or military equipment during an active conflict, the depreciation schedule gets massively compressed, and what was cutting-edge two years ago is basically scrap today.

Ironically, in a world where the single cleanest way to bet on the software trend of AI is by buying shares of Nvidia, the economics of software are starting to converge with those of hardware. There will be unpredictable replacement cycles, and higher marginal costs.[1] Hardware can be a fine category, in the right phase of the cycle. Betting on computers in general and PCs in particular used to mean choosing your favorite hardware company, with software either bundled with the purchase, written by the customer, or offered by weird private companies with names like Lotus and Micro-Soft.

That evolution was itself a function of the different economics of hardware and software: the more software commoditized hardware, the harder it was to argue that your particular beige box made Excel or Doom different from someone else's. And software companies had a nice tactical/strategic alignment: the more platforms they supported, the more the decision was about which software to use, with hardware as an afterthought. In the early days of PCs, upgrade cycles were extremely fast, too, but consistency in operating systems meant that upgrading to a new Dell, or switching to Compaq, didn't have much of a learning curve.

Something like that could happen in AI; to the extent that there's a part of software that will still retain more of the economics of software, it will be integration tools that are end-product-agnostic. If you have a business intelligence bot that you're using to check in on your company's performance, it won't matter whether the relevant numbers are in Excel or Google Sheets or a Pandas dataframe or a Postgres database; if the tool is smart enough, it will know how to convert your request into the appropriate query and get an answer. And that means that those underlying products will compete more on technical specs. There may come a time when founders realize their dream of not having to buy an Office license because the docx contracts they get will be reviewed by an automated lawyer, who will run a few changes by them, ask them natural-language questions about key terms, and edit the file, which they will never personally review.[2] These products will have an incredibly strong data loop, but they won't match the economics that operating systems and office software had in the 90s: the more adaptable the interface, the lower the switching cost. Dropping ChatGPT for Claude or vice-versa is easy, at least when they're mostly accessed through either discrete API calls or a chat interface. There may be some room for durable lock-in through product integrations, where Google, Apple, and Microsoft are all investing effort. But even then, those companies can't realistically stop a third party from building some other tool that offers the same services.[3]

The upshot of all of this is that, even as the volume of software produced increases, and even as interfaces get better by adapting to individual users, the economic rewards probably won't be as high. And that ends up being a nice bull case for everything else that technically/analytically-minded people could do instead of writing code. There will be plenty of money in software still, just as there's plenty of money in hardware. But the financial statements won't be as far apart as they used to be.

Disclosure: Long MSFT.

  1. In an accounting sense, it's easy to use cost of goods sold as a proxy for marginal cost, and that does get you the right directional view, but is incomplete. For inference-heavy products, COGS will be higher to support the same level of user activity as before. But even for companies that don't offer AI tools, if they're interacting with companies that ship faster as a result of those tools, or operating in an ecosystem that's more diverse because of them, those marginal costs will show up in opex instead. And not just in the R&D line—sales and marketing spend will also go up if the pace of change is fast enough that salespeople are constantly getting up to speed with new technologies while marketers are constantly updating copy, creative, and where to do the next ad buy. ↩︎

  2. This sounds high risk, and in the current state of the markets it's a very bad idea to let ChatGPT amend your contracts and send them over sight-unseen. But it's probably the case that at a sufficiently big company, there literally aren't enough hours in the day for the CEO to review every single contract that the company, and thus that CEO, is party to. They already delegate, so the only change is who they delegate to. ↩︎

  3. Such a tool would have an uphill battle since those big companies have unparalleled distribution. And most of the time, Fortune 500 customers would prefer one vendor to two. But those same companies can be reluctant to be too dependent on just one company—the fact that they'd trust the big three in this category might be all the evidence they need that it's unsound to give those companies too much of an information advantage. ↩︎

Diff Jobs

Companies in the Diff network are actively looking for talent. See a sampling of current open roles below:

Even if you don't see an exact match for your skills and interests right now, we're happy to talk early so we can let you know if a good opportunity comes up.

Find a Role

If you’re at a company that's looking for talent, we should talk! Diff Jobs works with companies across fintech, hard tech, consumer software, enterprise software, and other areas—any company where finding unusually effective people is a top priority.

Find Talent with Diff Jobs

Elsewhere

Vendor Risk

A small logistics company, NullShip, found a clever way to reduce shipping costs: printing fake shipping labels, often using other shipping companies' brand names. There were plenty of signs that this wasn't legitimate: aside from the pricing, the company apparently requested that payments be sent by Zelle and didn't provide tracking information because it didn't have any. Apparently, this did lead to packages getting shipped, but inconsistently; they got caught because shipments kept disappearing and they wouldn't inform the carriers who actually handled the delivery. It's an interesting category of low-level fraud: the price for shipping an individual package is low enough that a robust auditing process isn't necessarily a good investment. And a shipping label is a physical artifact of a digital product, which also makes it slightly inconvenient to check. Meanwhile, forging items that are worth a few dollars is usually a poor risk-reward, but online a company can scale fast, especially if its low prices still lead to high margins because it's not paying the supplier of the service it offers.

Hardware and Software

An early Diff piece on hardware economics noted that one solution to the difficulty of predicting hardware demand is to bundle it with a subscription to a software product. The makers of the Snoo robot bassinet are locking some of its smarter features behind a $20/month subscription. Economically, that makes sense, since there's a robust aftermarket in Snoos (I've bought and sold used ones). In practice, a $20/month subscription is probably a sub-10% price hike for most customers, since the SNoo is typically useful for a few months before it wears off. So this is closer to mild price discrimination. There might have been less backlash if they'd split the product line into a slightly cheaper basic edition, and a more expensive premium one that had access to those extra features. But it's a durable product that can be used by multiple customers, and pregnancy has a social contagion effect, so the first person in a friend group who buys one is disproportionately likely to know someone who will need it when they're ready to sell. But that's another difficulty in hardware: sometimes, you don't realize the strategic decisions you've made until you're locked in by the existing installed base of your product.

(Incidentally, if you're a new parent, the Snoo is probably an incredibly good investment. Anecdotally, it doesn't work for every kid, and doesn't work every time, but there are few points in your life when you're willing to put such a high dollar value on a few extra minutes of sleep, and the Snoo generally provides that.)

Compounding

Meb Faber has a fun table on the best-performing stocks of all time, or at least the best-performing stocks since 1926. There are some intuitions that come in handy here. First, 98 years of compounding is a very long time, and time rewards steady compounding that avoids drawdowns to zero. The obvious stocks that come to mind are going to be recent high-growth companies, where "recent" means "went public during the careers of people still working," rather than, say, in the last year. The winner is, naturally, a tobacco company: a century of compounding at 16% gives better results than a decade of compounding at twice that. But this also illustrates why this question is more of an academic exercise: most of us are not constructing portfolios meant to outlive us while untouched (though for more on that problem, see here). Instead, we're trying to get good risk-adjusted returns; picking the single best stock doesn't mean much in the context of a broader portfolio.

AI as an Interface

Electronic Arts has been regularly publishing NFL-themed football games, but their new college football game presented challenges around 3d player portraits: there are more college players than professionals, rosters are determined later, and those rosters change more. So it's infeasible to create three-dimensional scans of every player's head. Fortunately, many problems that only worked at small scale turn out to be training data for larger-scale implementations, so they used AI tools to convert headshots into 3D models, and then had designers touch them up ($, WSJ). This is a good example of the complementarity between AI and the workers it's nominally competing with: EA could have gotten a similar result if they'd had a vast supply of cheap, infinitely patient interns painstakingly building models one at a time, but they'd still need skilled professionals to get the details exactly right. Those workers turn out to be the supply-constrained complement to a suddenly-commoditized product.

Rotation

The WSJ digs into the record-setting small-cap rally of the last few days ($). Some of it's a rates bet: smaller companies have fewer financing options and are more likely to have floating-rate debt, while the biggest tech companies have a net cash position that makes their earnings look better when the interest on that cash goes up. (This is probably a smaller factor for these firms, since many of the marginal price-setting investors are likely to use EV/EBITDA rather than P/E to compare companies—that strips out differences in financial structure and interest to focus on operating fundamentals.) But small-caps get disproportionate interest relative to their size, because they're a less efficient market, there's (weak, questionable) evidence that they outperform long-term, and there are more of them. The Russell 2000, in the aggregate, is worth around $6 trillion, against $46tr for the S&P 500. Their small share of aggregate market cap means that a small shift from big to small companies can slightly trim big companies' valuations but cause the smaller stocks to rocket—if investors sell 1% of the S&P 500 to buy small-caps, they're buying about 8% of the Russell 2000.

The real risk for investors is a classic one: treating lucky factor exposure as evidence of skill. Small caps, especially cheap ones, have underperformed for a while, and many of them were getting enticingly cheap. But a rally like this is less driven by individual companies' fundamentals and more by broader positioning. So if you're way up this week, remember that you may or may not be good but are definitely currently lucky.