Twenty-five years ago, the prophets of the internet revolution preached a simple gospel: the middleman was doomed. Why pay a travel agent when you could book flights directly? Why pay a real estate agent’s commission when you could list your house online? Why pay a stockbroker’s fees when you could trade from your laptop? The web would create a frictionless world where producers and consumers dealt directly, cutting out the intermediaries who had long extracted rent from every transaction.
It was a seductive vision—and almost entirely wrong. Today, real estate agents still command commissions averaging 5.4 per cent of home sales, barely changed from the pre-internet era. Travel agencies comprise a 458.7 billion US dollars global market expected to more than double by 2034.
Insurance brokers and agents represent a 240.5 billion US dollars industry growing at nine per cent annually. Even where technology has transformed industries—such as stock trading, where many brokers now offer zero commission trades—intermediaries simply evolved new revenue models, extracting fees through payment for order flow and interest on cash balances.
More perversely still, the internet has birthed a new generation of super-middlemen who dwarf their analogue predecessors. Amazon didn’t eliminate retail intermediaries; it became the ultimate one. Uber didn’t remove transport brokers; it became a global taxi dispatcher. Google and Facebook didn’t disintermediate advertising; they became the world’s most powerful ad agencies.
The digital revolution didn’t kill the middleman—it crowned new kings.
The promise unfulfilled
The failure of disintermediation represents one of technology’s most spectacularly bad predictions. Early internet evangelists confidently proclaimed that digital networks would create direct connections between buyers and sellers, eliminating the need for go-betweens who added cost but little value. The logic seemed unassailable: why pay someone to do what technology could accomplish for free?
The answer, it turns out, lies in a profound misunderstanding of what intermediaries actually do. The internet did indeed reduce certain transaction costs—searching for products, comparing prices, accessing information. But these were never the only services that middlemen provided, nor necessarily the most important ones.
Consider the real estate agent, a profession that represents disintermediation’s most glaring failure. Armed with property databases, digital photography, and online listing services, any homeowner can now market their property to millions of potential buyers with a few clicks. The technology exists to make agents as obsolete as lamplighters. Yet 90 per cent of sellers in the US still use agents, paying high commissions. There are no accurate figures for Europe, but the percentage of sellers using agents is estimated to be over 80 per cent.
What’s certain is that despite decades of technological advancement and periodic attempts at disruption, the sector’s rent-seeking structure remains stubbornly unchanged.
The persistence of real estate agents reveals not the internet’s limitations, however, but the power of regulatory capture and manufactured complexity. Unlike insurance brokers who often have to deal with genuinely labyrinthine regulations, most estate agents perform routine administrative tasks that any competent adult could manage.
Their ‘expertise’ often consists of little more than access to a set of keys and knowledge of jargon designed to mystify straightforward transactions.
The complexity trap
Far from simplifying commerce, the digital age has often made it more bewildering. The proliferation of options that was supposed to empower consumers has instead paralysed them. Booking a simple holiday now involves choosing from thousands of flights, hundreds of hotels, dozens of car rental companies, and countless activities.
Indeed, the travel sector illustrates the middleman paradox perfectly. Despite decades of online booking tools, traditional travel agents not only survive but thrive, and the global market is expanding. The industry has split into two tiers: online platforms for simple transactions and human agents for complex itineraries. Modern travellers may book routine flights themselves, but they still turn to experts for multi-city business trips, family safaris, or honeymoon planning.
The same pattern emerges across industries. Insurance brokers have found new relevance as policy options multiply and regulations grow more complex. Investment advisers flourish as financial markets become more sophisticated and accessible. Even in retail, where Amazon has conquered vast territories, specialist intermediaries proliferate in categories requiring expertise—wine, art, industrial equipment, or technical services.
This isn’t evidence of consumer irrationality or technological failure. It reflects a fundamental truth about modern markets: abundance creates complexity, and complexity creates demand for curation. The internet delivered on its promise of infinite choice. It simply forgot to mention that infinite choice can be a curse as much as a blessing.
The trust premium
Perhaps the internet’s greatest miscalculation was assuming that technology could replace trust. The early web’s pioneers, steeped in engineering culture, believed that good information and efficient systems would naturally generate good outcomes. They underestimated humans’ stubborn preference for dealing with other humans, especially when large sums of money are involved.
Research on marketplace disintermediation shows that even when platforms successfully connect buyers and sellers, the participants often migrate to direct relationships once trust is established.
This ‘platform disintermediation’ creates a another paradox: the more successful a marketplace becomes at building trust, the more likely it is to lose customers to unmediated transactions.
The trust premium extends beyond individual transactions to systemic confidence. Financial markets, despite their digital transformation, remain heavily intermediated because investors value institutional oversight.
Even as stock trading commissions fell to zero, brokerage firms continue to profit by providing regulatory compliance, research, and customer service that individual investors cannot easily replicate.
The network effect nirvana
The internet’s most successful companies succeeded not by eliminating intermediaries but by becoming better ones. Platform businesses like Amazon, Google, and Facebook leveraged network effects to build moats wider than any traditional middleman ever enjoyed.
These digital platforms discovered that connectivity creates its own value. Each additional user makes the platform more valuable to all other users—a dynamic that traditional intermediaries could rarely achieve. A local real estate agent might know all the buyers and sellers in their area, but they cannot easily scale that knowledge to other markets. Amazon, by contrast, can use data from millions of transactions to benefit every customer and merchant on its platform.
The result is a concentration of intermediary power that would make nineteenth-century railroad barons envious. Amazon captures roughly 40 per cent of US e-commerce, Google processes 90 per cent of web searches, and Facebook reaches nearly half the world’s population. These platforms don’t just intermediate transactions; they control the infrastructure through which modern commerce flows.
This concentration creates a new form of disintermediation—not between buyers and sellers, but between old intermediaries and new ones. Traditional advertising agencies found themselves disintermediated by Google and Facebook, which offered direct access to audiences and better measurement tools. Local newspapers lost classified advertising and job listings, and most died. The revolution happened, but as is so often the case, the revolutionaries became the new establishment.
The regulatory shield
Government regulation has preserved intermediary roles in ways that early internet theorists did not anticipate, but with crucial distinctions between legitimate oversight and regulatory capture. Financial services and healthcare genuinely require professional oversight given their complexity and potential for catastrophic consumer harm. Insurance regulations mandate education and bonding requirements that reflect real expertise needs.
Real estate represents regulation’s darker side: professional licensing requirements that serve industry interests rather than consumer protection. Estate agent licensing (at least in the US) typically requires a few weeks of courses covering basic contract law and property terminology—hardly the extensive training that justifies professional monopolies in medicine or engineering. Yet these minimal requirements create legal barriers that exclude innovative competitors and preserve traditional commission structures.
Regulation also fragments markets in ways that favour local intermediaries over global platforms, but again with important distinctions. Insurance requirements genuinely vary by state due to different legal frameworks and risk profiles. Real estate regulations, by contrast, often reflect local agent associations’ lobbying power rather than meaningful jurisdictional differences in property law.
The regulatory environment explains why some of the most successful disintermediation has occurred in previously unregulated markets. Taxi services, hotel bookings, and retail sales faced fewer artificial barriers to disruption than real estate, which remains protected by laws that serve no purpose beyond preserving incumbent profits.
The cost mirage
The economic case for disintermediation also proved more complicated than its advocates imagined. While technology reduced some transaction costs, it imposed others. Building and maintaining digital platforms requires enormous capital investments in software, data centres, and customer acquisition. Providing customer service, handling disputes, and managing fraud may be less visible than a traditional intermediary’s commission, but these costs are no less real.
Payment processing, logistics, marketing, and customer support represent hidden costs that someone must bear. Traditional intermediaries bundled these services into their commissions; digital platforms often unbundle them but rarely eliminate them. The result may be lower headline prices but not necessarily lower total costs.
Take the travel industry, where online booking platforms now charge airlines and hotels substantial commissions while offering ‘free’ services to consumers. These platforms have recreated the traditional intermediary model in digital form, complete with opaque pricing and complex fee structures. The main difference is that consumers see fewer of the costs directly.
Stock brokers exemplify this cost-shifting. ‘Zero commission’ trading is subsidised by payment for order flow, margin lending, and premium services. The intermediary’s cut hasn’t disappeared; it has been redistributed across different revenue streams and customer segments. Sophisticated traders may benefit from explicit pricing, but casual investors may pay more through hidden costs.
The scale stratagem
The most successful digital intermediaries have achieved something their analogue predecessors never could: global scale with local relevance. Amazon’s marketplace serves millions of merchants and billions of customers while providing personalised recommendations to each user. Google’s advertising platform connects global brands with local audiences through automated systems that would be impossible to replicate manually.
This scale advantage creates winner-take-all dynamics that concentrate intermediary power in ways that traditional markets never experienced. A local real estate agent competes primarily with other local agents; Google competes with the entire world for attention and advertising dollars. The result is unprecedented market concentration among digital intermediaries.
Nevertheless, scale also creates vulnerabilities. Large platforms face regulatory scrutiny, customer revolt, and competitive threats in ways that small local intermediaries do not. They must serve diverse markets with standardised tools, creating opportunities for specialised competitors to serve niche segments better.
The future likely belongs to a hybrid model where global platforms handle routine transactions while local specialists manage complex or high-touch services. Travel booking already shows this pattern, with online platforms dominating simple reservations while human agents focus on complex itineraries and premium services.
The personal touch premium
Perhaps the most enduring advantage of human intermediaries is their ability to provide emotional support during stressful transactions. Buying a home, choosing insurance coverage, or planning a wedding involves anxiety that no algorithm can address. The intermediary’s role as counsellor, advocate, and hand-holder may be impossible to quantify, but it is clearly valuable to those who pay for it.
This emotional dimension explains why intermediary industries often segment by customer sophistication rather than transaction complexity. Experienced investors may prefer low-cost online platforms, while novices value full-service brokers who can explain options and provide reassurance. Business travellers may book routine trips online while using corporate travel agents for important meetings or complex itineraries.
The Covid-19 pandemic reinforced this pattern by highlighting the value of human expertise during uncertainty. Travel agents found new relevance helping clients through a web of changing restrictions, cancellation policies, and health requirements. Insurance brokers became more valuable as businesses faced novel risks that standard policies did not address.
The middleman’s last laugh
The failure of disintermediation offers several lessons for understanding technological change. First, genuine intermediate functions—those involving real expertise, complex judgment, or meaningful advocacy—are often more valuable than they appear to outsiders. Second, trust and specialised knowledge cannot be easily automated or replaced by information alone. Third, successful technological disruption often means becoming a better intermediary rather than eliminating intermediation entirely.
But the persistence of real estate agents offers a darker lesson: that artificial barriers can preserve unnecessary intermediation indefinitely. While travel agents evolved to focus on complex itineraries and insurance brokers developed new specialties in cyber risk and regulatory compliance, estate agents have simply lobbied harder and fought dirtier to maintain their grip on routine transactions.
The internet has indeed transformed how legitimate intermediaries operate, forcing them to provide more value, greater transparency, and better service. But it has failed spectacularly to eliminate rent-seeking intermediaries who add little value beyond regulatory compliance. If anything, the digital age has increased demand for genuine intermediation by creating more options, more complexity, and more uncertainty—while simultaneously preserving space for parasitic intermediaries who contribute nothing but artificial complexity.
The great disintermediation that was supposed to characterise the internet age has instead become the age of super-mediation. A handful of digital platforms now intermediate more transactions than any traditional middleman ever imagined possible, while industries like real estate prove that even the most obsolete intermediaries can survive with sufficient regulatory protection.
The middleman, reports of whose death have been greatly exaggerated, is having the last laugh after all. The internet promised to eliminate the rent-seekers who stood between producers and consumers. Instead, it created the most powerful rent-seekers in human history whilst preserving the most unnecessary ones from a previous era. The war has been won—just not by the side that expected to win it, and with some remarkably undeserving survivors along for the ride.
Photo: Dreamstime.
