Functional Institutions Are The Exception

Every great company is unique, but there are a few things that every business must get right at the beginning . I stress this so often that friends have teasingly nicknamed it "Thiel's Law": A startup messed up at its foundation cannot be fixed.

-Peter Thiel, Zero to One, page 107

Within nearly every institution bigger than a dozen people, insiders are often resigned to how hard it is to get things done. They maintain a coordinated competence at only barely above a failure level at which they wouldn’t even persist in the world. Perhaps worse, many institutions that persist, despite failing at their formal purposes, last for a surprisingly long time; they’ve fallen, unwittingly or not, into new reasons for being. Unprofitable companies & declining nations often last longer than their critics remain solvent.

Most things fail. Most things that exist have avoided failure. So far. The institutions we do see look functional mostly because of the selection effect, not because humans are particularly good at making them work.

In my research, I found something that puzzled me: in any given type of institution: state, church, for profit or non-profit, there are some that outperform the others by orders of magnitude. This is true even when comparing only those that have similar material wealth, human capital, and formal structures. I usually compare them in their ability to reshape the world, either in service of their formal purpose, their informal purpose, or merely just perpetuating themselves. Sometimes the artifacts they produce, as side effects or instruments, differ exceptionally in quality. Regardless of the exact measure, exceptional institutions do exist, but they are rare...

An elegant explanation that matches noted observations is that everything is broken. When something works the way it should, it appears exceptional. It’s not that starting off, they have more material wealth, nor is the quality of most of the people involved there much higher than their competitors. It is simply put together properly; the cogs and gears fit.

This explains Thiel's Law: A tornado cannot assemble a Boeing 747 by passing through a junkyard. The spontaneous unplanned generation of purposeful institutions out of non-purposeful ones doesn’t happen. The machinery, if it functions, was assembled by someone with good judgement; the Founder. If he has the knowledge and ability, the founder usually quickly builds something that works. He isn't going to be better positioned later in the story, though the environment might become more favourable for whatever working thing he has constructed. So, usually things that work do so from the start.

Most institutions are broken

Normal institutions don’t work very well, but they spasm in the general direction of their formal goal. Often, like in education or medicine, their public face is primarily there to be a plausible facade that the individuals involved in the aggregate find useful to maintain. Other times, they even achieve modest results. This provides a challenge to our theory of rare functionality.

Given the assumption that institutions don't actually work, in the sense of being machinery that performs a task, how do we explain this? The reality, under whatever form these organizations appear to take, is by default one of a poorly-run social club--a group of people with no stronger drive than fulfilling some of their social needs. Unfortunately, it usually isn't well optimized even for that; the formal purpose, when too weak to exert pull, becomes an obstacle. Many members don't directly notice this, or have to take symbolic measures to plausibly pretend not to notice.

Order, either spontaneous or designed above a very low level, doesn't emerge on its own. Specialization is haphazard. People often choose their fields based on their social needs that are not tightly correlated with achieving goals or the self-preservation of the wider entity. Bottlenecks result in much wasted effort and local information being thrown away needlessly. Much effort is also lost in communication and political struggles. The number of people involved is usually also too small to organize via market mechanisms, at least internally, and market mechanisms require certain working institutions to maintain them anyway.

In such an institution, efforts don't multiply each other, but merely accumulate linearly. The sum of this activity is a noticeable but very weak optimization force. The optimization force, together with naturally occurring hierarchies, is quite sufficient to govern small tribes under conditions quite similar to those prevailing for most of our evolutionary history. But most institutions try to be something different.

Sometimes, when any move to assemble working machinery will be met with political opposition, the environment of such an institution can be worse than nothing. A talented Founder can still walk into it and succeed. This happens by defeating everyone else at the relevant conflicts, in such a one-sided way that he establishes peace, a peace in which he can build. This represents an additional constraint in an already difficult process.

Working order is fragile

When order emerges it can still be a dysfunctional one. A well-built machine can still be a poorly designed one, based on faulty assumptions or incomplete knowledge. It can also be merely unlucky. Often, when there appears to be an outgrowth of impressive order without impressive results, it is a deception. Depending on the scale, this is sometimes maintained by charismatic individuals or by a smaller and less impressive order of coordinated and enforced deception. "Comrades, we have outperformed our quota!"

The order around us is also fragile, and often more an illusion than a reality. Examples of this are numerous. The formal charters of companies never capture the reality of the office politics actually constraining and initiating actions. Areas that rely only on the police for safety tend to be dangerous. An army’s morale is fickle. Should it falter, it reveals that the command structure has rested on quicksand. Soon after, it becomes unable to function.

Why are there so few true founders that can assemble this machinery? There are many preconditions, but I think the key one is planning, defined here as considering your actions in advance and improving the entire sequence, rather than just one step at a time. This activity is the exception rather than the rule.

We usually fail at it for many reasons. One is that we don’t have much time to figure things out. The world is large, and each of us has only a few decades at best in our prime.To make matters more difficult, much of the thought we do engage in is about making other humans treat us nicely or give us the things we want, rather than about discovering what is true. Desperate for social survival, we explicitly or implicitly agree to pay the long term price for immediate improvement.

The plan ceases to be a map of actual future action towards the goal it claims to have. Rather, it becomes an agreed-upon lie, aimed at solving the immediate political problems of the people collaborating. This means the activity called "planning" is often an exercise in persuasion rather than engineering, with predictably bad results.

Given relevant knowledge, complying even with a benevolent plan, one that eventually fulfils our needs, requires us to postpone their gratification, the self-domestication of mankind has barely begun to imprint this ability on the feral human animal. Unfortunately, self-domestication also imparts a strong urge towards conformity in thought. This is a useful feature in the components of the machine, as I will explain, but a bug for any would-be designer.

How we control coordination costs

An externality reducing the possibility of local planning is uncertainty about people’s behaviour. How can we overcome it without paying the high cost of deeply understanding others? We sometimes work around it by intentionally simplifying our behaviour -- that is, making it match a highly formulaic and even ritualized form, in order to increase predictability and standardize interactions. One example of this is what is usually called professionalism, another would be courtesy, another, the notion of being law-abiding. The most important, and arguably quite complicated in itself, is virtue. Failure to maintain all of these is apparent and common. When a community does merely marginally better at them compared to most, the pay-off is large. Judging a society by the quality of its people,especially its best, is a reliable proxy for civilizational adequacy.

When we do manage to correctly understand strangers, we still can't be sure they don't mean us ill. So we tend to assume that they do. In the absence of forces pushing against this, this is usually correct. We try and ameliorate this by self-sorting: making sure those we talk and interact with, are as similar to us as possible. This can work well, since even slight preferences end up almost perfectly sorting them. We also put effort into standardizing any new humans we come across, either by capture or manufacture, with measures like schooling and rewarding conformity.

Difficult communication and imperfect models of others entail uncertainty. Scarcity as well as justified assumptions of ill intent result in conflict. Ultimately, if no other means suffice, people reach first for local politics and then violence. As those struggles proceed, a costly process of reduced uncertainty takes place. Three men can keep a secret, if two are dead.

Even if we understand how they tend to think and what they are like, our allies remain hard to understand -- especially if they have thought about a subject with which we are unfamiliar. Enemies will try to disguise themselves as allies. Our coordination costs are typically high, and we pay them in forms so familiar that they are usually not noticed. There are also high costs to figuring out who is competent and who isn't. Relying on others to help map out how the world works-- a workaround to the limitations of our short-lived, small minds-- is only a sporadically good idea and has failures that are hard to detect from the inside. Epistemically sound collaboration is rare.

A great man is someone with a secret and a plan

Our puzzle leads us to an interesting conclusion. Starting at exceptional institutions as unexplainable anomalies, we saw that functionality is the anomaly, and then concluded the founder capable of bypassing some of the limitations of a typical human mind is an anomaly. Only once assembled and functional does the machine possess the capacity for purposeful self-improvement, beyond the founder's design. Sufficiently ancient and long-lived ones, radically so. Improvements in the social technology could conceivably accelerate this process. Social science is powerful when real. .

Great man history, disparaged in academic consensus starting in the late 19th century in favour of socio-economic forces, deserves a second look. Great forces are perhaps only unleashed by particular great minds. The recasting of the pre-modern approach as “Great Minds History” provides an uneasy prophecy -- one that extends beyond the human era. Those who find secrets, that is, correct and special knowledge about the world and opt to pursue the cognitive tools needed for planning, posess the building blocks of the next machine.