Human risk and total outcomes

There is something strange about this idea, from Peter Thiel's article The Optimistic Thought Experiment:

Let us assume that, in the event of successful globalization, a given business would be worth $100/share, but that there is only an intermediate chance (say 1:10) of successful globalization. The other case is too terrible to consider. Theoretically, the share should be worth $10, but in every world where investors survive, it will be worth $100. Would it make sense to pay more than $10, and indeed any price up to $100? Whether in hope or desperation, the perceived lack of alternatives may push valuations to much greater extremes than in nonapocalyptic times.

Thiel conjectures that the market bubbles of the past few years are in fact episodes in a single bubble. He then argues that the bubble reflects a sober evaluation by the market that there are no moderate outcomes in our future, but only the extremes of total globalization or apocalypse.

Market bubbles seem to me to be evidence not of sober long-term thinking, but rather of panicky getting while the getting is good (see Hyman Minsky's view). In any case, Thiel's hypothesis lacks explanatory power because (on the evidence he presents) it can't be separated from an abundance of others.

But back to the strange idea: We are used to thinking about the fair price of an asset as being the expected present value of the asset's future cash flow (see Wikipedia), and the market price as being the market's judgment of the fair price, mistaken (by misinformation or mania) as it may be. When taking the expectation, how could we be justified in filtering out the apocalyptic outcomes? In the extreme—suppose we believe the world has only a 1:1000000 chance of surviving—shouldn't we stop overpaying for stocks and start stockpiling ammunition?

I feel similarly uncomfortable with Scott Aaronson's idea of "anthropic computing", from NP-complete Problems and Physical Reality:

There is at least one foolproof way to solve 3SAT in polynomial time: given a formula ϕ, guess a random assignment x, then kill yourself if x does not satisfy ϕ. Conditioned on looking at anything at all, you will be looking at a satisfying assignment! Some would argue that this algorithm works even better if we assume the many-worlds interpretation of quantum mechanics.

Are we justified in filtering out the suicides when taking the expectation of this operation? How is one supposed to feel—as a surviving self, looking at a satisfying assignment—about the unfortunate others who guessed wrong? (The many-worlds interpretation seems to make meaningless all moral choices; in another world, I made a different choice.)

Humans are very bad at evaluating risk: we obsess over vanishingly improbable outcomes (terrorist attack) but ignore likely ones (car crash). I think this weakness has something to do with the strange idea above. Market participants live in a mostly frequentist world: they can think in terms of expectation because they will live to trade another day. (Of course there is always the possibility of a black swan and blowing up.) But individual humans have each only one life, and must reason about it as Bayesians. (To paraphrase E.T. Jaynes, the expectation is hardly ever expected.)

Bayesian thinking, it seems, is hard to do. Faced with the possibility of a total outcome (that is, an outcome which forecloses on the future; that is, an outcome in which we die), we either obsess over it or ignore it. Whether in hope or desperation, we hitch our minds to a single future, because a single future is all we are given.

Debugging meat

I found these nice papers via Lambda the Ultimate:

Each argues that in order to understand a biological system, we must view it at the right level of abstraction.

Lazebnik points out that we cannot understand a radio as a bag of components, but must consider the arrangement of components into a circuit--two identical transistors may have very different meanings in different places in a circuit--and in the same way we cannot understand a biological system at the component level, but must find the biological equivalent of a circuit.

Cardelli suggests that the problem for biology is much worse. The behavior of a tamagotchi is not deducible solely from its hardware; we must consider the software it runs, and even the runtime behavior of the software (which may not be easily deducible from the code)--we must debug it. In the same way, a biological system of any complexity has both hardware and software components, and we must understand the software to understand the system.

It seems to me that there is a further obstacle to understanding biological systems. Radios and tamagotchi are human-designed objects; they must be simple enough for a human to understand. And it is natural for a human to keep a complex system understandable by factoring it into layered abstractions (as in a large software system). But (natural) biological systems are evolved--whatever works, fitness-wise, works. The complexity of such systems is not bounded by the need for human understanding, and there is no reason they should be formed of layered abstractions.  So we see phenomena like photosynthesis making use of quantum effects and evolved FPGAs relying on circuit noise.

On the other hand, it seems like there should be some fitness advantage to simplicity. A simpler organism breaks down less often, and a simpler biological program can be stored and transmitted more reliably. Maybe there is even an advantage to layered abstractions. Still, natural organisms seem to be hodgepodges, much more complex than even the cruftiest old codebase. Can a programmer fix a paramecium? I bet not.