Book Summary 1: The Theory That Would Not Die: How Bayes’ Rule Cracked the Enigma Code
One of my goals for 2018 is to summarise at least 12 books. In an attempt to keep myself accountable, I’ll be including the summarise here each month. I have joined the Farnam Street Reading Club which has also helped me to retain and learn more from each book. The first book is: The Theory That Would Not Die: How Bayes’ Rule Cracked the Enigma Code
Bayes’ rule states: by updating our initial belief about something with objective new information, we get a new and improved belief.
Bayes differs from modern scientific techniques which requires objectivity and precision. Bayes uses estimates and often relies on hunches and it states that we can learn even from missing and inadequate data, from approximations.
A battle resulted between the Bayesians and the FreqFrequentistse Frequentists believe that you could only calculate a probability after an event has been occured. For example, the probability that a coin will be heads is based on all the times that it has been heads before, 50%.
When an economist was preparing a research budget for the U.S. Air Force at RAND, a California think tank, he asked visiting statistician David Blackwell how to assess the probability that a major war would occur within five years. Blackwell, who had not yet become a Bayesian, answered, “Oh, that question just doesn’t make sense. Probability applies to a long sequence of repeatable events, and this is clearly a unique situation. The probability is either 0 or 1, but we won’t know for five years.” The economist nodded and said, “I was afraid you were going to say that. I have spoken to several other statisticians, and they have all told me the same thing.
In Contrast a Bayesian estimating how much ice cream someone would eat in the coming year, for example, would combine data about the individual’s recent ice cream consumption with other information, such as national dessert trends.
Bayes Theorem was used at Bletchley park to crack the Enigma code. Bletchley Park clerks catalogued by hand 17,000 ways “ein” could be encrypted, and a special machine was constructed to screen for them. It would let Turing guess a stretch of letters in an Enigma message, hedge his bets, measure his belief in their validity by using Bayesian methods to assess their probabilities, and add more clues as they arrived. It would identify the settings for 2 of Enigma’s 3 wheels and reduce the number of wheel settings to be tested on the bombes from 336 to as few as 18.
Another example where Bayes has been effectively used is in Politics and voting: Say you’re working at a county level with data coming in. Suppose you had no results from one county. A strict non Bayesian would say, ‘I can’t tell you anything there,’ but a slightly Bayesian person would say, ‘I don’t know what’s happening in county A, but county B is very similar and it’s showing a swing 5% toward Republicans.’ You might say that county A might be going the same way, but not give it great weight, because you do have to come up with a number. . . . take a group of counties that are similar, weight the data you get in these counties, give zero weight to non-data counties, and upgrade, update it all the time.’” A decision with inadequate information, whatever knowledge existed was better than nothing
In 2006 Netflix launched a search for the best recommender system to improve its algorithm. The most important lesson learned from the Netflix competition originated as a Bayesian idea: sharing. Bayesian Model Averaging studies show that when two models that are not highly correlated are combined in a smart way, the combination often does better than either individual model.” The contest publicized Bayes’ reputation as a fertile approach to learning far beyond mere Bayesian technology.
According to Bayes, the brain stores a wide range of possibilities but assigns them high and low probabilities. Color vision is already known to operate this way. We think we perceive red, but we actually see an entire spectrum of colors, assign the highest probability to red, and keep in mind the outside possibilities that the color could be pink or purple. Bayesian thinking is basic to everything a human does, from speaking to acting. The biological brain has evolved to minimize the world’s uncertainties by thinking in a Bayesian way. In short, growing evidence suggests that we have Bayesian brains.
Leave a Comment
Subscribe to newsletter?