The Power of Probability

Intuitively, we know what probability is: the chance or likelihood of an event happening. Nothing bar death (not even taxes) is certain and probability is therefore critical for analyzing life and business decisions correctly.

Yet while most people understand what probability is, very few in my experience can actually work with them. Examples include the BBC reporting that a study found that vitamin E increased mortality by 14 percent or Professor Sir Roy Meadow, who testified that the chance of Sally Clark’s two children both suffering sudden infant death syndrome was 1 in 73 million. (Email me if you don’t know why these are ridiculous.)

Probability is one of the most important mathematical tools you can understand. While it can be hard to grasp I force it into almost every course I teach. Consider this problem. You take a HIV test and the result comes back positive. What does this mean?

Everybody who has ever watched a medical drama is aware of false positives. So do you need more information?

The most common type of HIV antibody tests is the ELISA which has a sensitivity of 99.8 per cent. Of 1000 people with HIV, 998 will test positive. So there’s a 0.2 per cent false negative rate. These tests also have 98.5 per cent specificity. Of 1000 people without HIV, 985 will test negative. There’s a 1.5 per cent false positive rate. This is a very accurate test.

So given a single positive test result, what’s the probability of having HIV? 99 per cent? No, it depends upon the prevalence of HIV in the population i.e. your prior probability of having HIV before you had the test. For people without any risk factors (e.g. intravenous drug user) the prevalence of HIV in the UK is 1 in 10,000 or 0.01 per cent.

Out of 10,000 people with no risk factors, we expect one to have HIV and to almost certainly test positive. However, of the 9,999 people who don’t have HIV almost 150 of them will falsely test positive. That means of the folk who test positive, 1 in 151 of them (0.7 %) will actually have HIV.

Obviously, you would take another test and update the prior probability using the results of the first HIV test.

The formula we just used for calculating this result is called Bayes’ Theorem and was first derived by the Rev Thomas Bayes to prove the existence of God. Today it has many applications including detecting whether an email is spam, the location of missing nuclear weapons (lost under sea) and helped the Bletchley Park codebreakers to crack Germany’s ciphers during World War II.

Bayes’ theorem is important because it tells us how to update our beliefs about a statement when new information comes to light. As John Maynard Keynes said: “When the facts change, I change my opinion. What do you do, sir?”

Please leave a comment - we all like them