[Algorithms] The Bayes Rule

Prior odd:

The idea is to take the odds for something happening (against it not happening), which we´ll write as prior odds.

For example:

The chances of rain are 206 in 365. Then the Prior odd = 206:159

Likelihood ratio:

However, after opening your eyes and taking a look outside, you notice it’s cloudy. Suppose the chances of having a cloudy morning on a rainy day are 9 out of 10 — that means that only one out of 10 rainy days start out with blue skies. But sometimes there are also clouds without rain: the chances of having clouds on a rainless day are 1 in 10. Now how much higher are the chances of clouds on a rainy day compared to a rainless day?

The answer is that the chances of clouds are nine times higher on a rainy day than on a rainless day: on a rainy day the chances are 9 out of 10, whereas on a rainless day the chances of clouds are 1 out of 10, and that makes nine times higher.

So we concluded that on a cloudy morning, we have: likelihood ratio = (9/10) / (1/10) = 9

Posterior odds:

posterior odds = likelihood ratio × prior odds

For example:

Apply the Bayes rule to calculate the posterior odds for rain having observed clouds in the morning in Helsinki.

As we calculated above, the prior odds for rain is 206:159 and the likelihood ratio for observing clouds is 9

posterior odds = 9 * 206 / 159

---

Consider mammographic screening for breast cancer. Using made up percentages for the sake of simplifying the numbers, let’s assume that five in 100 women have breast cancer. Suppose that if a person has breast cancer, then the mammograph test will find it 80 times out of 100. When the test comes out suggesting that breast cancer is present, we say that the result is positive, although of course there is nothing positive about this for the person being tested. (A technical way of saying this is that the sensitivity of the test is 80%.)

The test may also fail in the other direction, namely to indicate breast cancer when none exists. This is called a false positive finding. Suppose that if the person being tested actually doesn’t have breast cancer, the chances that the test nevertheless comes out positive are 10 in 100.

Based on the above probabilities, you are be able to calculate the likelihood ratio. You'll find use for it in the next exercise. If you forgot how the likelihood ratio is calculated, you may wish to check the terminology box earlier in this section and revisit the rain example.

Consider the above breast cancer scenario. An average woman takes the mammograph test and gets a positive test result suggesting breast cancer. What do you think are the odds that she has breast cancer given the observation that the test is positive?

Hints:

1. Start by calculating the prior odds.
2. Determine the probability of the observation in case of the event (cancer).
3. Determine the probability of the observation in case of no event (no cancer).
4. Obtain the likelihood ratio as the ratio of the above two probabilities.
5. Finally, multiply the prior odds by the likelihood ratio.

Enter the posterior odds as your solution below. Give the answer in the form xx:yy where xx and yy are numbers, without simplifying the expression even if both sides have a common factor.

---

40:95

Prior: 5:95

Likelihood Ratio: (0.05 * 0.8) / (0.05 * 0.1) = 8

Posterior = 8 * 5 : 95 = 40: 95

First, let's express the probabilities in terms of odds. The prior odds describe the situation before getting the test result. Since five out of 100 women have breast cancer, there is on the average five women with breast cancer for every 95 women without breast cancer, and therefore, the prior odds are 5:95. The likelihood ratio is the probability of a positive result in case of cancer divided by the probability of a positive result in case of no cancer. With the above numbers, this is given by 80/100 divided by 10/100, which is 8. The Bayes rule now gives the posterior odds of breast cancer given the positive test result: posterior odds = 8 × 5:95 = 40:95, which is the correct answer. So despite the positive test result, the odds are actually against the person having breast cancer: among the women who are tested positive, there are on the average 40 women with breast cancer for every 95 women without breast cancer. Note: If we would like to express the chances of breast cancer given the positive test result as a probability (even though this is not what the exercise asked for), we would consider the 40 cases with cancer and the 95 cases without cancer together, and calculate what portion of the total 40 + 95 = 135 individuals have cancer. This gives the result 40 out of 135, or about 30%. This is much higher than the prevalence of breast cancer, 5 in 100, or 5%, but still the chances are that the person has no cancer. If you compare the solution to your intuitive answer, they tend to be quite different for most people. This demonstrates how poorly suited out intuition is for handling uncertain and conflicting information.

Resource:

https://www.youtube.com/watch?v=tRE6mKAIkno

https://course.elementsofai.com/3/2