Krischal's Digital Garden

2. How to apply Bayes Theorem to update belief

7 min read

By: Krischal Khanal

We've established the Bayes' Theorem as following formula that allowed inverting the conditional probabilities:

Now let's see how we could use the same theorem for the second, and arguably more profound application: updating belief.

Have you got the disease?

Suppose there's a rare disease. Only 1 in 1000 people have it. You suspect you may have it, and take a diagnostic test. The test comes back positive. How worried should you be?

If you thought it depends on the accuracy of the test, you are on the right path. The test's accuracy is as follows:

  • If you have the disease, the test catches 99% of the time (sensitivity, recall).
  • If you don't have the disease, the test incorrectly flags you 5% of the time (false positive rate).

Normally, if test is positive, people tend to think this is a worrying result. It has a high accuracy after all. This is known as falling for "Bayesian Trap". In reality, if the disease is so rare, most of the positive cases are false positives than the true ones.

Let's unravel how we could go about reasoning clearly ourselves.

The disease affects only 1 in 1000 people people. In absence of any clues, it's fair to assume I am equally susceptible to the disease.
Thus I choose to believe the chance of me having the disease is 1 in 1000.

Now that the test result is positive, it's either:
A: I do have the disease, and test result is positive. It's 99% chance inside 0.1% of having disease = .099%
B: I don't have the disease, but still the test result is positive. It's 5% chance inside 99.9% of me not having the disease = 4.995%

Conclusion 1: Since, I am still more likely to get a positive result in case B, it's probably the case that I do not have the disease.

Conclusion 2: However, the odds now have changed to A:(A+B) = 1.9% Thus, my suspicion of me having disease have risen to nearly 2%(which was only 0.1% originally). My suspicion rose from 1 in 1000 to 1 in 50.

Bayes Theorem allows us to do just that using a single formula.

Let's understand some terminologies first:

Priori and Posteriori

The hypothesis before seeing an evidence is a priori. Examples are:

  • I choose to believe the chance of me having the disease is 1 in 1000, just as the normal population. P(H) = 0.001 = 0.1%
  • I have a few other symptoms as well, thus I think it's 1 in 9 chance that I have this disease. P(H) = 0.111
  • I am male, thus chance of me being pregnant is zero. P(H) = 0

The your original belief (the priori) gets updated after you observed the evidence and becomes your new belief (a posteriori). Examples:

  • Given the positive test result, my suspicion has risen to 2%. P(H | E=positive) = 2%
  • Given the negative test result, I think my suspicion has come down. My P(H|E=negative) = 0.0013
  • Even when pregnancy test says positive, my chance of being pregnant is still zero. P(H|E) = 0

Actually applying the Formula

In the disease example:

  • : You have the disease (Hypothesis)
  • : You tested positive (Evidence)
  • (Prior)
  • (likelihood)
  • (Prior's complement)
  • (Complementary likelihood)

The marginal is the total probability of testing positive. This value depends on your priors and the likelihoods.

Here,

Now applying Bayes' Theorem

So, after taking the test, if it turns out positive, your suspicion raises to 2%. That forms your new belief, an updated hypothesis in the light of an evidence.

Bayesian reasoning process

"In general, 1 in 1000 people have this disease, so before any test, let's start with that as a starting belief.

Now, if I did have it, this test would be positive 99% of the time. And if I didn't, it would still be positive 5% of the time.

The disease is so rare that even a 5% false alarm rate across 999 healthy people produces far more false positives than the test produces true positives from the 1 sick person.

So yes, I tested positive but most people who test positive are healthy. I'm probably fine, but I am more suspicious that I may actually have it."

What data do we need for Bayesian reasoning?

One and only important information we need is the family of likelihoods.

  • Likelihood : How probable the evidence is, when the hypothesis is true. This is same as the True Positive Rate, Recall, or sensitivity.
  • Complementary likelihood : How probable the evidence is when the hypothesis is false. This is same as the False Positive Rate or Type-1 Error rate.

We also depend on the following assumption.

  • Prior : Although this is only your belief (an assumption or an educated guess), and not an empirical data, this also affects your Bayesian reasoning.

How prior affects the bayesian reasoning.


You can view this graph interactively

As you can see, this graph always start at (0,0) and ends at (1,1).

This means if you prior is 0 (complete denial), the Bayes' Theorem cannot update your belief. And so is the case if your prior is 1 (absolute certainty). This leads to a conclusion that:

You cannot update a certainty.

Even mathematically, if :

And if :

Do you feel sometimes, when trying to convince someone who is completely certain of something, no matter what evidence or fact you show up, you can never convince them otherwise?

Bayes' theorem provides some insight about why that's the case. Of course, real life situation is never that simple. But this provides strong mathematical reason to avoid holding the belief of certainty. Never blindly accept or completely reject the hypothesis. Always keep an open mind.

In statistics, this is known as the Cromwell's Rule. This rules warns against assigning the prior with the probability of 0 or 1.

In the next blog, we'll look into coding the Bayes' Theorem for updating belief.

Code blocks are licensed under the MIT License. All other content is licensed under CC-BY 4.0 Copyright © 2026 Krischal Khanal.

Graph View