By: Krischal Khanal
Textbook definition
The following is the mathematical definition for the Bayes' Theorem in standard textbook:
Bayes' theorem
Sometimes it's also written as:
Where the meaning of the symbols are as follows:
: A symbol for a hypothesis : A symbol for an evidence/event : Negation symbol ( means the negation of Hypothesis ) : Probability of H conditioned on E And the standard terminologies:
: Posterior Probability : Likelihood : Prior Probability : Marginal Probability
While trivial to some, there might still be something new in this blog. It might be worth sticking around.
There are two main application of the Bayes' Theorem:
- Inverting the conditional probability
- Updating the belief.
In this blog, we are going to deal with the first. We will deal with the second one in the upcoming blog series.
Simple proof
First, let's greatly simplify Bayes' Theorem:
Its just same as the following:
Isn't this the same formula for calculating conditional probability from Marginal and joint probabilities?
Yes. Indeed it is.
and it's also the case that,
Using (1) and (2)
With a simple rearrangement:
Which is the same as the Bayes' Theorem.
Marginal and joint probabilities
If you already know this part you can skip this.
Example
You have a deck of cards. Your interested property of the card are (i) if the card is red and (ii) if the card's rank is 4.
| A randomly chosen card | Color (Joint probability) | Rank (Marginal probability) | ||
|---|---|---|---|---|
| Red | Not Red | |||
| Rank (Joint Probability) | 4 | P(4 and Red) | P(4 but not Red) | P(4) |
| Not 4 | P(Red but not 4) | P(Neither Red nor 4) | P(Not 4) | |
| Color (Marginal probability) | P(Red) | P(Not Red) | 100% | |
The joint probabilities consider all the cases, and the marginal probabilities are calculated by summing the joint probabilities of all such cases.
Thus, P(card is 4) = P(card is 4, card is red) + P(card is 4, card is not red)
Mathematically,
Conditional probability
Continuing on the above example, the conditional probability is the probability when you already know some of the case are true. For example,
P(card is 4 | card is red) = if you already know the card is red, what is the probability that the card is 4?
You can use conditional probability and marginal probability to calculate joint probability.
What is the probability that the card is red and card is 4? It has to be red first, then it has to be also 4 when it is red.
P(card is red, card is 4) = P(card is red )
Mathematically,
Rearranging this, we get the following formula
Which is the simplified version of Bayes' Theorem in the Simple proof
There is also missing half of the Bayes' Theorem, not normally covered in the textbook and most resources online. For that, stay tuned in for the upcoming blog in the series.
Code blocks are licensed under the MIT License. All other content is licensed under CC-BY 4.0 Copyright © 2026 Krischal Khanal.