Probability: complete lesson (sample space, events, conditional, independence)

Basic vocabulary

• Random experiment: an experiment whose outcome cannot be predicted.
• Sample space $\Omega$: set of all possible outcomes.
• Event: subset of $\Omega$.
• Elementary event: $\{ \omega \}$ (a single outcome).

Probability: 3 axioms

A probability $P$ satisfies:

1. $0 \leq P(A) \leq 1$ for any event $A$.
2. $P(\Omega) = 1$.
3. If $A$ and $B$ are incompatible ($A \cap B = \emptyset$): $P(A \cup B) = P(A) + P(B)$.

Equally likely outcomes (uniform)

If all outcomes are equally likely:

$P(A) = \dfrac{\text{number of favorable outcomes}}{\text{number of possible outcomes}} = \dfrac{|A|}{|\Omega|}$

Key formulas

• Complement: $P(\overline{A}) = 1 - P(A)$.
• Union: $P(A \cup B) = P(A) + P(B) - P(A \cap B)$.

Conditional probability

$P(A | B) = \dfrac{P(A \cap B)}{P(B)} \quad (\text{if } P(B) \neq 0)$

“Probability of $A$ given $B$”. Careful: $P(A | B) \neq P(B | A)$ in general.

Independent events

$A$ and $B$ are independent iff:

$P(A \cap B) = P(A) \times P(B)$

Equivalently: $P(A | B) = P(A)$ (if $P(B) \neq 0$).

Tree diagram

On a weighted tree:

• The sum of probabilities leaving a node equals 1.
• The probability of a path = product of branch probabilities.
• The probability of an event = sum of path probabilities leading to it.

Law of total probability

If $(B_1, B_2, \ldots, B_n)$ form a partition of $\Omega$ (all incompatible, union = $\Omega$):

$P(A) = \sum_{i=1}^{n} P(A \cap B_i) = \sum_{i=1}^{n} P(A | B_i) \cdot P(B_i)$

Bayes’ formula

$P(B | A) = \dfrac{P(A | B) \cdot P(B)}{P(A)}$

Lets you reverse the conditional. Often used with the law of total probability.

Random variable

A random variable $X$ maps each outcome in $\Omega$ to a number.

• Distribution of $X$: list of pairs $(x_i, P(X = x_i))$.
• Expectation: $E(X) = \sum x_i \cdot P(X = x_i)$.
• Variance: $V(X) = E(X^2) - E(X)^2$.
• Standard deviation: $\sigma(X) = \sqrt{V(X)}$.

Common mistakes

1. Confusing $P(A | B)$ and $P(B | A)$.
2. Summing $P(A) + P(B)$ for union without subtracting $P(A \cap B)$.
3. Confusing independent with incompatible (different notions).
4. Forgetting branch sums at a tree node equal 1.