Probability Theory

Large N & Leeuwenhoek (70700173)

Yue Hu

Overview

Probability as a Concept
Probabilities as Estimand
Probabilities as Statistics

1 Probability: Concept

1.1 Determinism vs. Probabilism

1.2 Perspectives of Probability

Frequentist (“objective”)
- “Coin has two sides, balance” ⇒ Pr(head) = 0.5.
Relative frequency
- You are observing a random sample.
- \[\lim_{n\to \infty}\frac{f(x)}{n}\]
Evolutionary (“subjective”): Bayesian, based on beliefs (prior), game theory

2 Probability: Estimand

2.1 How to Know the Probability in Statistics

Sample Space: Space of all possible outcomes

∑ Pr = 1
All of the probabilities constitute the probability distribution
For all event in the space, \(P(E) + P(\tilde E) = 1\)

Methods to “discover” the probability in a sample space

Sample Point Method
Event Composition Method
Bayesian Method

2.2 Sample Point Method

“Count out”

Define experiment event;
Define sample space;
Assign probabilities, P(E_i) ≥ 0, ∑ P(E_i) = 1;
Define event of interest, A;
Find Pr(A) by summing probability of A.
- With or without replacement?

2.3 Replacement

W/o replacement means once being picked out, never put it back.

Five soccer balls and five basketballs (in the same size and surface).

What’s the chance (a.k.a., probability) to get 5 soccer balls in five spilling out?

W. replacement: \(0.5^5\);
W/o replacement: \(\frac{5}{10}\times\frac{4}{9}\times\frac{3}{8}\times\frac{2}{7}\times\frac{1}{6} = {10 \choose 5}^{-1} = \frac{5!(10 - 5)!}{10!}.\)

2.4 Think deeper

Is the cross-sectional survey sampling a w. or w.o. replacement process?

2.5 Event Composition Method (ECM)

“Calculate”

Applying multiplicative and additive rules.

E₁: m outcomes
E₂: n outcomes

⇒ \(m\times n\) possible outcomes for E₁E₂.

Six heads, 15 bodys, 25 colors, how many distinctive types of toys?

\[6 \times 15 \times 25 = 2,250.\]

2.6 Rules of ECM

Permutation: Ordered

\[P(n, r) = \frac{n!}{(n - r)!}.\]

Combination: ~~Order~~

\[{n \choose r} = \frac{n!}{r!(n - r)!}.\]

Intersection: Event G and H, G∩H = \(\{S_i: S_i\in G\ and\ S_i\in H\}\)
Union: G∪H = \(\{S_i: S_i\in G\ or\ S_i\in H\}\)
- P(G∪H) = P(G) + P(H) - P(G)∩P(H).
Exclusive: G∩H = ∅
Complement: J ⊂ -G ∩ H
Conditional: \(P(H|G) = \frac{P(G\cap H)}{P(G)}\)
- Independent: P(H|G) = P(H); P(G∩H) = P(G)P(H)

2.7 Baysian Method

“Figure out”

Bayes’ Theorem

\[ \begin{aligned} P(A|B)P(B) =& P(B|A)P(A) = P(A\cap B);\\ P(A|B)=& \frac{P(B|A)P(A)}{P(B)},\\ =& \frac{P(B|A)P(A)}{P(B|A)P(A) + P(B|\tilde A)P(\tilde A)} \end{aligned} \]

Thomas Bayes (1702–1761)

2.8 Bayesian probability of your roommate is a pervert

2.9 Diss

Frequentist 🔫 Bayesian

Repeated sample: When the population data are collected, the repeat-sample makes no sense.
Future event: It is impossible to have repeat samples for future.

Bayesian 🔫 Frequentist

3 Probability: Statistics

3.1 Four Common Ways to Presenting Probability

Percentage ✓
Relative risk
Odds Ratio
Bayesian method

3.2 Relative Risk

\(\hat{\pi}_{1|1}/\hat{\pi}_{1|2} = \frac{n_{11}/n_{\bullet 1}}{n_{12}/n_{\bullet 2}}\)

E.g., A hospital investigated whether alcoholism relates to fatal illness.

Alcoholism	Yes	No
Fatal	1,601	510
Nonfatal	162,527	412,368

\[Risk_{fatal} = \frac{1601/(1601 + 162527)}{510/(510 + 412368)} = 7.897\]

When people drink, they are 7.9 more times to die in a fatal accident than those w/o.

The above line distinguishes a scholar from a student.

3.3 Odds Ratio

\[\begin{align} O_{12} =& \frac{\pi_{1|1}/\pi_{1|2}}{\pi_{2|1}/\pi_{2|2}}, \\ =& \frac{n_{11}/n_{12}}{n_{21}/n_{22}}, \\ =& n_{11}n_{22}/n_{21}n_{12}. \end{align}\]

The previous e.g.,

Alcoholism	Yes	No
Fatal	1,601	510
Nonfatal	162,527	412,368

\(O_{fatal} = \frac{1601/510}{162527/412368}\) = 7.965

The fatal risk of alcoholism is 8 times larger than not.

Log odds: mapping the range (0,1) to (-∞, +∞) using log of odds,

A.k.a., “logistic unit” (logit)
If independent, then log odds = 0.

3.4 RR vs. OR

For common cases, RR is better.
For rare cases, both ok.

3.5 Puting Probability Together

(Probability) distribution: The mathematical function that gives the probabilities of occurrence of different possible outcomes for an experiment.

Marginal and joint totals

\[\begin{align} n_{i\bullet} =& \sum_j n_{ij}; n_{\bullet j} = \sum_i n_{ij}; \\ n_{\bullet\bullet} =& \sum\sum n_{ij}. \end{align}\]

Conditional distribution

\[\begin{align} \pi_{i|j} = \pi_{ij} / \pi_{\bullet j}; \\ \hat{\pi_{i|j}} = n_{ij}/n_{\bullet j}. \end{align}\]

Marginal and joint distributions

Marginal \[\begin{align} \pi_{i\bullet} =& \sum_j \pi_{ij}, \hat{\pi_{i\bullet}} = n_{i\bullet}/n_{\bullet\bullet}; \\ \pi_{\bullet j} =& \sum_i \pi_{ij}, \hat{\pi_{\bullet j}} = n_{\bullet j}/n_{\bullet\bullet}. \end{align}\]
Joint: \(\pi_{\bullet\bullet}\) = ∑∑ π_ij = 1.

3.6 Calculate the Distributions

E.g., Into Young Woman and the Sea?

Gender	Yes	No	Total
Male	5	12	17
Female	6	5	11
Total	11	17	28

What’s the probability for a female that don’t like?
- \(Pr(FN) = 5/28.\) (event probability)
What’s the probability for finding a disliker?
- \(Pr(N) = 17/28.\) (marginal distribution)
What’s the probability of disliking given the respondent is a female?
- \(P(N|F) = 5/11.\) (conditional distribution)

3.7 Independence

\[P(X\cap Y) = P(X)P(Y); P(Y|X) = P(Y).\]

Such event’s probability is:

Population: \(\pi_i = n_i/N\);
Sample: \(\hat{\pi_i} = n_i/n_{\bullet\bullet}.\)

E.g., Young Woman and the Sea?

Gender	Yes	No	Total
Male	5	12	17
Female	6	5	11
Total	11	17	28

Are gender and approval independent?
- Pr(MY) = 11/28 × 17/28 ≠ 5/28