Basic probability, conditional probability, and Bayes' theorem
This tutorial sheet builds your foundation in probability theory. Don't rush through—each question teaches a distinct concept. Use the hints before jumping to solutions. Remember: in exams, the approach matters as much as the answer.
A dart hits a target with probability 1/4. Three darts are thrown. Find the probability of at least one hit. Solve in two ways. Are all outcomes equally likely?
You're throwing 3 darts at a target. Each dart has a 1/4 chance of hitting. The question wants you to:
In simple terms: What are your chances of not completely failing (i.e., hitting at least once)?
The probability of event A happening = 1 - P(A not happening)
Each dart throw is independent. Probability of multiple independent events occurring together:
For exactly k successes in n trials:
P(at least one hit) = 1 - P(no hits)
P(no hits) means all three darts miss the target:
P(all miss) = P(miss) × P(miss) × P(miss)
Therefore:
P(at least one hit) = P(exactly 1 hit) + P(exactly 2 hits) + P(exactly 3 hits)
Exactly 1 hit:
Exactly 2 hits:
Exactly 3 hits:
Total:
No. Since P(hit) = 1/4 and P(miss) = 3/4, they are not equal. Missing is 3 times more likely than hitting.
The outcomes HIT and MISS are not equally likely for each individual throw.
Two balls are drawn sequentially (without replacement) from an urn with 3 red, 4 white, 5 blue balls. Find: (a) P(first red and second blue), (b) P(second white if first replaced), (c) P(second white if first not replaced).
You have an urn with 12 balls total: 3 red (R), 4 white (W), 5 blue (B).
You're drawing two balls one after another. The question has three parts:
For events happening in sequence:
where P(B|A) means "probability of B given that A happened"
After drawing first ball, total decreases:
• Initial: 12 balls total
• After 1st draw: 11 balls remain
First ball is returned before second draw:
• Total remains 12 for both draws
• Draws become independent
For part (c), we partition by first ball color:
P(1st red) = 3/12 = 1/4
After drawing 1 red ball:
P(2nd blue | 1st red) = 5/11
Since the first ball is replaced, the composition remains the same for the second draw.
The second draw is independent of the first.
Note: It doesn't matter what happened in the first draw when we replace the ball.
We must consider all possible outcomes for the first draw using the law of total probability:
Case 1: First ball is Red
P(1st Red) = 3/12
P(2nd White | 1st Red) = 4/11 (white balls unchanged, 11 total remain)
Contribution: (3/12) × (4/11) = 12/132
Case 2: First ball is White
P(1st White) = 4/12
P(2nd White | 1st White) = 3/11 (one white removed, 11 total remain)
Contribution: (4/12) × (3/11) = 12/132
Case 3: First ball is Blue
P(1st Blue) = 5/12
P(2nd White | 1st Blue) = 4/11 (white balls unchanged, 11 total remain)
Contribution: (5/12) × (4/11) = 20/132
Total:
Interesting observation: Same as part (b)! This is because of symmetry—every ball has equal chance of being in any position.
PCs from plant A: 15% defective, production = 1,000,000/year. PCs from plant B: 5% defective, production = 150,000/year. Find probability of buying a defective PC.
Two plants manufacture PCs:
You randomly buy a PC from the market (which pools both plants' output). What's the probability your PC is defective?
In simple terms: Both good and bad factories contribute to the market. What's your overall risk?
When an event can happen through multiple pathways:
where events A and B partition the sample space
The defect rate is weighted by production volume, not a simple average.
Total production = 1,000,000 + 150,000 = 1,150,000
P(from Plant A) = 1,000,000/1,150,000 = 100/115 = 20/23
P(from Plant B) = 150,000/1,150,000 = 15/115 = 3/23
P(Defective | Plant A) = 0.15 = 15/100
P(Defective | Plant B) = 0.05 = 5/100
P(Defective) = P(D|A)×P(A) + P(D|B)×P(B)
Alternative calculation:
If A and B are mutually exclusive, can they be independent? Explain.
This is a conceptual question testing whether you understand the difference between two important probability concepts:
Can the same two events be both mutually exclusive AND independent at the same time?
Events A and B are mutually exclusive if:
They cannot occur together. If A happens, B cannot happen.
Events A and B are independent if:
Occurrence of A doesn't change probability of B.
Assume A and B are mutually exclusive with non-zero probabilities.
This means: P(A) > 0, P(B) > 0, and P(A∩B) = 0
For independence, we need:
But we know P(A∩B) = 0 (mutually exclusive)
So we need: 0 = P(A) × P(B)
This is only possible if P(A) = 0 OR P(B) = 0
NO, mutually exclusive events (with non-zero probabilities) CANNOT be independent.
Why not? If A and B are mutually exclusive:
Intuitively: Mutually exclusive events are maximally dependent—one completely determines the other cannot happen.
How many equations are needed to establish independence of n events?
When we have n events (A₁, A₂, ..., Aₙ), how many independence conditions must we check to confirm they are all mutually independent?
In simple terms: Independence isn't just about pairs—we need to check all possible combinations. How many checks is that?
Events A₁, A₂, ..., Aₙ are mutually independent if:
for ALL possible subsets of size k ≥ 2
Number of ways to choose k items from n:
For complete independence, we need to verify independence for:
Total equations = C(n,2) + C(n,3) + C(n,4) + ... + C(n,n)
We know from binomial theorem:
Subtracting C(n,0) and C(n,1):
n = 2: 2² - 1 - 2 = 1 equation ✓
(Just check P(A∩B) = P(A)P(B))
n = 3: 2³ - 1 - 3 = 4 equations ✓
(Check A∩B, A∩C, B∩C, and A∩B∩C)
n = 4: 2⁴ - 1 - 4 = 11 equations
If (A ∩ B = φ), find relation between probabilities of A and (Bᶜ).
You are given two events A and B that are mutually exclusive (their intersection is empty, φ).
You need to find the mathematical relationship (inequality or equality) between the probability of A and the probability of "not B" (Bᶜ).
In simple terms: If A and B can't happen together, does simple logic tell us something about how likely A is compared to "not B"?
1. Mutually Exclusive: A ∩ B = φ implies A and B have no common elements.
2. Complement: Bᶜ is the set of all outcomes NOT in B.
3. Subset Property: If A ⊆ C, then P(A) ≤ P(C).
Since A ∩ B = φ, the sets A and B are disjoint. They do not overlap.
The set Bᶜ consists of everything outside B.
Since A has no overlap with B, all elements of A must lie outside B.
Therefore, A is a subset of Bᶜ.
For any two events X and Y, if X ⊆ Y, then P(X) ≤ P(Y).
Substituting X = A and Y = Bᶜ:
P(A) = P(A ∩ Bᶜ) + P(A ∩ B)
Since A ∩ B = φ, P(A ∩ B) = 0.
So, P(A) = P(A ∩ Bᶜ).
We know P(Bᶜ) = P(A ∩ Bᶜ) + P(Aᶜ ∩ Bᶜ).
Since probabilities are non-negative, P(Aᶜ ∩ Bᶜ) ≥ 0.
Therefore, P(Bᶜ) ≥ P(A ∩ Bᶜ) = P(A).
Urn has b black, r red balls. One ball drawn, replaced with c additional balls of same colour. Another ball drawn. Find P(first was black | second is red).
We are drawing balls in two stages.
Stage 1: Draw one ball. Note color. Put it back PLUS 'c' more of the SAME color.
Stage 2: Draw second ball.
Question: We know the second ball is RED. Given this information, what is the probability that the FIRST ball was BLACK?
To find the denominator P(B):
Total balls initially n = b + r.
B1 = First ball Black. P(B1) = b / (b + r)
R1 = First ball Red. P(R1) = r / (b + r)
Case 1: First was Black (B1)
We added c black balls. New composition: (b + c) Black, r Red. Total = b + r + c.
P(R2 | B1) = r / (b + r + c)
Case 2: First was Red (R1)
We added c red balls. New composition: b Black, (r + c) Red. Total = b + r + c.
P(R2 | R1) = (r + c) / (b + r + c)
P(R2) = P(R2 | B1)P(B1) + P(R2 | R1)P(R1)
Surprise! P(R2) = P(R1). The unconditional probability doesn't change!
P(B1 | R2) = [P(R2 | B1) × P(B1)] / P(R2)
Three prisoners: one to be executed, two freed. Prisoner A asks jailer to name one freed prisoner. Jailer refuses, saying it increases A's chance of execution. Evaluate jailer’s reasoning.
Prisoner A thinks: "Currently my chance of dying is 1/3. If the jailer names B as safe, then it's just between me and C, so my chance becomes 1/2."
The Jailer agrees with this logic and refuses to speak to "avoid hurting A".
Task: Is this reasoning correct? Does knowing "B is safe" actually change A's probability of execution?
Information only changes probability if the information itself was essentially uncertain or provided a distinction.
Here, the jailer MUST name someone. Does naming 'B' strictly rule out 'A'?
P(A dies) = P(B dies) = P(C dies) = 1/3.
Jailer logic constraints:
Let J_B be the event "Jailer names B".
We want P(A dies | J_B).
Numerator: P(J_B | A dies) × P(A dies)
= (1/2) × (1/3) = 1/6
Denominator: P(J_B)
P(J_B) = P(J_B | A dies)P(A) + P(J_B | B dies)P(B) + P(J_B | C dies)P(C)
= (1/2)(1/3) + (0)(1/3) + (1)(1/3)
= 1/6 + 0 + 1/3 = 1/2
The probability remains 1/3. The jailer's reasoning (and A's fear) is incorrect.
Intuition: The Jailer *had* to give a name. Hearing "B" tells A nothing new about himself, because he already knew at least one of B or C was safe. The risk "shifts" entirely to C (whose prob becomes 2/3).
Gambler has a fair coin and a two-headed coin. (a) One flip → heads. P(coin is fair)? (b) Second flip → heads again. P(fair coin)? (c) Third flip → tails. P(fair coin)?
We choose one coin randomly (50-50 chance). One is Fair (H, T), one is Fake (H, H).
We flip it and observe results. After each result, we update our belief (probability) that we are holding the Fair coin.
Posterior ∝ Likelihood × Prior
P(F) = 0.5, P(T) = 0.5
P(H|F) = 1/2, P(H|T) = 1
P(H) = (1/2)(1/2) + (1)(1/2) = 1/4 + 1/2 = 3/4
We can restart or use posterior from (a). Let's restart (Evidence = HH).
P(HH|F) = 1/4, P(HH|T) = 1
P(HH) = (1/4)(1/2) + (1)(1/2) = 1/8 + 4/8 = 5/8
(Probability of fair coin creates decreases as we see more heads)
Two-headed coin CANNOT produce tails. P(Tails | T) = 0.
Therefore, if we see a tail, it is IMPOSSIBLE for it to be the two-headed coin.
Roulette strategy: bet on red only after 10 consecutive blacks. Evaluate reasoning.
A gambler believes due to "law of averages", Red is MORE likely to appear after a long streak of Blacks to "balance things out". Is this mathematically true?
The belief that independent events are self-correcting in the short run.
Independence: P(A | B) = P(A).
Let R₁₁ be result of 11th spin.
Spinning a roulette wheel is an independent process.
The probability of Red is fixed (e.g., 18/38 or 18/37 depending on wheel).
The reasoning is FLAWED.
The probability of winning on the 11th spin is exactly the same as winning on the 1st spin. The past history of 10 blacks has zero influence on the next spin.
Class: 4 freshman boys, 6 freshman girls, 6 sophomore boys. How many sophomore girls for independence between sex and class?
We are building a class with:
Find 'x' such that a randomly chosen student's sex is statistically independent of their class year.
In simple terms: The proportion of boys should be the same in both the freshman group and the sophomore group.
Variables A and B are independent if:
Here: P(Boy | Freshman) = P(Boy | Sophomore)
| Freshman | Sophomore | Total | |
|---|---|---|---|
| Boys | 4 | 6 | 10 |
| Girls | 6 | x | 6+x |
| Total | 10 | 6+x | 16+x |
P(Boy | Freshman) = P(Boy | Sophomore)
Or equivalently: Ratio of Boys/Girls in Freshman = Ratio of Boys/Girls in Sophomore
Freshman ratio (Boys/Girls) = 4/6 = 2/3.
Sophomore ratio (Boys/Girls) = 6/x.
If x=9, total students = 16+9 = 25.
P(Boy) = 10/25 = 0.4.
P(Sophomore) = 15/25 = 0.6.
P(Boy ∩ Sophomore) = 6/25 = 0.24.
Check: P(Boy) × P(Soph) = 0.4 × 0.6 = 0.24. Matches! ✓
Two boxes: Box 1: 1 black, 1 white; Box 2: 2 black, 1 white. Random box and marble selected. (a) P(marble is black) (b) P(first box selected | marble is white).
Box 1: [B, W] (50% Black)
Box 2: [B, B, W] (67% Black)
1. Pick box randomly (50-50), pick marble. Prob it's Black?
2. If you see White, did it come from Box 1?
P(Black) = P(B|Box1)P(Box1) + P(B|Box2)P(Box2)
P(B|Box1) = 1/2
P(B|Box2) = 2/3
First, find P(White).
P(White) = 1 - P(Black) = 1 - 7/12 = 5/12.
Numerator: P(W|Box1)P(Box1) = (1/2)(1/2) = 1/4 = 3/12.
Genetics: black dominates brown. (a) P(black rat is pure black given brown sibling) (b) If mated with brown rat gives 5 black offspring → P(pure black).
Genetics 101: Black (B) is dominant. Brown (b) is recessive.
Part (a): We have a Black rat. We know it has a Brown sibling (bb). This implies parents were both hybrids (Bb). Given our rat is Black (so not bb), what is probability it is Pure (BB)?
Part (b): We mate this rat with a brown one (bb). We get 5 babies, ALL BLACK. Update the probability that our rat is Pure (BB).
Bb × Bb offspring probabilities: 1/4 BB, 1/2 Bb, 1/4 bb.
Update prior belief with observational evidence (offspring colors).
Parents must be Bb × Bb (to produce a bb sibling).
Offspring space for OUR rat: {BB, Bb, bB, bb}.
But we know our rat is BLACK, so it is NOT bb. Space reduces to {BB, Bb, bB}.
Favorable for Pure (BB): 1 case.
So Prior P(BB) = 1/3, P(Bb) = 2/3.
Evidence E: 5 black offspring when mated with bb.
Case 1: Rat is Pure (BB)
BB × bb → All offspring are Bb (Black). P(Black) = 1.
P(E | BB) = 1⁵ = 1
Case 2: Rat is Hybrid (Bb)
Bb × bb → Offspring 50% Black (Bb), 50% Brown (bb).
P(E | Bb) = (1/2)⁵ = 1/32
Numerator: P(E|BB)P(BB) = 1 × (1/3) = 1/3
Denominator P(E):
= P(E|BB)P(BB) + P(E|Bb)P(Bb)
= 1(1/3) + (1/32)(2/3)
= 1/3 + 2/96 = 1/3 + 1/48 = 16/48 + 1/48 = 17/48
Ten coins: ith coin gives heads with probability i/10. One coin randomly chosen and gives heads. Find P(it was 5th coin).
We pick one coin from 10 distinct coins. Coin #1 has 10% bias, Coin #2 has 20%... Coin #10 has 100% chance of Heads.
We flip the chosen coin and get Heads. How likely is it that we picked exactly Coin #5?
P(C_i) = 1/10 (constant for all i)
P(H|C_i) = i/10
P(C_5 | H) = [P(H|C_5)P(C_5)] / Σ [P(H|C_j)P(C_j)]
Since P(C_j) = 1/10 is constant, it cancels out from numerator and denominator!
Sum 1+2+...+10 = 10(11)/2 = 55.
P(C_5 | H) = 5 / 55 = 1 / 11.
Give an example of three events A, B, C that are pairwise independent but not mutually independent.
Construct a counter-example to show that:
P(A∩B) = P(A)P(B) (and similarly for B,C and A,C)
DOES NOT IMPLY
P(A∩B∩C) = P(A)P(B)P(C).
Pairwise: Every pair is independent.
Mutual: Every pair AND the triple (and all subsets) are independent.
Consider a fair tetrahedron (4-sided die) with faces colored:
Toss it. Let R = Red face down, B = Blue down, G = Green down.
Total outcomes n=4 (equally likely).
Red appears on Face 1 and Face 4. So P(R) = 2/4 = 1/2.
Similarly, P(B) = 1/2, P(G) = 1/2.
R ∩ B means "face has both Red and Blue". Only Face 4 has both.
P(R ∩ B) = 1/4.
Check: P(R)P(B) = (1/2)(1/2) = 1/4.
By symmetry, B & G and R & G are also independent.
R ∩ B ∩ G means "face has all three". Only Face 4.
P(R ∩ B ∩ G) = 1/4.
But P(R)P(B)P(G) = (1/2)(1/2)(1/2) = 1/8.
Alice and Bob alternate tossing a fair coin. Alice goes first. The first to throw a Head wins. Find probability that Alice wins.
Game of "Sudden Death" with a coin.
Turn 1: Alice. If H, A wins. If T, give to Bob.
Turn 2: Bob. If H, B wins. If T, give to Alice.
Turn 3: Alice...
Find P(Alice Wins).
where |r| < 1.
1st way: H (Alice wins immediately). Prob = 1/2.
2nd way: T(Alice) -> T(Bob) -> H(Alice). Prob = (1/2)(1/2)(1/2) = 1/8.
3rd way: T T T T H. Prob = (1/2)⁵ = 1/32.
S = 1/2 + 1/8 + 1/32 + ...
First term a = 1/2.
Common ratio r = 1/4 (since 1/8 divided by 1/2 is 1/4).
System has 3 components. A and B in parallel, and this unit is in series with C. Probabilities of working: P(A)=0.9, P(B)=0.9, P(C)=0.8. Find system reliability.
Find the probability that a connection exists from Input to Output.
Path goes through (A OR B), AND then through C.
Parallel (A,B): P(Work) = 1 - P(Both Fail) = 1 - (1-Pa)(1-Pb)
Series (X,Y): P(Work) = P(X) × P(Y)
P(A fails) = 1 - 0.9 = 0.1
P(B fails) = 1 - 0.9 = 0.1
P(Both A and B fail) = 0.1 × 0.1 = 0.01.
P(A || B works) = 1 - 0.01 = 0.99.
P(System) = P(A||B works) × P(C works)
Disease prevalence 0.1%. Test has 99% sensitivity (true positive) and 95% specificity (true negative). Patient tests positive. Find probability they have disease.
A random person walks in and gets a Positive result.
Is it "99% certain" they are sick? Or much less?
Find the Positive Predictive Value.
Prevalence is the "Prior". The test result updates this prior.
Base Rate Neglect: Don't ignore the 99.9% distinct healthy population!
P(D) = 0.001 (Sick)
P(H) = 0.999 (Healthy)
P(Pos | D) = 0.99 (True Positive)
P(Pos | H) = 1 - 0.95 = 0.05 (False Positive)
P(Pos ∩ D) = 0.99 × 0.001 = 0.00099
P(Pos) = P(Pos ∩ D) + P(Pos ∩ H)
= 0.00099 + (0.05 × 0.999)
= 0.00099 + 0.04995
= 0.05094
Shocking result: Even with a "99%" test, you are likely healthy (because the disease is so rare).