Statistics.

1 · What statistics is, and the basic vocabulary

"Statistics is the art of learning from data." The work splits in two: descriptive statistics — summarising the data you have with tables, graphs and numbers — and inferential statistics — drawing conclusions about a larger group from a sample. This chapter is the descriptive half, and it's the foundation for everything later.

Five words you'll use constantly:

• Population: the whole collection you care about (e.g. all LUISS students).
• Sample: the subset you actually observe (e.g. the students in one Statistics class).
• Units (or subjects): the members — people, countries, days, objects.
• Variable: the feature you record on each unit (age, height, grade).
• Modalities: the possible values a variable can take (for age: 18, 19, 20, …).

Think of the data as a table: one row per unit, one column per variable. Everything in this chapter is a way of squeezing such a table into something you can actually read.

2 · Types of variable — this decides what you can do

Before computing anything, classify the variable — it dictates which summaries make sense.

• Categorical (qualitative) — values are categories, not real magnitudes.
  • Nominal: categories with no order — hair colour, favourite team, laptop OS.
  • Ordinal: categories with a natural order — education level, hotel stars (1–5), film ratings (G, PG, …).
• Quantitative (numerical) — values are genuine numbers.
  • Discrete: countable values — number of goals, exam grade.
  • Continuous: any value in a range — height, time, the length of an episode.

Careful: a number used as a label is still categorical. Hotel stars run 1–5 but they're ordered categories, not a measured quantity. You wouldn't average shirt numbers on a football team.

Why it matters: for a categorical variable the useful summary is "what fraction falls in each category"; for a quantitative one it's "where is the centre and how spread out is it".

3 · Frequency tables

The first summary: count how often each value appears.

• Absolute frequency \(n_i\): how many units have value \(v_i\) (just count).
• Relative frequency \(f_i=n_i/n\): the proportion — these sum to 1.
• Cumulative relative frequency \(F_i=f_1+\dots+f_i\): the proportion of units up to value \(v_i\) (only for ordered data).

Course example — favourite Thanksgiving pie (2238 US adults, 2015): Pumpkin 729 (\(f=0.33\)), Apple 514 (0.23), Pecan 342 (0.15), … summing to 1. A wall of 2238 raw answers tells you nothing; the table tells you Pumpkin wins.

Cumulative example — 1000 customers' satisfaction (ordinal): Very unhappy 0.315, Quite unhappy 0.123 → \(F=0.438\) are unhappy; the rest are neutral-or-better. Cumulative frequencies answer "what proportion is at most this level?"

For a continuous variable every value is basically unique, so you group into classes (intervals) like \([0,10),[10,20),\dots\) and count per class (classes needn't be equal width).

4 · Graphs: pie, bar, histogram

Pick the graph to match the variable type:

• Pie chart (categorical): each category is a slice; slice angle \(=360^\circ\cdot f_i\).
• Bar plot (categorical or discrete): each value is a bar whose height = its frequency.
• Histogram (continuous, grouped): each class is a bar whose area = its frequency. This is the key difference from a bar plot — with unequal class widths, area (not height) carries the count, so a wide class isn't made to look bigger than it is.

A histogram reveals the data's shape at a glance: where the values pile up, whether it's symmetric, gaps, and outliers. When comparing two groups of different sizes, switch to relative frequencies first so the comparison is fair.

5 · Where is the centre? Mode, median, mean

Three ways to say "typical value":

• Mode: the most frequent value (can be more than one). For grouped data, the modal class = tallest histogram bar. Works for any variable type.
• Median: the middle of the ordered data. Order the values; if \(n\) is odd it's the middle one, if even the average of the two middle ones. Equivalently, the smallest value with \(F_i\ge0.5\). It ignores how extreme the values are — only their order.
• Mean \(\bar x=\dfrac1n\sum_i x_i\): the arithmetic average. From a frequency table it's the weighted form \(\bar x=\sum_j v_j f_j\); for grouped data, use class midpoints \(c_j\): \(\bar x\approx\sum_j c_j f_j\).

Mean vs median — the robustness lesson. The mean uses every value, so a few extreme ones drag it; the median doesn't budge. Pets per family in a building: values mostly 0–2 but a couple of 16–17 → mean \(\approx3.05\), median \(=1\). The median better reflects "a typical family". Mean and median coincide only when the data is symmetric.

6 · Percentiles & quartiles

The 100p-th percentile is the value with a fraction \(p\) of the data at or below it. Rule to locate it in the ordered sample of size \(n\):

• if \(np\) is not an integer → take the value at position \(\lceil np\rceil\) (round up);
• if \(np\) is an integer → average the values at positions \(np\) and \(np+1\).

Quartiles are the 25th, 50th (=median) and 75th percentiles: \(Q_1,Q_2,Q_3\). They cut the data into four equal quarters.

Worked (18 bowling scores, ordered). \(Q_1\): \(0.25\cdot18=4.5\to\) 5th value \(=145\). Median: \(0.5\cdot18=9\to\) average of 9th & 10th \(=159.5\). \(Q_3\): \(0.75\cdot18=13.5\to\) 14th value \(=177\). (Exercise 2 below walks this through.)

7 · How spread out? Range, variance, standard deviation

Centre isn't enough — two datasets can share a mean yet look completely different. Measures of spread:

• Range \(=x_{(n)}-x_{(1)}\) (max − min). Interquartile range \(\mathrm{IQR}=Q_3-Q_1\) — the width holding the middle 50%, unaffected by outliers.
• Variance. Natural idea: average the distances from the mean. But \(\sum_i(x_i-\bar x)=0\) always (positives cancel negatives), so we square the deviations first:

The two forms are equal; the right one is faster by hand. The divisor is \(n-1\), not \(n\) — "for technical reasons that become clear in the estimation chapter" (it makes \(s^2\) unbiased; you'll prove it later).

• Standard deviation \(s=\sqrt{s^2}\): same units as the data, so it's the interpretable one.

Worked (first 10 of the course's exam grades): 25,16,30,24,24,21,30,27,24,30. \(\sum x_i=251\Rightarrow\bar x=25.1\); \(\sum x_i^2=6479\); \(s^2=\dfrac{6479-10(25.1)^2}{9}=\dfrac{178.9}{9}\approx19.9\), so \(s\approx4.46\). (Exercise 1.)

Chebyshev's inequality ties spread to concentration for any shape: at most \(1/k^2\) of the data lies more than \(k\) standard deviations from the mean (so at most \(1/4\) lies outside \(\bar x\pm2s\)).

8 · Shape: symmetric, skewed, normal

The histogram's shape matters as much as its centre.

• Approximately normal: bell-shaped — tallest in the middle, symmetric, tapering on both sides. Lots of real data looks like this.
• Skewed: one long tail. Right-skewed = long tail to the right (income, pets-per-family); left-skewed = long tail to the left. A quick tell: if mean and median differ noticeably, the data is skewed (the mean is pulled toward the long tail).

A boxplot draws the five-number summary (min, \(Q_1\), median, \(Q_3\), max) — the box spans \(Q_1\) to \(Q_3\) with a line at the median — and is the easiest way to compare groups or spot skew (median off-centre in the box).

For approximately normal data, the Empirical Rule: about 68% of values lie within \(\bar x\pm s\), 95% within \(\bar x\pm2s\), 99.7% within \(\bar x\pm3s\). (This is the bridge to the Normal distribution chapter.)

9 · Two variables together: covariance & correlation

Often you want to know whether two variables move together. Plot each unit as a point \((x_i,y_i)\) — a scatterplot — and look.

Covariance measures the direction of a linear relationship:

When big-\(x\) goes with big-\(y\), the products are positive → positive covariance; opposite movement → negative. But its size depends on the units, so you can't tell "strong" from "weak". Fix that by rescaling into the correlation coefficient:

\(r\) is unit-free. \(|r|\) near 1 = strong linear relationship (\(r=+1\) or \(-1\) means the points lie exactly on a line); \(r\) near 0 = weak linear relationship. Course example: LSD dose vs maths score gives \(r\approx-0.93\) — strong negative.

Two warnings. (1) \(r\) only sees linear structure — a perfect parabola can give \(r\approx0\), so "\(r\approx0\)" doesn't mean "unrelated". (2) Correlation is not causation: deaths from falling out of bed correlate \(0.96\) with the number of lawyers in Puerto Rico — a coincidence, or a lurking third variable, not cause and effect.

10 · Where the data comes from: sampling design

Summaries are only as good as the sample. A simple random sample picks units so that every possible group of that size is equally likely — the best general safeguard for representativeness. (Sampling the first 100 people entering a library to estimate a city's average age fails: libraries over-represent students and retirees.)

A stratified random sample first splits the population into groups (strata) and samples each in proportion. If customers are 70% teenagers and 30% adults, take 70%/30% — guaranteeing both groups appear. Stratification helps most when the strata genuinely differ on what you're measuring.

Finally, the distinction that drives all of inference: a parameter is a (usually unknown) number describing the whole population; a sample statistic (like \(\bar x\) or \(s\)) is what you compute from the sample to estimate it. Everything from chapter 8 onward is about going from statistic back to parameter.

11 · What the exam asks from this chapter

Descriptive questions on past papers cluster into three recognisable types:

Exercises 1–2 build the raw skills (mean, variance, quartiles); exercises 3–8 are the real exam problems of each type.

1 · What probability actually is

Start with the picture, not the formula. An experiment is any situation whose result you can't predict for sure, but where you do know the list of possible results. Tossing a coin is an experiment; so is "which operating system will my next laptop run?" or "what will Apple's share price be on Monday?"

Each single result is an outcome. The full list of possible outcomes is the sample space, written \(S\).

• Coin toss: \(S=\{\text{Heads},\text{Tails}\}\).
• Roll a die: \(S=\{1,2,3,4,5,6\}\).
• Next laptop OS: \(S=\{\text{Windows},\text{MacOS},\text{Linux},\dots\}\).

An event is just a statement about the result — "the die shows an even number", "I get Heads". Formally it's a subset of \(S\): the even-number event is the set \(\{2,4,6\}\). We say the event occurs when the actual outcome is one of the outcomes inside it.

So what is a probability? The most useful picture is long-run frequency: if you repeated the experiment over and over, the probability of an event is the proportion of times it would happen. Flip a fair coin thousands of times and the fraction landing Tails settles near \(0.5\) — that limiting fraction is the probability. (Real example: across the world about 105 boys are born for every 100 girls, year after year — so \(P(\text{newborn is male})\approx0.51\).) A probability is always a number between 0 and 1: 0 = never, 1 = certain.

2 · Combining events: AND, OR, NOT

Events are sets, so we combine them like sets. Picture a rectangle for \(S\) and circles inside it for events (a Venn diagram).

• OR — union \(A\cup B\): the outcomes in \(A\), or in \(B\), or in both. It occurs if at least one of them happens.
• AND — intersection \(A\cap B\): the outcomes in both. It occurs only if they happen together.
• NOT — complement \(A^c\): everything in \(S\) that is not in \(A\). It occurs exactly when \(A\) doesn't.

Mutually exclusive (disjoint) events can't happen at the same time — their intersection is empty, \(A\cap B=\varnothing\). "The die shows 2" and "the die shows 5" are mutually exclusive. The empty set \(\varnothing\) is the impossible event.

One identity worth seeing now, because it drives a lot of exam problems: "in \(A\) but not in \(B\)" is \(A\cap B^c\), and it equals what's in \(A\) minus the overlap. We'll turn that into numbers next.

3 · The rules of probability

Three basic rules (they just codify the frequency picture):

1. \(0\le P(A)\le1\) — proportions live between 0 and 1.
2. \(P(S)=1\) — something in the list always happens.
3. If \(A,B\) are mutually exclusive, \(P(A\cup B)=P(A)+P(B)\) — non-overlapping chances just add.

From these you derive the two you'll actually use:

Complement rule. Since \(A\) and \(A^c\) split \(S\): \(P(A^c)=1-P(A)\). (Heads has probability 0.4 ⇒ Tails has 0.6.) This is the engine behind "at least one" problems — it's usually easier to compute the opposite and subtract.

Addition rule (when events can overlap): \[ P(A\cup B)=P(A)+P(B)-P(A\cap B). \] Why subtract? If you just add \(P(A)+P(B)\), the overlap \(A\cap B\) gets counted twice, so you remove it once.

Concrete example (Ross 4.3). A shop takes Amex or VISA. 22% of customers carry Amex, 58% carry VISA, 14% carry both. Probability a customer has at least one card: \(0.22+0.58-0.14=0.66\). And "VISA but not Amex" \(=0.58-0.14=0.44\) — exactly the \(A\cap B^c\) idea from section 2.

4 · Equally likely outcomes — just count

When every outcome in \(S\) is equally likely (a fair die, a well-shuffled deck, "a person chosen at random"), probability becomes pure counting:

The phrase "chosen at random" is your signal that outcomes are equally likely.

• Fair die: \(P(\text{even})=3/6=1/2\).
• European roulette, bet on odd: numbers \(\{0,1,\dots,36\}\), 18 are odd ⇒ \(18/37\).
• Retirement centre (Ross 4.4): 420 members, 144 smokers ⇒ \(P(\text{smoker})=144/420=12/35\).

This is why counting techniques (section 9) matter: to get a probability you often just need to count the favourable outcomes and divide by the total.

5 · Conditional probability — updating on information

Often you learn something partway through. Conditional probability \(P(B\mid A)\) is "the probability of \(B\) given that \(A\) has happened."

The intuition (no formula yet). Roll two dice; you're told the first die is a 4. That knowledge shrinks the world: only six outcomes are still possible — \((4,1),\dots,(4,6)\). Among those, only \((4,6)\) makes the sum 10, so the chance is \(1/6\). You re-computed the probability inside a reduced sample space: \(A\) became your new \(S\).

Turning that into a formula — measure the overlap relative to the thing you now know:

The famous trap (Ross 4.10 / the two-children problem). A couple has two children; you learn at least one is a girl. Probability both are girls? Equally likely families are \(\{(g,g),(g,b),(b,g),(b,b)\}\). "At least one girl" rules out only \((b,b)\), leaving three equally likely cases; just one is \((g,g)\). So the answer is \(1/3\), not \(1/2\) — the extra information reshapes the sample space. (This exact reasoning powers exam exercise 5 below.)

Rearranging the formula gives the multiplication rule: \(P(A\cap B)=P(A)\,P(B\mid A)\) — useful for "draw two without replacement" problems, where the second draw's odds depend on the first.

6 · Independence — when information doesn't help

Sometimes knowing \(A\) tells you nothing about \(B\). Then \(P(B\mid A)=P(B)\), and the multiplication rule simplifies to the test you'll use:

Signals of independence: "with replacement", "i.i.d.", "each toss is fair" — separate trials that don't influence each other.

Why it's not automatic (Ross 4.13). Two fair dice. Let \(A=\)"first die is 3". Compare two events: \(B=\)"sum is 8" and \(C=\)"sum is 7". Knowing the first die is 3 changes the chance of an 8 (now you just need a 5 next) — so \(A,B\) are dependent. But the chance of a 7 stays \(1/6\) whatever the first die shows (there's always exactly one matching second die) — so \(A,C\) are independent. Same first event, opposite verdicts: independence is something you check, not assume.

For "at least one" across independent trials, lean on the complement: three children, \(P(\text{at least one girl})=1-P(\text{all boys})=1-(1/2)^3=7/8\).

7 · Total probability — averaging over cases

Often the thing you want depends on a hidden "case" or "cause". Split the problem by the case, compute each piece, and combine. If \(B\) either happens or not, then for any \(A\):

Read it as a weighted average: the chance of \(A\) in each case, weighted by how likely that case is. (It generalises to any set of mutually-exclusive cases \(B_1,\dots,B_k\): \(P(A)=\sum_i P(A\mid B_i)P(B_i)\).)

Concrete example (insurance). 30% of drivers are high-risk, 70% low-risk. A high-risk driver has an accident this year with probability 0.4, a low-risk one with probability 0.2. Overall chance a random driver has an accident: \[ P(\text{accident})=0.4(0.30)+0.2(0.70)=0.12+0.14=0.26. \] You couldn't answer without splitting by risk type — that's the whole move.

8 · Bayes' theorem — reasoning backwards

Total probability runs cause → effect. Bayes runs it backwards: you observed the effect, and you want the probability of the cause. "It rained — how likely is it that the morning had been sunny?" "The test is positive — how likely is the disease?"

Start from the definition \(P(\text{cause}\mid\text{effect})=\dfrac{P(\text{cause}\cap\text{effect})}{P(\text{effect})}\), write the top as \(P(\text{effect}\mid\text{cause})P(\text{cause})\), and expand the bottom with total probability:

The procedure: (1) name the causes \(H\) and the observed effect \(E\); (2) write the priors \(P(H)\) and the likelihoods \(P(E\mid H)\) straight from the text; (3) total probability gives the denominator; (4) divide.

The showcase example — why a positive test can still be reassuring (Ross 4.17). A blood test is 99% accurate when the disease is present (\(P(E\mid H)=0.99\)) and gives a false positive 2% of the time (\(P(E\mid H^c)=0.02\)). Only 0.5% of people have the disease (\(P(H)=0.005\)). You test positive — what's the chance you're actually sick? \[ P(H\mid E)=\frac{0.99(0.005)}{0.99(0.005)+0.02(0.995)}\approx0.199. \] About 20% — surprisingly low, because the huge healthy population produces many false positives that swamp the few true cases. This is exactly the engine behind exam exercises 3 and 4.

9 · Counting — tools for equally-likely problems

Section 4 said probability is often just counting favourable ÷ total. Here are the tools, introduced only because we need them.

Basic principle. If step 1 has \(n\) options and step 2 has \(m\), together there are \(n\cdot m\). (One man from 8, one woman from 12 → \(96\) pairs.)

Permutations / factorial. The number of orderings of \(n\) distinct objects is \(n!=n(n-1)\cdots2\cdot1\) (with \(0!=1\)). Order matters here.

Combinations. When order does not matter — choosing a group of \(k\) from \(n\):

Two reflexes for the exam: "all different" → count the complement (the few repeats) and subtract; "these specific items are included" → fix them, then count the choices for the remaining slots. Watch for repeated elements — e.g. the two identical P's in APPLE: \(\binom{5}{2}=10\) selections, only one is the pair PP.

10 · Which tool? — recognition guide

Once the concepts are in place, exam problems are about spotting which one applies.

From outcomes to numbers: what a random variable is

Until now an experiment gave you outcomes — "Heads", "the die shows 3", "the customer is left-handed". A random variable is one simple move on top of that: attach a number to each outcome.

Formally, a random variable \(X\) is a rule that assigns a real number to every outcome of an experiment. We write it with a capital letter (\(X,Y,N\)); the actual value it takes after the experiment is a lowercase letter (\(x\)).

• Toss a coin three times, let \(X=\) number of Heads. Then \(X\) can be \(0,1,2,3\).
• Pick a student, let \(X=1\) if left-handed, \(0\) if not. Then \(X\in\{0,1\}\).
• Play a game, let \(X=\) euros you walk away with. \(X\) might be \(-5,\,0,\,+10,\dots\)

Why bother? Because once outcomes are numbers you can average them, measure how spread they are, and feed them into every formula in the rest of the course.

Discrete vs continuous. A random variable is discrete when its possible values are separate points you can list — finite (\(0,1,2,3\)) or an endless but listable sequence (\(1,2,3,\dots\)). It is continuous when it can be any value in an interval (a height like \(172.4\,\)cm). This whole chapter is about the discrete case; continuous ones come later.

The probability mass function (pmf): the full ID card of a discrete RV

To know a discrete random variable completely you need two columns: the list of values it can take, and the probability of each. That table is the probability mass function (pmf), written

\[ p(x)=P\{X=x\}. \]

Read it as "the probability that \(X\) lands exactly on \(x\)". Ross also writes it \(p_X(x)\) when several variables are around.

A list of numbers is a valid pmf exactly when:

• (1) every probability is \(\ge 0\): \(p(x)\ge 0\);
• (2) they add up to one: \(\sum_x p(x)=1\).

Concrete example. In a population \(10\%\) of people are left-handed. Pick two people independently and let \(X=\) how many of the two are left-handed. Working out the cases gives the pmf:

Check it is valid: all entries \(\ge0\), and \(0.81+0.18+0.01=1\). Good. This one table is the variable — every question below ("what's the average? the spread?") is answered just by reading off these numbers.

Tip. If a problem gives you frequencies (counts) instead of probabilities, divide each count by the total to turn the frequency table into a pmf. That single step unlocks every formula in this chapter.

Expected value E[X]: the long-run average

The first number you summarise a random variable with is its expected value (also called the mean), written \(E[X]\) or \(\mu\). It is the weighted average of the possible values, each weighted by its probability:

\[ E[X]=\sum_i x_i\,p(x_i). \]

What it really means. \(E[X]\) is not "the value you should expect to see" on a single try — a fair die never shows \(3.5\). It is the average you would get over a long run of repeated experiments. Physically it is the balance point of the pmf: put weight \(p(x_i)\) at position \(x_i\) on a ruler, and \(E[X]\) is where the ruler balances.

Example — fair die. Values \(1,\dots,6\) each with probability \(\tfrac16\):

\[ E[X]=1\cdot\tfrac16+2\cdot\tfrac16+\dots+6\cdot\tfrac16=\frac{21}{6}=3.5. \]

Example — indicator. If \(X=1\) when an event \(A\) happens and \(0\) otherwise, then \(E[X]=1\cdot P(A)+0\cdot P(A^c)=P(A)\). The mean of a 0/1 variable is just the probability of the "1".

For the left-handed pmf above: \(E[X]=0(0.81)+1(0.18)+2(0.01)=0.20\). On average \(0.2\) of the two people are left-handed — a perfectly sensible number even though \(X\) itself is only ever \(0,1,\) or \(2\).

E[g(X)]: averaging a function of X (and why E[X²] ≠ (E[X])²)

Often you don't care about \(X\) itself but about some function of it — a payoff, a squared error, a transformed score. If \(Y=g(X)\), you do not need to build a new pmf for \(Y\). Just weight \(g\) by the original probabilities:

\[ E[g(X)]=\sum_i g(x_i)\,p(x_i). \]

(Sometimes called the "law of the unconscious statistician".)

Game example. Draw \(5\) balls; you win €1 per white ball and lose €1 per non-white. If \(X\) is the number of white balls, your balance is \(Y=g(X)=2X-5\). To get the expected balance you just plug \(2x_i-5\) into the sum — no separate table needed.

The crucial special case: \(E[X^2]\). Take \(g(x)=x^2\):

\[ E[X^2]=\sum_i x_i^2\,p(x_i). \]

For the left-handed pmf: \(E[X^2]=0^2(0.81)+1^2(0.18)+2^2(0.01)=0.18+0.04=0.22\).

Notice \(E[X^2]=0.22\) but \((E[X])^2=(0.20)^2=0.04\). They are not equal. For a non-linear \(g\), \(E[g(X)]\neq g(E[X])\) in general. You can only "push the average inside" for linear functions and sums — exactly the next two sections. This gap between \(E[X^2]\) and \((E[X])^2\) is precisely what variance measures.

Variance and standard deviation: measuring spread

The mean tells you the center; the variance tells you how far the values typically sit from that center. Definition:

\[ \operatorname{Var}(X)=E\big[(X-\mu)^2\big],\qquad \mu=E[X]. \]

You square the distance from the mean (so positives and negatives don't cancel) and average it. Bigger variance = more spread.

The formula you actually use. Expanding the square gives a much faster equivalent:

\[ \operatorname{Var}(X)=E[X^2]-(E[X])^2. \]

So the recipe is always: compute \(E[X]\), compute \(E[X^2]\), subtract the square of the first from the second.

Standard deviation. Variance is in squared units (euros², years²…), which is awkward. Take the square root to get back to the original units:

\[ \sigma=\operatorname{SD}(X)=\sqrt{\operatorname{Var}(X)}. \]

Example — fair die. We found \(E[X]=3.5\) and \(E[X^2]=\tfrac{91}{6}\). So

\[ \operatorname{Var}(X)=\frac{91}{6}-\left(\frac72\right)^2=\frac{91}{6}-\frac{49}{4}=\frac{35}{12}\approx2.917. \]

Example — 0/1 (Bernoulli) variable. If \(P(X=1)=p\), then \(E[X]=p\) and (since \(X^2=X\)) \(E[X^2]=p\), so

\[ \operatorname{Var}(X)=p-p^2=p(1-p). \]

Linear transformations: E[aX+b] and Var(aX+b)

Rescaling and shifting a variable — change of units, a fee added to a payoff — is a linear transformation \(Y=aX+b\). Two clean rules:

\[ E[aX+b]=a\,E[X]+b, \qquad \operatorname{Var}(aX+b)=a^2\,\operatorname{Var}(X). \]

Read the rules. For the mean, the constant \(b\) shifts it and the factor \(a\) scales it — average moves exactly the way the data moves. For the variance, adding \(b\) does nothing (shifting everything by the same amount doesn't change spread), and the scale factor comes out squared because variance is built from squared distances.

Worked example. Let \(X\) have \(E[X]=3\) and \(\operatorname{Var}(X)=2\). For \(Y=4+3X\) (so \(a=3,\,b=4\)):

\[ E[Y]=4+3(3)=13,\qquad \operatorname{Var}(Y)=3^2\cdot 2=18. \]

The standard deviation scales by \(|a|\) (not \(a^2\)): \(\operatorname{SD}(Y)=3\,\operatorname{SD}(X)\).

Sums of random variables: when do means and variances add?

Real problems often add several random variables (total earnings of a couple, total Heads in many tosses). Two rules, and one important asymmetry between them.

Means always add. No conditions, ever:

\[ E[X+Y]=E[X]+E[Y], \qquad E\!\left[\sum_{i=1}^{k}X_i\right]=\sum_{i=1}^{k}E[X_i]. \]

Variances add only when there is no interaction. In general

\[ \operatorname{Var}(X+Y)=\operatorname{Var}(X)+\operatorname{Var}(Y)+2\operatorname{Cov}(X,Y), \]

where the extra term \(\operatorname{Cov}(X,Y)=E[(X-\mu_X)(Y-\mu_Y)]\) is the covariance — it captures whether the two move together. When \(X\) and \(Y\) are independent (knowing one tells you nothing about the other), the covariance is \(0\) and the variances simply add:

\[ \operatorname{Var}(X+Y)=\operatorname{Var}(X)+\operatorname{Var}(Y),\qquad \operatorname{Var}\!\left(\sum_{i=1}^{k}X_i\right)=\sum_{i=1}^{k}\operatorname{Var}(X_i)\quad(\text{independent}). \]

Also useful: for independent \(X,Y\), \(E[XY]=E[X]\,E[Y]\). (Covariance and joint behaviour get a full chapter later; here you only need: means add unconditionally, variances add under independence.)

Decision box — when can I push the average inside? \(E[g(X)]=g(E[X])\) is safe only for: linear functions \(aX+b\), sums, and products of independent variables. For anything non-linear (like \(X^2\)), it fails.

Recap — the discrete-RV toolkit

Everything in this chapter is computed by reading a pmf table and summing.

• pmf valid: \(p(x)\ge0\) and \(\sum p(x)=1\). Frequencies → divide by total.
• Mean: \(E[X]=\sum x_i\,p(x_i)\) — long-run average / balance point.
• Function: \(E[g(X)]=\sum g(x_i)\,p(x_i)\); in particular \(E[X^2]=\sum x_i^2 p(x_i)\).
• Variance: \(\operatorname{Var}(X)=E[X^2]-(E[X])^2\); \(\operatorname{SD}=\sqrt{\operatorname{Var}}\).
• Linear: \(E[aX+b]=aE[X]+b\), \(\operatorname{Var}(aX+b)=a^2\operatorname{Var}(X)\).
• Sums: means always add; variances add when independent.

Exam reflex: a table of values with probabilities + a question about "average / expected / fair price" → \(E[X]\). About "spread / variability / standard deviation / risk" → \(\operatorname{Var}(X)=E[X^2]-(E[X])^2\). About "expected payoff of a deal / which option" → compute \(E[\cdot]\) of each option and compare.

Why named distributions: recognise the story, pull the formula off the shelf

Chapter 3 gave you the general machinery (pmf, \(E[X]\), \(\operatorname{Var}\)). But a handful of situations come up again and again — counting successes, waiting for a first success, counting rare events. For each, statisticians have already worked out the pmf, mean, and variance once and for all. These are the named discrete distributions.

Your job in an exam is almost never to derive them. It is to recognise the story in the problem text and then read the right formula off the shelf. So for each model below, learn three things: the story (when it applies), the pmf, and \(E[X]\) and \(\operatorname{Var}(X)\).

The four you must know: Bernoulli (one yes/no trial), Binomial (count successes in a fixed number of trials), Geometric (how long until the first success), Poisson (count of rare events at a known rate). A fifth, Hypergeometric, is covered lightly — it is out of exam scope but appears in the slides.

Bernoulli: a single yes/no trial

Story. One trial with exactly two outcomes — success (\(1\)) with probability \(p\), failure (\(0\)) with probability \(1-p\). A single coin flip, one inspected item, one customer who either buys or not.

pmf: \[ p(1)=p,\qquad p(0)=1-p, \] compactly \(P(X=x)=p^x(1-p)^{1-x}\) for \(x\in\{0,1\}\).

Mean and variance: \[ E[X]=p,\qquad \operatorname{Var}(X)=p(1-p). \]

(Both shown in Chapter 3: \(E[X]=p\) because the mean of a 0/1 variable is the probability of the 1; \(\operatorname{Var}=p(1-p)\) because \(X^2=X\).) Bernoulli is the atom: every Binomial is a sum of independent Bernoullis.

Binomial: count successes in n independent trials

Story. You repeat the same Bernoulli trial \(n\) times, independently, each with success probability \(p\). \(X=\) total number of successes. Recognition cues: a fixed number of trials \(n\), each trial independent, same \(p\), and you count how many succeed. "Out of 10 items, how many defective?" "Out of 4 components, how many work?"

pmf: \[ P(X=i)=\binom{n}{i}p^i(1-p)^{n-i},\qquad i=0,1,\dots,n, \] where \(\binom{n}{i}=\dfrac{n!}{i!\,(n-i)!}\) counts the ways to choose which \(i\) trials succeed.

Mean and variance: \[ E[X]=np,\qquad \operatorname{Var}(X)=np(1-p). \]

Both follow instantly from writing \(X=X_1+\dots+X_n\) (independent Bernoullis): means add (\(np\)) and, by independence, variances add (\(np(1-p)\)).

Common asks. "At least one" is easiest via the complement: \(P(X\ge1)=1-P(X=0)=1-(1-p)^n\). "At most one": \(P(X\le1)=P(X=0)+P(X=1)\). Reverse-engineering \(n,p\) from a stated \(E[X]\) and \(\operatorname{Var}(X)\) is a classic exam twist — solve \(np\) and \(np(1-p)\) together.

Geometric: how many trials until the first success

Story. Repeat an independent Bernoulli trial (prob \(p\)) until the first success, and let \(X=\) the trial on which it happens. Recognition cue: "draw/try until the first…", "how many attempts needed?". Unlike Binomial, the number of trials is not fixed — it is what you are counting.

pmf: \[ P(X=k)=p\,(1-p)^{k-1},\qquad k=1,2,3,\dots \] (the first \(k-1\) trials fail, the \(k\)-th succeeds).

Mean and variance: \[ E[X]=\frac1p,\qquad \operatorname{Var}(X)=\frac{1-p}{p^2}. \]

The mean \(1/p\) matches intuition: if success has probability \(\tfrac15\), you wait about \(5\) trials on average. The pmf \(p(1-p)^{k-1}\) (sometimes written with parameter \(\theta\)) also shows up in estimation problems, where its parameter is estimated by \(\hat p=1/\bar X\).

Poisson: counting rare events at a known rate

Story. Count how many times a rare event happens in a fixed interval of time or space, when events occur independently at a constant average rate \(\lambda\). "On average \(3\) accidents per week — probability of at least one?" "\(5\) claims per day on average." It is the law of rare events: misprints per page, calls per minute, particles per second, customers per hour.

pmf: \[ P(X=i)=e^{-\lambda}\,\frac{\lambda^i}{i!},\qquad i=0,1,2,\dots \] where \(\lambda>0\) is the average count for the interval considered.

Mean and variance — both equal \(\lambda\): \[ E[X]=\lambda,\qquad \operatorname{Var}(X)=\lambda. \] (So \(\operatorname{SD}=\sqrt\lambda\).) If the mean and variance of a count are roughly equal, Poisson is a natural model.

Scaling the rate. If the rate is "\(1\) per \(2\) minutes" and you look at a \(5\)-minute window, set \(\lambda = 5/2 = 2.5\) for that window. Match \(\lambda\) to the interval in the question. The handy complement again: \(P(X\ge1)=1-e^{-\lambda}\).

Poisson as an approximation to the Binomial

When you have many trials with a tiny success probability — \(n\) large, \(p\) small — the Binomial is awkward to compute but is almost exactly Poisson with

\[ \lambda = np. \]

\[ \binom{n}{i}p^i(1-p)^{n-i}\;\approx\;e^{-\lambda}\frac{\lambda^i}{i!},\qquad \lambda=np. \]

Rule of thumb: good when \(n\) is large (say \(\ge 20\)) and \(p\) small (say \(\le 0.05\)). Recognition cue in exams: a binomial setup with a huge \(n\) and a tiny \(p\) ("\(1000\) poker hands, each a full house with prob \(0.0014\)"). Switch to Poisson with \(\lambda=np\) and the arithmetic becomes trivial.

Hypergeometric (out of exam scope): sampling without replacement

Marked OUT-OF-EXAM — taught in the slides but not tested. Know the story so you can tell it apart from Binomial.

Story. A finite population has \(N\) "good" and \(M\) "bad" items. You draw \(n\) without replacement; \(X=\) number of good ones drawn. Because draws are not replaced, the trials are not independent — that is exactly why it is not Binomial.

pmf: \[ P(X=i)=\frac{\binom{N}{i}\binom{M}{n-i}}{\binom{N+M}{n}}. \]

Mean: with \(p=\dfrac{N}{N+M}\), \(E[X]=np\) (same as Binomial), but the variance carries a finite-population correction \(\operatorname{Var}(X)=np(1-p)\big(1-\tfrac{n-1}{N+M-1}\big)\).

Key distinction. With replacement (or a huge population) → Binomial. Without replacement from a small population → Hypergeometric. When \(N+M\) is large relative to \(n\), the two coincide.

Decision box — which discrete model is it?

Read the problem and match the cue:

• One yes/no trial → Bernoulli(\(p\)): \(E=p,\ \operatorname{Var}=p(1-p)\).
• Fixed \(n\) independent trials, count successes → Binomial(\(n,p\)): \(P=\binom{n}{i}p^i(1-p)^{n-i}\), \(E=np,\ \operatorname{Var}=np(1-p)\).
• Trials until first success → Geometric(\(p\)): \(P=p(1-p)^{k-1}\), \(E=1/p,\ \operatorname{Var}=(1-p)/p^2\).
• Count of rare events at rate \(\lambda\) per interval → Poisson(\(\lambda\)): \(P=e^{-\lambda}\lambda^i/i!\), \(E=\operatorname{Var}=\lambda\).
• Big \(n\), tiny \(p\) → approximate Binomial by Poisson with \(\lambda=np\).
• Draw without replacement, count good → Hypergeometric (out of exam).

Reflexes: "at least one" → \(1-P(0)\). "with/without replacement" decides Binomial vs Hypergeometric. "fixed trials, count hits" = Binomial; "how long until" = Geometric.

Why one variable at a time isn't enough

So far each random variable lived alone. But real questions involve two at once: height and weight, a student's satisfaction and their year, today's return and tomorrow's. To study how two variables move together, their separate distributions are not enough.

Here's the catch that motivates the whole chapter: the individual (marginal) distributions of \(X\) and \(Y\) do not determine how they relate. Two completely different relationships — one where \(X\) and \(Y\) are unrelated, one where high \(X\) forces high \(Y\) — can have the same marginals. To see the relationship you need the joint distribution: the probability of each pair of values.

The joint pmf: probability of each pair

For two discrete random variables \(X\) and \(Y\), the joint probability mass function gives the probability they take a particular pair of values:

\[ p(x_i,y_j)=P\{X=x_i,\ Y=y_j\}. \]

You lay it out as a table: rows are the values of \(X\), columns the values of \(Y\), each cell a probability. As with any pmf, all cells are \(\ge0\) and

\[ \sum_i\sum_j p(x_i,y_j)=1. \]

Example (independence by construction). Daily stock changes are independent and identically distributed, with \(P(\text{change}=0)=0.30\), \(P(\pm1)=0.20\), \(P(\pm2)=0.10\), \(P(\pm3)=0.05\). The probability of a specific 3-day path multiplies: \(P\{X_1=1,X_2=2,X_3=0\}=(0.20)(0.10)(0.30)=0.006\).

Marginals: recovering each variable from the table

Given the joint table you can always get each variable's own distribution back by summing across the other. The marginal pmf of \(X\) is the row sums; the marginal of \(Y\) is the column sums:

\[ p_X(x_i)=\sum_j p(x_i,y_j),\qquad p_Y(y_j)=\sum_i p(x_i,y_j). \]

(The name comes from writing these totals in the margins of the table.) Each marginal must itself sum to \(1\) — a handy check.

One-way street. You can always go joint → marginals. You cannot in general go marginals → joint: many different joint tables share the same row and column totals. That's exactly why the joint distribution carries information the marginals don't.

Independence of two random variables

\(X\) and \(Y\) are independent when knowing one tells you nothing about the other. For discrete variables this has a clean test: the joint factors into the product of the marginals, for every cell:

\[ p(x_i,y_j)=p_X(x_i)\,p_Y(y_j)\quad\text{for all }i,j. \]

How to check. Compute the marginals (row/column sums), then verify every cell equals row-total × column-total. If even one cell fails, the variables are dependent.

Under independence two shortcuts hold: \(E[XY]=E[X]\,E[Y]\), and (as we'll see) covariance is \(0\) so variances of sums simply add.

Expectation of functions of two variables

To average any function \(g(X,Y)\) — a combined score, a product, a sum — weight it by the joint probabilities:

\[ E[g(X,Y)]=\sum_i\sum_j g(x_i,y_j)\,p(x_i,y_j). \]

Two cases dominate:

• Sum: \(E[X+Y]=E[X]+E[Y]\) — always, no independence needed.
• Product: \(E[XY]=\sum_i\sum_j x_i y_j\,p(x_i,y_j)\). This equals \(E[X]\,E[Y]\) only when \(X,Y\) are independent.

The gap between \(E[XY]\) and \(E[X]E[Y]\) is precisely what the next idea — covariance — measures.

Covariance: do they move together?

Covariance measures the linear tendency of two variables to move together:

\[ \operatorname{Cov}(X,Y)=E[(X-\mu_X)(Y-\mu_Y)]. \]

When \(X\) is above its mean at the same time \(Y\) is above its mean (and vice versa), the product is positive on average → positive covariance. If one tends to be high when the other is low, it's negative.

Computational formula (the one you use):

\[ \operatorname{Cov}(X,Y)=E[XY]-E[X]\,E[Y]=E[XY]-\mu_X\mu_Y. \]

Sign tells the story: \(>0\) move together; \(<0\) move oppositely; \(=0\) no linear trend.

Properties: \(\operatorname{Cov}(X,Y)=\operatorname{Cov}(Y,X)\); \(\operatorname{Cov}(X,X)=\operatorname{Var}(X)\); \(\operatorname{Cov}(aX,Y)=a\operatorname{Cov}(X,Y)\); and if \(X,Y\) are independent then \(\operatorname{Cov}(X,Y)=0\).

Correlation: covariance on a fixed scale

Covariance has awkward units (units of \(X\) times units of \(Y\)) and its size depends on scale, so you can't tell "strong" from "weak". The correlation coefficient fixes this by dividing out both standard deviations:

\[ \operatorname{Corr}(X,Y)=\frac{\operatorname{Cov}(X,Y)}{\sqrt{\operatorname{Var}(X)\,\operatorname{Var}(Y)}}=\frac{\operatorname{Cov}(X,Y)}{\sigma_X\,\sigma_Y}. \]

It is always between \(-1\) and \(1\):

\[ -1\le\operatorname{Corr}(X,Y)\le 1. \]

• \(+1\): perfect increasing linear relation \(Y=a+bX\) with \(b>0\).
• \(-1\): perfect decreasing linear relation (\(b<0\)).
• \(0\): no linear relation (but a non-linear one can still exist).

The closer to \(\pm1\), the tighter the linear association; the sign matches the sign of the covariance.

Variance of a sum, with the covariance term

Chapter 3 promised the full rule; here it is. Adding two variables, the variance is not just the sum of variances — there is a covariance correction:

\[ \operatorname{Var}(X+Y)=\operatorname{Var}(X)+\operatorname{Var}(Y)+2\operatorname{Cov}(X,Y). \]

For many variables:

\[ \operatorname{Var}\!\left(\sum_{i=1}^n X_i\right)=\sum_{i=1}^n\operatorname{Var}(X_i)+\sum_{i\neq j}\operatorname{Cov}(X_i,X_j). \]

When the variables are independent, every covariance is \(0\) and the variances simply add — which is why, back in Chapters 3–4, "independent" was the magic word that let \(\operatorname{Var}(np(1-p))\) and friends fall out so cleanly.

Warning: zero covariance does NOT mean independent

Independence \(\Rightarrow\) \(\operatorname{Cov}=0\). The reverse is false: two variables can have zero covariance yet be completely dependent, because covariance only sees linear association.

Classic counterexample. Let \(X\) take \(-1,0,1\) each with probability \(\tfrac13\), and set \(Y=X^2\). Then \(Y\) is totally determined by \(X\) (as dependent as can be), yet:

\[ E[X]=0,\quad E[XY]=E[X^3]=\tfrac{-1+0+1}{3}=0,\quad \operatorname{Cov}(X,Y)=0-0\cdot E[Y]=0. \]

Covariance is \(0\) but \(X\) and \(Y\) are dependent (e.g. \(p(0,1)=0\neq p_X(0)\,p_Y(1)=\tfrac13\cdot\tfrac23\)). Takeaway: use the factorisation test \(p(x,y)=p_X(x)p_Y(y)\) to decide independence — never infer it from \(\operatorname{Cov}=0\) alone.

From mass to density: when values form a continuum

Discrete variables sat on separate points, each carrying a chunk of probability (the pmf). But many quantities — a waiting time, a height, a lifetime — can be any value in an interval. There are infinitely many possible values, so no single one can carry positive probability. We need a new tool: the probability density function \(f(x)\).

The shift in one sentence: for discrete variables, probability is the height of a bar; for continuous variables, probability is the area under a curve. Nothing else about expectation or variance changes in spirit — sums just become integrals.

The density function: probability is area

A continuous random variable \(X\) has a density \(f(x)\ge0\) such that the probability of landing in an interval is the area under \(f\) over that interval:

\[ P(a\le X\le b)=\int_a^b f(x)\,dx. \]

For \(f\) to be a valid density it must be nonnegative and enclose total area \(1\):

\[ f(x)\ge0,\qquad \int_{-\infty}^{\infty} f(x)\,dx=1. \]

A single point has zero probability. Since the area over a point is zero, \[ P(X=a)=\int_a^a f(x)\,dx=0. \] A practical consequence: for continuous variables \(\le\) and \(<\) (and \(\ge\) and \(>\)) give the same probability — endpoints never matter. \(f(x)\) itself is not a probability (it can exceed \(1\)); only areas are probabilities.

Finding the normalising constant

A very common exam setup gives a density "up to a constant" — \(f(x)=c\cdot(\text{shape})\) — and asks you to find \(c\). The rule is always the same: the total area must be \(1\), so

\[ \int_{-\infty}^{\infty} f(x)\,dx=1 \quad\Longrightarrow\quad c=\frac{1}{\int(\text{shape})\,dx}. \]

Example. Let \(f(x)=c(1-x^2)\) on \((-1,1)\), \(0\) elsewhere. Then

\[ \int_{-1}^{1} c(1-x^2)\,dx=c\Big[x-\tfrac{x^3}{3}\Big]_{-1}^{1}=c\cdot\tfrac43=1\ \Rightarrow\ c=\tfrac34. \]

Once \(c\) is known you compute any probability as an area, e.g. \(P(0

Expectation and variance by integration

Everything from Chapter 3 carries over with sums replaced by integrals:

\[ E[X]=\int_{-\infty}^{\infty} x\,f(x)\,dx,\qquad E[g(X)]=\int_{-\infty}^{\infty} g(x)\,f(x)\,dx, \]

\[ \operatorname{Var}(X)=E[X^2]-(E[X])^2=\int x^2 f(x)\,dx-(E[X])^2. \]

The linear-transformation rules are unchanged too: \(E[aX+b]=aE[X]+b\) and \(\operatorname{Var}(aX+b)=a^2\operatorname{Var}(X)\).

Example. For \(f(x)=x/2\) on \([0,2]\): \(E[X]=\int_0^2 x\cdot\tfrac{x}{2}\,dx=\tfrac12\cdot\tfrac{8}{3}=\tfrac43\), \(E[X^2]=\int_0^2 x^2\cdot\tfrac{x}{2}\,dx=2\), so \(\operatorname{Var}(X)=2-\tfrac{16}{9}=\tfrac29\).

The continuous Uniform on [α, β]

Story. Any value in \([\alpha,\beta]\) is equally likely — no part of the interval is favoured. The density is a flat line whose height makes the area \(1\):

\[ f(x)=\frac{1}{\beta-\alpha}\quad\text{for }x\in[\alpha,\beta]\ \ (0\text{ outside}). \]

(Base \(\times\) height \(=(\beta-\alpha)\cdot\frac{1}{\beta-\alpha}=1\).)

Probabilities are length ratios. For a sub-interval \([a,b]\subseteq[\alpha,\beta]\): \[ P(a

Mean and variance: \[ E[X]=\frac{\alpha+\beta}{2},\qquad \operatorname{Var}(X)=\frac{(\beta-\alpha)^2}{12}. \]

The mean is just the midpoint. Recognition cue: "arrives at a time uniformly between …", "no information, equally likely anywhere in the interval".

The Exponential: waiting times with no memory

Story. The time you wait for the next event when events happen at a constant rate \(\lambda\) (the continuous-time cousin of the Poisson). Lifetimes of components that don't wear out, time to the next phone call, service times in a queue.

Density (rate \(\lambda>0\)): \[ f(x)=\lambda e^{-\lambda x}\quad\text{for }x\ge0\ \ (0\text{ for }x<0). \]

Tail probability (the handy one — no integral needed): \[ P(X>t)=e^{-\lambda t},\qquad P(X\le t)=1-e^{-\lambda t}. \]

Mean and variance: \[ E[X]=\frac1\lambda,\qquad \operatorname{Var}(X)=\frac1{\lambda^2}. \] So a higher rate \(\lambda\) means shorter expected wait. Careful: \(\lambda\) is the rate; the mean is its reciprocal \(1/\lambda\).

Memoryless property. The exponential forgets how long it has already waited: \[ P(X>s+t\mid X>s)=P(X>t). \] An old component that hasn't failed is "as good as new" — its remaining life has the same distribution as a fresh one. This is the defining feature of the exponential.

Recap — the continuous toolkit

• Density: probability = area, \(P(a\le X\le b)=\int_a^b f\). Valid if \(f\ge0\) and \(\int f=1\). \(P(X=a)=0\), so \(\le\) and \(<\) coincide.
• Find a constant: set \(\int f=1\) and solve.
• Moments: \(E[X]=\int x f\), \(\operatorname{Var}=E[X^2]-(E[X])^2\); \(E[aX+b]=aE[X]+b\), \(\operatorname{Var}(aX+b)=a^2\operatorname{Var}(X)\).
• Uniform\([\alpha,\beta]\): \(f=\tfrac{1}{\beta-\alpha}\), \(E=\tfrac{\alpha+\beta}{2}\), \(\operatorname{Var}=\tfrac{(\beta-\alpha)^2}{12}\); sub-interval prob = length ratio.
• Exponential\((\lambda)\): \(f=\lambda e^{-\lambda x}\), \(P(X>t)=e^{-\lambda t}\), \(E=\tfrac1\lambda\), \(\operatorname{Var}=\tfrac1{\lambda^2}\), memoryless.

Exam reflex: "density given, find constant / a probability" → normalise then integrate. "equally likely in an interval" → Uniform, use length ratios. "waiting time / lifetime at constant rate / memoryless" → Exponential, use \(e^{-\lambda t}\).