# Definitions

• Probability is the quantification of event uncertainty. For instance, probability of getting (H)eads in a coin toss is $$1/2$$. Deterministic models will give the same results given the same inputs (e.g. 2 times 2 is 4), but probabilistic models might yield different outcomes.

• An experiment is a process that generates data. For instance, tossing a coin is an experiment. Outcome is the realization of an experiment. Possible outcomes for a coin toss is Heads and Tails.

• Sample space ($$\mathbb{S}$$) is the collection of all the possible outcomes of an experiment. Sample space of the coin toss is $$\mathbb{S} = \{H,T\}$$. Sample space of two coin tosses experiment is $$\mathbb{S} = \{HH,HT,TH,TT\}$$. Sample space can be discrete (i.e. coin tosses) as well as continuous (i.e. All real numbers between 1 and 3. $$\mathbb{S} = \{x | 1 \le x \le 3, x \in \mathbb{R}\}$$) (Side note: Sample space is not always well defined.)

• An event is a subset of sample space. While outcome represents a realization, event is an information. Probability of an event $$P(A)$$, say getting two Heads in two coin tosses is $$P(A) = 1/4$$.

• A random variable represents an event is dependent on a probabilistic process. On the other hand, a deterministic variable is either a constant or a decision variable. For instance, value of the dollar tomorrow can be considered a random variable but the amount I will invest is a decision variable (subject to no probabilistic process) and spot (current) price of the dollar is a constant.

## Set Operations

• Complement of an event ($$A^\prime$$) with respect to the sample space represents all elements of the sample space that are not included by the event (A). For instance, complement of event $$A=\{HH\}$$ is $$A^\prime=\{HT,TH,TT\}$$

• Union of two events $$A$$ and $$B$$ ($$A \cup B$$) is a set of events which contains all elements of the respective events. For example, say $$A$$ is the set that contains events which double Heads occur ($$A = \{HH,HT,TH\}$$) and $$B$$ is the set which Tails occur at least once ($$B = \{TT,HT,TH\}$$). The union is $$A \cup B = \{HH,TH,HT,TT\}$$.

• Intersection of two events $$A$$ and $$B$$ ($$A \cap B$$) contains the common elements of the events. For example, say $$A$$ is the set that contains events which Heads occur at least once ($$A = \{HH,HT,TH\}$$) and $$B$$ is the set which Tails occur at least once ($$B = \{TT,HT,TH\}$$). The intersection is $$A \cap B = \{TH,HT\}$$.

• Mutually exclusive or disjoint events mean that two events have empty intersection ($$A \cap B = \emptyset$$) and their union ($$A \cup B$$) contains the same amount of elements as the sum of their respective number of elements. Also $$P(A \cap B) = 0$$ and $$P(A \cup B) = P(A) + P(B)$$. For example getting double Heads ($$HH$$) and double Tails ($$TT$$) are mutually exclusive events.

### Axioms of Probability

1. Any event $$A$$ belonging to the sample space $$A \in \mathbb{S}$$ should have nonnegative probability ($$P(A) \ge 0$$).
2. Probability of the sample space is one ($$P(\mathbb{S}) = 1$$).
3. Any disjoint events ($$A_i \cap A_j = \emptyset \ \forall_{i,j \in 1 \dots n}$$) satisfies $$P(A_1 \cup A_2 \cup \dots \cup A_n) = P(A_1) + P(A_2) + \dots + P(A_n)$$.

### Other Set and Probability Rules

• $$(A^\prime)^\prime = A$$
• $$S^\prime = \emptyset$$
• $$\emptyset^\prime = S$$
• $$(A \cap B)^\prime = A^\prime \cup B^\prime$$
• $$(A \cup B)^\prime = A^\prime \cap B^\prime$$
• $$(A \cup B) \cap C = (A \cap C) \cup (B \cap C)$$
• $$(A \cap B) \cup C = (A \cup C) \cap (B \cup C)$$
• $$(A \cup B) \cup C = A \cup (C \cup B)$$
• $$(A \cap B) \cap C = A \cap (C \cap B)$$
• $$A \cup A^\prime = \mathbb{S}$$ and $$A \cap A^\prime = \emptyset$$ so $$P(A) = 1 - P(A^\prime)$$. This is especially useful for many problems. For example the probability of getting at least one Heads in a three coin tosses in a row is $$1 - P(\{TTT\}) = 7/8$$, the complement of no Heads in a three coin tosses in a row. Otherwise, you should calculate the following expression.

$P(\{HTT\}) + P(\{THT\}) + P(\{TTH\}) + P(\{HHT\}) + P(\{HTH\}) + P(\{THH\}) + P(\{HHH\}) = 7/8$

• If $$A \subseteq B$$ then $$P(A) \le P(B)$$.
• $$P(A \cup B) = P(A) + P(B) - P(A \cap B)$$.
• $$P(A \cup B \cup C) = P(A) + P(B) + P(C) - P(A \cap B) - P(B \cap C) - P(A \cap C) + P(A \cap B \cap C)$$

# Counting

Counting rules will help us enumerate the sample space. It will include multiplication rule, permutation and combination.

## Multiplication Rule

If I have a series of independent events, say $$1$$ to $$k$$, and number of possible outcomes are denoted with $$n_1$$ to $$n_k$$; total number of outcomes in the sample space would be $$n_1n_2\dots n_k$$.

Take a series of coin tosses in a row. If I toss a coin its sample space consists of 2 elements such as $$\{H,T\}$$. If I toss 2 coins the sample space would be 2*2 $$\{HH,HT,TH,TT\}$$. If I toss 3 coins, the sample space would be 2*2*2 $$\{HHH,HTH,THH,TTH,HHT,HTT,THT,TTT\}$$.

A poker card consists of a type and a rank. There are four types of playing cards (clubs, diamonds, hearts and spades) and 13 ranks (A - 2 to 10 - J - Q - K). Number of cards in a deck is 4*13 = 52.

## Permutation Rule

Permutation is the arrangement of all or a subset of items.

• Given a set of items, say $$A = {a,b,c}$$ in how many different ways I can order the elements? Answer is n!. In our case it is, $$3! = 3.2.1 = 6$$.

$A = \{a,b,c\},\{b,a,c\},\{b,c,a\},\{c,a,b\},\{c,b,a\},\{a,c,b\}$

• Suppose there are 10 (n) participants in a competition and 3 (r) medals (gold, silver and bronze). How many possible outcomes are there? Answer is $$n(n-1)(n-2)\dots (n-r+1) = \dfrac{n!}{(n-r)!} = \dfrac{10!}{(10-3)!} = 720$$.

• If there are more than one same type items in a sample, then the permutation becomes $$\dfrac{n!}{n_1!n_2!\dots n_k!}$$ where $$\sum n_i = n$$.

For example enumerate the different outcomes of four coin tosses which result in 2 heads and 2 tails. Answer is $$\dfrac{4!}{2!2!} = 6$$

$A = \{HHTT,HTTH,HTHT,THTH,THHT,TTHH\}$

## Combination Rule

Suppose we want to select $$r$$ items from $$n$$ items and the order does not matter. So the number of different outcomes can be found using $$\binom{n}{r} = \dfrac{n!}{(n-r)!r!}$$.

Out of 10 students how many different groups of 2 students can we generate? Answer $$\dfrac{10!}{8!2!} = 45$$