**Probability**is the quantification of event uncertainty. For instance, probability of getting (H)eads in a coin toss is \(1/2\). Deterministic models will give the same results given the same inputs (e.g. 2 times 2 is 4), but probabilistic models might yield different outcomes.An

**experiment**is a process that generates data. For instance, tossing a coin is an experiment.**Outcome**is the realization of an experiment. Possible outcomes for a coin toss is Heads and Tails.**Sample space**(\(\mathbb{S}\)) is the collection of all the possible outcomes of an experiment. Sample space of the coin toss is \(\mathbb{S} = \{H,T\}\). Sample space of two coin tosses experiment is \(\mathbb{S} = \{HH,HT,TH,TT\}\). Sample space can be discrete (i.e. coin tosses) as well as continuous (i.e. All real numbers between 1 and 3. \(\mathbb{S} = \{x | 1 \le x \le 3, x \in \mathbb{R}\}\))*(Side note: Sample space is not always well defined.)*An

**event**is a subset of sample space. While outcome represents a realization, event is an information. Probability of an event \(P(A)\), say getting two Heads in two coin tosses is \(P(A) = 1/4\).A

**random variable**represents an event is dependent on a probabilistic process. On the other hand, a**deterministic variable**is either a constant or a decision variable. For instance, value of the dollar tomorrow can be considered a random variable but the amount I will invest is a decision variable (subject to no probabilistic process) and spot (current) price of the dollar is a constant.

**Complement**of an event (\(A^\prime\)) with respect to the sample space represents all elements of the sample space that are not included by the event (A). For instance, complement of event \(A=\{HH\}\) is \(A^\prime=\{HT,TH,TT\}\)**Union**of two events \(A\) and \(B\) (\(A \cup B\)) is a set of events which contains all elements of the respective events. For example, say \(A\) is the set that contains events which double Heads occur (\(A = \{HH,HT,TH\}\)) and \(B\) is the set which Tails occur at least once (\(B = \{TT,HT,TH\}\)). The union is \(A \cup B = \{HH,TH,HT,TT\}\).**Intersection**of two events \(A\) and \(B\) (\(A \cap B\)) contains the common elements of the events. For example, say \(A\) is the set that contains events which Heads occur at least once (\(A = \{HH,HT,TH\}\)) and \(B\) is the set which Tails occur at least once (\(B = \{TT,HT,TH\}\)). The intersection is \(A \cap B = \{TH,HT\}\).**Mutually exclusive**or disjoint events mean that two events have empty intersection (\(A \cap B = \emptyset\)) and their union (\(A \cup B\)) contains the same amount of elements as the sum of their respective number of elements. Also \(P(A \cap B) = 0\) and \(P(A \cup B) = P(A) + P(B)\). For example getting double Heads (\(HH\)) and double Tails (\(TT\)) are mutually exclusive events.

- Any event \(A\) belonging to the sample space \(A \in \mathbb{S}\) should have nonnegative probability (\(P(A) \ge 0\)).
- Probability of the sample space is one (\(P(\mathbb{S}) = 1\)).
- Any disjoint events (\(A_i \cap A_j = \emptyset \ \forall_{i,j \in 1 \dots n}\)) satisfies \(P(A_1 \cup A_2 \cup \dots \cup A_n) = P(A_1) + P(A_2) + \dots + P(A_n)\).

- \((A^\prime)^\prime = A\)
- \(S^\prime = \emptyset\)
- \(\emptyset^\prime = S\)
- \((A \cap B)^\prime = A^\prime \cup B^\prime\)
- \((A \cup B)^\prime = A^\prime \cap B^\prime\)
- \((A \cup B) \cap C = (A \cap C) \cup (B \cap C)\)
- \((A \cap B) \cup C = (A \cup C) \cap (B \cup C)\)
- \((A \cup B) \cup C = A \cup (C \cup B)\)
- \((A \cap B) \cap C = A \cap (C \cap B)\)
\(A \cup A^\prime = \mathbb{S}\) and \(A \cap A^\prime = \emptyset\) so \(P(A) = 1 - P(A^\prime)\). This is especially useful for many problems. For example the probability of getting at least one Heads in a three coin tosses in a row is \(1 - P(\{TTT\}) = 7/8\), the complement of no Heads in a three coin tosses in a row. Otherwise, you should calculate the following expression.

\[P(\{HTT\}) + P(\{THT\}) + P(\{TTH\}) + P(\{HHT\}) + P(\{HTH\}) + P(\{THH\}) + P(\{HHH\}) = 7/8 \]

- If \(A \subseteq B\) then \(P(A) \le P(B)\).
- \(P(A \cup B) = P(A) + P(B) - P(A \cap B)\).
\(P(A \cup B \cup C) = P(A) + P(B) + P(C) - P(A \cap B) - P(B \cap C) - P(A \cap C) + P(A \cap B \cap C)\)

Counting rules will help us enumerate the sample space. It will include multiplication rule, permutation and combination.

If I have a series of independent events, say \(1\) to \(k\), and number of possible outcomes are denoted with \(n_1\) to \(n_k\); total number of outcomes in the sample space would be \(n_1n_2\dots n_k\).

Take a series of coin tosses in a row. If I toss a coin its sample space consists of 2 elements such as \(\{H,T\}\). If I toss 2 coins the sample space would be 2*2 \(\{HH,HT,TH,TT\}\). If I toss 3 coins, the sample space would be 2*2*2 \(\{HHH,HTH,THH,TTH,HHT,HTT,THT,TTT\}\).

A poker card consists of a type and a rank. There are four types of playing cards (clubs, diamonds, hearts and spades) and 13 ranks (A - 2 to 10 - J - Q - K). Number of cards in a deck is 4*13 = 52.

Permutation is the arrangement of all or a subset of items.

Given a set of items, say \(A = {a,b,c}\) in how many different ways I can order the elements? Answer is n!. In our case it is, \(3! = 3.2.1 = 6\).

\[A = \{a,b,c\},\{b,a,c\},\{b,c,a\},\{c,a,b\},\{c,b,a\},\{a,c,b\}\]

Suppose there are 10 (n) participants in a competition and 3 (r) medals (gold, silver and bronze). How many possible outcomes are there? Answer is \(n(n-1)(n-2)\dots (n-r+1) = \dfrac{n!}{(n-r)!} = \dfrac{10!}{(10-3)!} = 720\).

If there are more than one same type items in a sample, then the permutation becomes \(\dfrac{n!}{n_1!n_2!\dots n_k!}\) where \(\sum n_i = n\).

For example enumerate the different outcomes of four coin tosses which result in 2 heads and 2 tails. Answer is \(\dfrac{4!}{2!2!} = 6\)

\[A = \{HHTT,HTTH,HTHT,THTH,THHT,TTHH\}\]

Suppose we want to select \(r\) items from \(n\) items and the order does not matter. So the number of different outcomes can be found using \(\binom{n}{r} = \dfrac{n!}{(n-r)!r!}\).

Out of 10 students how many different groups of 2 students can we generate? Answer \(\dfrac{10!}{8!2!} = 45\)