Chapter 1 Initial Concepts of Probability

Probability is the quantification of event uncertainty. For instance, probability of getting (H)eads in a coin toss is \(1/2\). Deterministic models will give the same results given the same inputs (e.g. 2 times 2 is 4), but probabilistic models might yield different outcomes.
An experiment is a process that generates data. For instance, tossing a coin is an experiment. Outcome is the realization of an experiment. Possible outcomes for a coin toss is Heads and Tails.
Sample space (\(\mathbb{S}\)) is the collection of all the possible outcomes of an experiment. Sample space of the coin toss is \(\mathbb{S} = \{H,T\}\). Sample space of two coin tosses experiment is \(\mathbb{S} = \{HH,HT,TH,TT\}\). Sample space can be discrete (i.e. coin tosses) as well as continuous (i.e. All real numbers between 1 and 3. \(\mathbb{S} = \{x | 1 \le x \le 3, x \in \mathbb{R}\}\)) (Side note: Sample space is not always well defined.)
An event is a subset of sample space. While outcome represents a realization, event is an information. Probability of an event \(P(A)\), say getting two Heads in two coin tosses is \(P(A) = 1/4\).
A random variable represents an event is dependent on a probabilistic process. On the other hand, a deterministic variable is either a constant or a decision variable. For instance, value of the dollar tomorrow can be considered a random variable but the amount I will invest is a decision variable (subject to no probabilistic process) and spot (current) price of the dollar is a constant.

1.1 Set Operations

Complement of an event (\(A^\prime\)) with respect to the sample space represents all elements of the sample space that are not included by the event (A). For instance, complement of event \(A=\{HH\}\) is \(A^\prime=\{HT,TH,TT\}\)
Union of two events \(A\) and \(B\) (\(A \cup B\)) is a set of events which contains all elements of the respective events. For example, say \(A\) is the set that contains events which double Heads occur (\(A = \{HH,HT,TH\}\)) and \(B\) is the set which Tails occur at least once (\(B = \{TT,HT,TH\}\)). The union is \(A \cup B = \{HH,TH,HT,TT\}\).
Intersection of two events \(A\) and \(B\) (\(A \cap B\)) contains the common elements of the events. For example, say \(A\) is the set that contains events which Heads occur at least once (\(A = \{HH,HT,TH\}\)) and \(B\) is the set which Tails occur at least once (\(B = \{TT,HT,TH\}\)). The intersection is \(A \cap B = \{TH,HT\}\).
Mutually exclusive or disjoint events mean that two events have empty intersection (\(A \cap B = \emptyset\)) and their union (\(A \cup B\)) contains the same amount of elements as the sum of their respective number of elements. Also \(P(A \cap B) = 0\) and \(P(A \cup B) = P(A) + P(B)\). For example getting double Heads (\(HH\)) and double Tails (\(TT\)) are mutually exclusive events.

1.2 Axioms of Probability

Any event \(A\) belonging to the sample space \(A \in \mathbb{S}\) should have nonnegative probability (\(P(A) \ge 0\)).
Probability of the sample space is one (\(P(\mathbb{S}) = 1\)).
Any disjoint events (\(A_i \cap A_j = \emptyset \ \forall_{i,j \in 1 \dots n}\)) satisfies \(P(A_1 \cup A_2 \cup \dots \cup A_n) = P(A_1) + P(A_2) + \dots + P(A_n)\).

1.3 Other Set and Probability Rules

\((A^\prime)^\prime = A\)
\(S^\prime = \emptyset\)
\(\emptyset^\prime = S\)
\((A \cap B)^\prime = A^\prime \cup B^\prime\)
\((A \cup B)^\prime = A^\prime \cap B^\prime\)
\((A \cup B) \cap C = (A \cap C) \cup (B \cap C)\)
\((A \cap B) \cup C = (A \cup C) \cap (B \cup C)\)
\((A \cup B) \cup C = A \cup (C \cup B)\)
\((A \cap B) \cap C = A \cap (C \cap B)\)
\(A \cup A^\prime = \mathbb{S}\) and \(A \cap A^\prime = \emptyset\) so \(P(A) = 1 - P(A^\prime)\). This is especially useful for many problems. For example the probability of getting at least one Heads in a three coin tosses in a row is \(1 - P(\{TTT\}) = 7/8\), the complement of no Heads in a three coin tosses in a row. Otherwise, you should calculate the following expression.

\[P(\{HTT\}) + P(\{THT\}) + P(\{TTH\}) + P(\{HHT\}) + P(\{HTH\}) + P(\{THH\}) + P(\{HHH\}) = 7/8 \]
If \(A \subseteq B\) then \(P(A) \le P(B)\).
\(P(A \cup B) = P(A) + P(B) - P(A \cap B)\).
\(P(A \cup B \cup C) = P(A) + P(B) + P(C) - P(A \cap B) - P(B \cap C) - P(A \cap C) + P(A \cap B \cap C)\)

1.4 Counting

Counting rules will help us enumerate the sample space. It will include multiplication rule, permutation and combination.

1.4.1 Multiplication Rule

If I have a series of independent events, say \(1\) to \(k\), and number of possible outcomes are denoted with \(n_1\) to \(n_k\); total number of outcomes in the sample space would be \(n_1n_2\dots n_k\).

Take a series of coin tosses in a row. If I toss a coin its sample space consists of 2 elements such as \(\{H,T\}\). If I toss 2 coins the sample space would be 2*2 \(\{HH,HT,TH,TT\}\). If I toss 3 coins, the sample space would be 2*2*2 \(\{HHH,HTH,THH,TTH,HHT,HTT,THT,TTT\}\).

A poker card consists of a type and a rank. There are four types of playing cards (clubs, diamonds, hearts and spades) and 13 ranks (A - 2 to 10 - J - Q - K). Number of cards in a deck is 4*13 = 52.

1.4.2 Permutation Rule

Permutation is the arrangement of all or a subset of items.

Given a set of items, say \(A = {a,b,c}\) in how many different ways I can order the elements? Answer is n!. In our case it is, \(3! = 3.2.1 = 6\).

\[A = \{a,b,c\},\{b,a,c\},\{b,c,a\},\{c,a,b\},\{c,b,a\},\{a,c,b\}\]
Suppose there are 10 (n) participants in a competition and 3 (r) medals (gold, silver and bronze). How many possible outcomes are there? Answer is \(n(n-1)(n-2)\dots (n-r+1) = \dfrac{n!}{(n-r)!} = \dfrac{10!}{(10-3)!} = 720\).
If there are more than one same type items in a sample, then the permutation becomes \(\dfrac{n!}{n_1!n_2!\dots n_k!}\) where \(\sum n_i = n\).

For example enumerate the different outcomes of four coin tosses which result in 2 heads and 2 tails. Answer is \(\dfrac{4!}{2!2!} = 6\)

\[A = \{HHTT,HTTH,HTHT,THTH,THHT,TTHH\}\]

1.4.3 Combination Rule

Suppose we want to select \(r\) items from \(n\) items and the order does not matter. So the number of different outcomes can be found using \(\binom{n}{r} = \dfrac{n!}{(n-r)!r!}\).

Out of 10 students how many different groups of 2 students can we generate? Answer \(\dfrac{10!}{8!2!} = 45\)

1.5 Chapter Problems

Suppose I toss a coin, roll a die and draw a card from the deck. How many different number of outcomes are there for this experiment?

Solution: Multiplication rule. \(n_1n_2n_3 = 2.6.52 = 624\).
```
n1 <- 2 #A coin toss has two potential outcomes.
n2 <- 6 #A die roll has six potential outcomes.
n3 <- 52 #A card draw has 52 potential outcomes.
```
In how many ways can I order the Teletubbies? (Tinky-Winky, Dipsy, Laa Laa and Po) For instance, (TW - Dipsy - Po - Laa Laa) is an ordering and (Dipsy - Po - TW - Laa Laa) is another.

Solution: Permutation rule. \(n! = 4! = 24\)
```
n_tubbies <- 4 #Number of teletubbies
factorial(n_tubbies) #By permtuation it is 4!
```
```
## [1] 24
```

I want to reorder the letters of the phrase “GOODGRADES”. In how many ways can I do it?.

Solution: Remember the permutation rule with identical items. There are two “G”s, two “D”s and two “O”s. Remember the formula \(\dfrac{n!}{n_1!n_2!\dots n_k!}\). So the result should be \(\dfrac{10!}{2!2!2!1!1!1!1!} = 453600\).

the_phrase <- "GOODGRADES"
freq_table <- table(strsplit(the_phrase, split = "")[[1]])  #Let's create a frequency table first
print(freq_table)  #Let's show it

## 
## A D E G O R S 
## 1 2 1 2 2 1 1

the_dividend <- factorial(nchar(the_phrase))  #Dividend part is 10 characters so 10!
the_divisor <- prod(factorial(freq_table))  #Get multiplication of factorials for the divisor
the_dividend/the_divisor

## [1] 453600

I want to make two letter words from “GRADES” such as “GA”, “ED” or “DE” (it doesn’t have to make sense). Find the number of permutations.

Solution: Permutation of \(r\) items from \(n\) items is \(\dfrac{n!}{(n-r)!}\). So the result is \(\dfrac{6!}{4!} = 30\).
```
the_phrase<-"GRADES"
letter_length <- 2 #We want two letter words
#Since all letters are different no need for special permutation.
factorial(nchar(the_phrase))/factorial(nchar(the_phrase)-letter_length)
```
```
## [1] 30
```
Suppose I am drawing a hand of 5 cards from a playing deck of 52 cards. How many different hands there can be? (Each card is different. See the bottom of this document for details.)

Solution: Since in a hand you do not care for the order, it is the combination \(\binom{52}{5} = \dfrac{52!}{(52-5)!5!} = 2598960\).
```
#Combination (a.k.a binomial coefficient) function is choose
choose(52,5)
```
```
## [1] 2598960
```

1.6 Extra Problems

Question 1 Suppose we draw three cards from a deck and roll two dice. Answer the following questions.
1. What is the experiment?
  
  The experiment is “drawing three cards from a deck and rolling two dice”.
2. What is “getting two-sixes and three-kings or five-one (in any order) one queen one king one ace”? Pick one (Event / Outcome / Sample Space)
  
  Event.
3. Give an example of two mutually exclusive events. (6 pts)
  
  Event A: Queen of Hearts / Queen of Spades / Queen of Diamonds / 6 / 5 Event B: Ace of Clubs / King of Clubs / Queen of Clubs / 4 / 4
4. What is the probability of getting four-three (in any order) in dice roll and three queens in card draw?
```
#First roll can either be 3 or 4 and second roll should be the other
# 2/6 * 1/6
#Getting the first Queen has probability of 4/52
#Getting the second Queen has probability of 3/51
#Getting the third Queen has probability of 2/50
2/6*1/6*4/52*3/51*2/50
```
```
## [1] 1.00553e-05
```
5. How many different outcomes can there be? This time assume ordering is important (e.g. 6-1 and 1-6 are different outcomes).
```
#Six outcomes per die
#52 outcomes for the first card draw
#51 outcomes for the second card draw
#50 outcomes for the second card draw
#Multiplication rule
#You can also use permutation rule for cards
6*6*52*51*50
```
```
## [1] 4773600
```

Question 2 In how many ways can you arrange the letters of “HOUSEPARTY”?

Any order.

the_phrase<-"HOUSEPARTY"
#No repetitive letters
#Permutation rule
factorial(nchar(the_phrase))

## [1] 3628800

Vowels together?

#4 vowels, 6 consonants
#Assume all vowels are a single "letter". So 8 characters.
#But vowels permutate within the single "letter".
#Multiplication rule
factorial(6+1)*factorial(4)

## [1] 120960

Vowels in alphabetical order?

#We start with all the permutations 10!
#For any permutation there can be only one ordering of vowels.
#For instance HOUSEPARTY is not valid but HAESOPURTY is valid
#So remove invalid permutations with division
factorial(10)/factorial(4)

## [1] 151200

There should be no consecutive vowels?

#There are 6 consonants, 4 vowels.
# Assume Xs are consonants and .s are potent vowel places.
# .X.X.X.X.X.X.
#Consonants can permutate in any order so 6! there
#7 places for vowels but only 4 vowels.
# So it is a permutation of 4 out of 8 places.
factorial(6)*(factorial(7)/factorial(7-4))

## [1] 604800

Question 3 In how many ways can you arrange the letters of “CAMARADERIE”?

# 11 characters.
# 6 vowels, 5 consonants
# 3 As, 2 Es, 2 Rs

Any order.

#By the formula of permutation with repetitive letters
#Assign the value to all_perms object
all_perms<-factorial(11)/(factorial(3)*factorial(2)*factorial(2))
all_perms

## [1] 1663200

Vowels together?

#Assume all vowels are single "character" again. So 6 characters
(factorial(5+1)/(factorial(2)))*(factorial(6)/(factorial(3)*factorial(2)))

## [1] 21600

Vowels in alphabetical order?

#Same as the last question. But be careful about identical vowels.
all_perms/(factorial(6)/(factorial(3)*factorial(2)))

## [1] 27720

There should be no consecutive vowels?

#Same as the last question. But be careful about identical vowels.
(factorial(5)/factorial(2))*(factorial(6)/factorial(6-6))/(factorial(3)*factorial(2))

## [1] 3600

Question 4

Suppose you are putting the top 12 basketball teams in 4 groups evenly (each group should consist of 3 teams). In how many different ways can you arrange the teams?
```
#It is either a chain of combinations or just grouping combination
choose(12,3)*choose(9,3)*choose(6,3)
```
```
## [1] 369600
```
Question 5 (20 pts - all equal)

There are 18 people; 10 from Izmir, 8 from Mugla.
1. Suppose you want to form a group of 5 people with at least 1 person from Izmir and Mugla. In how many ways can you form such a group?
```
#Calculate as if no rules. It is the combination of 18 to 5.
#Then remove the combinations of all Izmir or all Mugla people
choose(18,5) - choose(10,5) - choose(8,5)
```
```
## [1] 8260
```
2. In how many ways can you form a group of 3 people from Izmir and 4 people from Mugla?
```
#Simply separate combinations with multiplication rule.
choose(10,3)*choose(8,4)
```
```
## [1] 8400
```

1.7 Coins, Dice and Cards

When questions mention about coins, dice and cards they are commonly referred items. Nevertheless, you can refer to .

Coin tosses: Two possible outcomes. Heads or Tails.
Dice rolling: Six possible outcomes. 1-2-3-4-5-6.
Card drawing: 52 possible outcomes. There are 4 types (clubs, diamonds, spades and hearts) and 13 ranks for each type. (A)ce - 2 - 3 - 4 - 5 - 6 - 7 - 8 - 9 - 10 - (J)ack - (Q)ueen - (K)ing.