Set Operations

  • Complement of an event (\(A^\prime\)) with respect to the sample space represents all elements of the sample space that are not included by the event (A). For instance, complement of event \(A=\{HH\}\) is \(A^\prime=\{HT,TH,TT\}\)

  • Union of two events \(A\) and \(B\) (\(A \cup B\)) is a set of events which contains all elements of the respective events. For example, say \(A\) is the set that contains events which double Heads occur (\(A = \{HH,HT,TH\}\)) and \(B\) is the set which Tails occur at least once (\(B = \{TT,HT,TH\}\)). The union is \(A \cup B = \{HH,TH,HT,TT\}\).

  • Intersection of two events \(A\) and \(B\) (\(A \cap B\)) contains the common elements of the events. For example, say \(A\) is the set that contains events which Heads occur at least once (\(A = \{HH,HT,TH\}\)) and \(B\) is the set which Tails occur at least once (\(B = \{TT,HT,TH\}\)). The intersection is \(A \cap B = \{TH,HT\}\).

  • Mutually exclusive or disjoint events mean that two events have empty intersection (\(A \cap B = \emptyset\)) and their union (\(A \cup B\)) contains the same amount of elements as the sum of their respective number of elements. Also \(P(A \cap B) = 0\) and \(P(A \cup B) = P(A) + P(B)\). For example getting double Heads (\(HH\)) and double Tails (\(TT\)) are mutually exclusive events.

Axioms of Probability

  1. Any event \(A\) belonging to the sample space \(A \in \mathbb{S}\) should have nonnegative probability (\(P(A) \ge 0\)).
  2. Probability of the sample space is one (\(P(\mathbb{S}) = 1\)).
  3. Any disjoint events (\(A_i \cap A_j = \emptyset \ \forall_{i,j \in 1 \dots n}\)) satisfies \(P(A_1 \cup A_2 \cup \dots \cup A_n) = P(A_1) + P(A_2) + \dots + P(A_n)\).

Other Set and Probability Rules

  • \((A^\prime)^\prime = A\)
  • \(S^\prime = \emptyset\)
  • \(\emptyset^\prime = S\)
  • \((A \cap B)^\prime = A^\prime \cup B^\prime\)
  • \((A \cup B)^\prime = A^\prime \cap B^\prime\)
  • \((A \cup B) \cap C = (A \cap C) \cup (B \cap C)\)
  • \((A \cap B) \cup C = (A \cup C) \cap (B \cup C)\)
  • \((A \cup B) \cup C = A \cup (C \cup B)\)
  • \((A \cap B) \cap C = A \cap (C \cap B)\)
  • \(A \cup A^\prime = \mathbb{S}\) and \(A \cap A^\prime = \emptyset\) so \(P(A) = 1 - P(A^\prime)\). This is especially useful for many problems. For example the probability of getting at least one Heads in a three coin tosses in a row is \(1 - P(\{TTT\}) = 7/8\), the complement of no Heads in a three coin tosses in a row. Otherwise, you should calculate the following expression.

    \[P(\{HTT\}) + P(\{THT\}) + P(\{TTH\}) + P(\{HHT\}) + P(\{HTH\}) + P(\{THH\}) + P(\{HHH\}) = 7/8 \]

  • If \(A \subseteq B\) then \(P(A) \le P(B)\).
  • \(P(A \cup B) = P(A) + P(B) - P(A \cap B)\).
  • \(P(A \cup B \cup C) = P(A) + P(B) + P(C) - P(A \cap B) - P(B \cap C) - P(A \cap C) + P(A \cap B \cap C)\)


Counting rules will help us enumerate the sample space. It will include multiplication rule, permutation and combination.

Multiplication Rule

If I have a series of independent events, say \(1\) to \(k\), and number of possible outcomes are denoted with \(n_1\) to \(n_k\); total number of outcomes in the sample space would be \(n_1n_2\dots n_k\).

Take a series of coin tosses in a row. If I toss a coin its sample space consists of 2 elements such as \(\{H,T\}\). If I toss 2 coins the sample space would be 2*2 \(\{HH,HT,TH,TT\}\). If I toss 3 coins, the sample space would be 2*2*2 \(\{HHH,HTH,THH,TTH,HHT,HTT,THT,TTT\}\).

A poker card consists of a type and a rank. There are four types of playing cards (clubs, diamonds, hearts and spades) and 13 ranks (A - 2 to 10 - J - Q - K). Number of cards in a deck is 4*13 = 52.

Permutation Rule

Permutation is the arrangement of all or a subset of items.

  • Given a set of items, say \(A = {a,b,c}\) in how many different ways I can order the elements? Answer is n!. In our case it is, \(3! = 3.2.1 = 6\).

    \[A = \{a,b,c\},\{b,a,c\},\{b,c,a\},\{c,a,b\},\{c,b,a\},\{a,c,b\}\]

  • Suppose there are 10 (n) participants in a competition and 3 (r) medals (gold, silver and bronze). How many possible outcomes are there? Answer is \(n(n-1)(n-2)\dots (n-r+1) = \dfrac{n!}{(n-r)!} = \dfrac{10!}{(10-3)!} = 720\).

  • If there are more than one same type items in a sample, then the permutation becomes \(\dfrac{n!}{n_1!n_2!\dots n_k!}\) where \(\sum n_i = n\).

    For example enumerate the different outcomes of four coin tosses which result in 2 heads and 2 tails. Answer is \(\dfrac{4!}{2!2!} = 6\)


Combination Rule

Suppose we want to select \(r\) items from \(n\) items and the order does not matter. So the number of different outcomes can be found using \(\binom{n}{r} = \dfrac{n!}{(n-r)!r!}\).

Out of 10 students how many different groups of 2 students can we generate? Answer \(\dfrac{10!}{8!2!} = 45\)