## Thursday, 7 August 2014

### chapter 4- Example of Drawing a Card

Scenario-3:
Suppose that you have with you a standard deck of $52$ cards, and you shuffle it really well. You now draw a card at random from this deck. It is obvious that for a well-shuffled deck, you are equally likely to draw any of the $52$ cards. For example, if $E$ is the event that the card drawn is the king of Hearts while $F$ is the event that the card drawn is the seven of Diamonds, we have
 $P\left( E \right) = P\left( F \right)$
Since there are $52$ cards and each is equally likely to turn up, and also, one of the $52$ cards must turn up, the numerical value of the probability of any card being drawn is therefore $\dfrac{1}{{52}}$. For example,
 $P\left( E \right) = P\left( F \right) = \dfrac{1}{{52}}$
Now, let $G$ be the event that a spades is drawn. Since $G$ can occur in $13$ ways (there are $13$ cards in any suit), or in other words, the number of cases favorable to $G$ is $13$ out of the total $52$ possibilities, we must have
 $P\left( G \right) = \dfrac{{13}}{{52}} = \dfrac{1}{4}$
This value of $P\left( G \right)$ should have been obvious directly, since there are only $4$suits in the deck, and any suit is equally likely to turn up.
Going further, let $H$ be the event that a black card is drawn. We have ($\#$represents ‘Number of’),
 $\# {\rm{Black cards}}\,\,\,{\bf{ = }}\,\,\,\,\,\# \,\,{\rm{Spades}} \,\,\,\, {\rm{ + }}\,\,\,\,\,{\rm{\# }}\,\,{\rm{Clubs}}$ $\Rightarrow \,\,\,\, \dfrac{{\# {\rm{Black cards}}}}{{{\rm{\# }}\,{\rm{All cards}}}} = \,\,\,\dfrac{{\# \,\,{\rm{Spades}}}}{{{\rm{\# }}\,{\rm{All cards}}}} + \,\,\,\dfrac{{\# \,\,\,{\rm{Clubs}}}}{{\,{\rm{\# }}\,{\rm{All cards}}}}$ $\Rightarrow \,\,\,\,\dfrac{{26}}{{52}} = \dfrac{{13}}{{52}} + \dfrac{{13}}{{52}}$
We see that $P\left( H \right) = \dfrac{{26}}{{52}} = \dfrac{1}{2}$, since there are $26$ cases favorable to $H$. Again, the value of $P(H)$ should have been obvious directly, since there are only $2$colors in the deck (red, black), and each color is equally likely to turn up.
From this discussion, you should by now have understood the gist of the concept of probability. We can, taking cue from the preceeding examples, define the probability of an event as the number of outcomes favorable to that event, divided by the total number of possible outcomes (assuming each outcome to be equally likely).
Extending this further, the general idea is that if we have two events $E$ and $F$with no outcomes in common, the first consisting of $x$ outcomes and the second of $y$ outcomes, then the event $G$ which consists of all the outcomes in $E$ and $F$ consists of $x + y$ outcomes, i.e,
 $\# G = \# E + \# F$ (if $E$ and $F$ have no outcomes in common)
An example of this is what we just saw:
 $\#\, {\rm{Black \, cards }}=\,\# \,{\rm{Spades}} + \,\# \,{\rm{Clubs}}$
Thus, if the total number of outcomes is $Z$,
we have,
 $P\left( G \right) = \dfrac{{\# G}}{Z}$ $= \dfrac{{\# E + \# F}}{Z}$ $= \dfrac{x}{z} + \dfrac{y}{z}$ $= P(E) + P(F)$
That the probabilities add when we consider the event $G$ consisting of all outcomes in $E$ and $F$ is justified only if $E$ and $F$ have no outcomes in common, as we’ve been seeing till now.
What about the probability of an event consisting of all outcomes in two events$E$ and $F$ which may have some outcomes in common? For example, let $E$ and $F$ be events defined as follows:
 $E$ : The card drawn is even $F$ : The card drawn is black
The number of outcomes in $E$ is $24$ (why ? ). The number of outcomes in $F$ is $26$. However, if we now consider the event $G$ which consists of all the outcomes in $E$ and $F$, we find that the relation
 $\# G = \# E + \# F$
is not satisfied. Why? Because the events $E$ and $F$ have some outcomes in common. Let us understand this visually.
We let the large rectangle below represent all the $52$ possible outcomes. The events $E$ and $F$ are then subsets of this large rectangle as shown.
Note that $E$ and $F$ have some elements in common, which is technically stated by saying that $E$ and $F$ are not mutually exclusive. (Thus, mutually exclusive events have no outcomes in common).
As stated earlier, the event $G$ is defined to consist of all the outcomes in $E$ and $F$, which (in easier language!) means that $G$ is the event that the card drawn is either even or black (or both). How many elements are there in $G$?
Note that when we write
 $\# G = \# E + \# F$ $(\times\,{\rm{ not\, correct}})$
the outcomes common to $E$ and $F$ (represented by the dark regions in Figure –$3$) are counted twice on the right side. The actual number of outcomes in $G$ will thus be obtained by subtracting from the $RHS$ the number of these common outcomes, and thus we can write
 $\# G = \# E + \# F - \# EF$ $({\rm{Correct}})$
where $\# EF$ represents the number of common outcomes. In this case, $EF$would represent the event that the card drawn is both even and black. There are obviously $12$ such cards, as is evident from Figure – $3$, and thus,
 $\# G\,\, = \,\,\,\,24\,\,\,\, + \,\,\,\,\,26\,\,\,\,\,-\,\,\,\,\,12$ $\,\# E\,\,\,\, + \,\,\,\,\# F\,\,\,\,-\,\,\,\# (EF)$ $=38$
You are urged to verify this result by explicitly counting the number of cards lying within the (total) shaded regions in Figure – $3$.
In passing, it should be mentioned that the event $G$ which has been defined as the event consisting of all the outcomes in $E$ and $F$ is technically termed as the union of the events $E$ and $F$, and this fact is symbolically written as
 ${G\,\, = \,\,E\, \cup \,F}$ Union of events
You can think of the union of two events to be the event which can be said to occur when either of the two given events (or both) occurs.
Analogously, the event $H$ defined to consist of all outcomes common to $E$ and$F$ is technically termed as the intersection of the events $E$ and $F$, and this fact is symbolically written as
 ${H\,\, = \,\,E\,\, \cap \,\,F}$ Intersection of events
You can think of the intersection of two events to be the event which can be said to occur when both the two given events occur simultaneously. Finally, as we’ve seen above, we always have
 ${\# \left( {E\,\, \cup \,\,F} \right)\,\, = \,\,\# E\,\, + \,\,\# F\,\,-\,\,\# \,\left( {E\,\, \cap \,\,F} \right)}$ $\ldots(5)$
If $E$ and $F$ have no outcomes in common, that is, if $E$ and $F$ are mutually exclusive:
 $\left( {\# \,\left( {E\,\, \cap \,\,F} \right)\,\, = \,\,0} \right),$
this relation reduces to
 ${\# \left( {E\,\, \cap \,\,F} \right)\,\, = \,\,\# E\,\, + \,\,\# F}$ $\ldots(6)$
The relations $(5)$ and $(6)$, when written in terms of probability (by dividing both sides with the total number of outcomes), become,
 ${P\left( {E\, \cup \,F} \right)\,\, = \,\,P\left( E \right)\,\, + \,\,P\left( F \right)\,\,-\,\,P\left( {E\,\, \cap \,\,F} \right)}$ General relation $\ldots(7)$ ${P\left( {E\, \cup \,F} \right)\,\, = \,\,P\left( E \right)\,\, + \,\,P\left( F \right)}$ The particular case of $E$and $F$ being mutually exclusive $\ldots(8)$
If the concept of mutually exclusive events is now clear to you, the significance of the relation $(2)$ on Page – $3$ immediately becomes apparent, where we simply summed the six probabilities and wrote the sum as $1$. All the six outcomes are mutually exclusive, and they exhaust all possibilities, so $(2)$ is justified.
Now is a good point to learn some more terminology. In Figure – $5$ above, each of the six possible outcomes is the most elementary outcome possible. For example, if $E$ is the event of a $4$ showing up, the event $E$ consists of only one outcome, the number $4$$E$ would therefore be called an elementary event. Now consider the event $F$ of an odd number showing up. $F$ consists of three outcomes, namely $1$$3$ and $5$$F$ would therefore be termed a compound event.
As a further example, in the drawing of a card at random from a standard deck of $52$ cards, let $G$ and $H$ be events defined as follows:
 $G$ : The card drawn is the king of hearts $H$ : The card drawn is a Hearts
$G$ is an elementary event since $G$ consists of only one possible outcome, the king of Hearts (stated differently, $\# G = 1$). However, $H$ is a compound event since $H$ consists of $13$ possible outcomes, namely the $13$ cards of Hearts. In other words, $\# H = 13$

In the experiment of drawing one card at random from a standard well-shuffled deck of $52$ cards, consider the event $E$ defined as:
 $E$ : The card drawn is a king
Obviously, we have
 $P\left( E \right) = \dfrac{{\# {\rm{Kings}}}}{{\# {\rm{All}}\,{\rm{cards}}}}$ $= \dfrac{4}{{52}}$ $= \dfrac{1}{{13}}$
However, if you are now asked to find the probability of event $E$ occurring given that the card drawn has a number greater than $10$, what would you say?
To answer correctly, you must understand that now the situation is entirely different from earlier. Now we already know that the card drawn has a number greater than $10$, which means that the number of possibilities for the card has reduced; earlier, there were $52$ possibilities, but now the number of possible cards are the $4$ Jacks, the $4$ Queens and the $4$ Kings, which means that now there are only $12$ possibilities for the card. Of these, the favorable possibilities are $4$ ; thus,
 $P\left\{ \begin{array}{l} {\rm{Event }}\,E\,{\rm{ given\, that \,the \,card\, drawn }}\\ {\rm{has\, a\, number\, greater\, that \,10\,}} \end{array} \right\} = \dfrac{{\# \,\,{\rm{Kings}}}} {{{\rm{\# \,Reduced\, possibilities}}}}$ $= \dfrac{4}{{12}}$ $= \dfrac{1}{3}$
Let us denote by $F$ the event that the card drawn has a number greater than $10$. The probability just calculated above is then written in standard notation as
 $P\left( {E/F} \right)$
 $P\left( {E\,\,{\rm{given}}\,\,F} \right)$
i.e. “the probability of event $E$ occurring given that $F$ has occurred”.
Let us consider another example. In the random experiment of throwing two dice, let events $G$ and $H$ be defined as
 $G$ : The sum of the two numbers on top is $6$ $H$ : One of the numbers on top is $4$
First, let us find the probability of $G$ occurring. This is simply
 $P\left( G \right) = \dfrac{{\# {\rm{favorable}}\,\,{\rm{possibilities}}}}{{\# \,{\rm{total}}\,\,{\rm{possibilites}}}}$ $= \dfrac{{\left( {1,5} \right)\,\,\left( {2,4} \right)\,\,\left( {3,3} \right)\,\,\left( {4,2} \right)\,\,\left( {5,1} \right)}}{{6 \times 6}}$ $= \dfrac{5}{{36}}$
Now, suppose we are given that $H$ has already occurred, meaning we now know that one of the numbers on top is $4$. What is the probability of $G$ now when we already possess this information?
The number of total possibilities now are $11$; we list them explicitly:
 $\left\{ {\begin{array}{*{20}{c}} {\left( {4,1} \right)}&{\left( {4,2} \right)}&{\left( {4,3} \right)}&{\left( {4,4} \right)}&{\left( {4,5} \right)}&{\left( {4,6} \right)}\\ {\left( {1,4} \right)}&{\left( {2,4} \right)}&{\left( {3,4} \right)}&{}&{\left( {5,4} \right)}&{\left( {6,4} \right)} \end{array}} \right\}$
Out of these, the favorable possibilities are only $2$, namely: $(4, 2)$ and $(2, 4)$. Thus
 $P\left( {G/H} \right) = \dfrac{{\# {\rm{favorable}}\,\,{\rm{possibilities}}}}{{\# {\rm{reduced \,possibilities}}}}$ $=\dfrac{2}{11}$
In both the preceding examples, what happens is that with the possession of some information, the number of possibilities reduce, i.e., the sample space reduces. In the first example, with the information that $F$ has occurred, the number of possibilities reduces from $52$ to $12$. We then looked for favorable cases only within this reduced sample space of $12$ outcomes. Similarly, in the second example, when we possess the information that $H$ has occurred, the number of possibilities reduces from $36$ to $11$; it is in this reduced sample space of outcomes that we then looked for the favorable possibilities.
In passing we should mention that probabilities of the form $P(A/B)$ are called conditional probabilities. $P(A/B)$ is said to be the conditional probability of $A$ given that $B$ has occurred.
Let us understand all this somewhat more deeply. In particular, let us ponder over the following question: Let $E$ and $F$ be two events. Let the probability of $E$ occurring be $P(E)$. Now, if we come to know that $F$ has occurred, will the probability of $E$ occurring always increase, like it did in the last two examples, or can it decrease too? Can it remain the same?
It turns out that all the three are possible, which should even be obvious to an alert reader.
Let us consider the case where the information that $F$ has occurred decreases the probability of $E$ occurring. Let $E$ and $F$ be, in the card drawing experiment of the first example
 $E$ : The card drawn has a number which is a multiple of four $F$ : The card drawn has a number greater than eight
The original probability of $E$ occuring is
 $P\left( E \right) = \dfrac{{\# {\rm{favorable}}\,\,{\rm{possibilities}}}}{{\# \,{\rm{total}}\,\,{\rm{possibilites}}}}$ $= \dfrac{{\left\{ {4,8,12} \right\}{\rm{per\, suit}}\,\, \times \,\,{\rm{4}}\,\,{\rm{suits}}}}{{52}}$ $= \dfrac{{12}}{{52}}\,\, = \,\,\dfrac{3}{{13}}$
But when $F$ has already occurred, the probability of $E$ occurring is
 $P\left( {E/F} \right) = \dfrac{{\# {\rm{favorable}}\,\,{\rm{possibilities}}}}{{\# \,\,{\rm{reduced}}\,\,{\rm{possibilites}}}}$ $= \dfrac{{4\,\,{\rm{Queens}}}}{{\left\{ {9,10,J,Q,K} \right\}\,{\rm{ per \,suit }} \times \,\,\,4\,{\rm{suits}}}}$ $= \dfrac{4}{{{\rm{20}}}} = \dfrac{1}{5}$
which is lesser than the original probability of $E$ occurring. This should be intuitively obvious: In the first case, there are $3$ multiples of four per suit of $13$cards. In the second, when we are told that the number of the card is greater than eight, the multiple of four possible is only $1$ per suit which is the queen. Thus the favorable possibilities decrease to one-third. The total possibilities also decrease i.e, from $13$ to $5$ per suit, but it can be easily appreciated that the percentage reduction in favorable possibilities is greater than the percentage reduction in total possibilities.
We now come to the third very important question: for two events $E$ and $F$, can $P(E/F)$ be the same as $P(E)$? You must appreciate that this is equivalent to saying that the probability of $E$ occurring is not affected by the occurrence or non-occurrence of $F$.
As we said earlier, this is possible, and such events are called independent events:
 If $P(E/F) = P(E)$ $\Rightarrow \,\,\,\,E$ and $F$ are independent events
Let us see an example. In the card-drawing experiment, let events $E$ and $F$ be defined as
 $E$ : The card has a number greater than $8$ $F$ : The card is black
The alert reader will immediately realize that $P(E/F)$ is the same as $P(E)$. Why? Because, the knowledge that the card is black does not change the number of cards per suit that are greater than eight. Stated explicitly,
 $P(E) = \dfrac{{\{ 9,\;10,\;J,\;Q,\;K\} \;{\rm{per}}\;{\rm{suit}}\; \times \;4\;{\rm{suits}}}}{{52}}$ $= \dfrac{{20}}{{52}} = \dfrac{5}{{13}}$ and $P(E/F) = \dfrac{{\{ 9,\;10,\;J,\;Q,\;K\} \;{\rm{per}}\;{\rm{suit}}\; \times 2\;{\rm{suits}}}}{{26}}$ because there are only two black suits $= \dfrac{5}{{13}}$
There is one important point the reader must notice and appreciate:
If $P(E/F) = P(E)$
 $\rightarrow$ this means that $E$ and $F$ are independent events $\rightarrow$ this should also mean that $P(E/F)$ should be the same as $P(F)$
Let us verify this in the example above. We have,
 $P(F) = \dfrac{{\# \;{\rm{black}}\;{\rm{cards}}}}{{\# \;{\rm{total}}\;{\rm{cards}}}}$ $= \dfrac{{26}}{{52}} = \dfrac{1}{2}$
and,
 $P(F/E) = \dfrac{{\# \;{\rm{black}}\;{\rm{cards}}\;{\rm{greater}}\;{\rm{than}}\;{\rm{eight}}}}{{\# \;{\rm{total}}\;{\rm{cards}}\;{\rm{greater}}\;{\rm{than}}\;{\rm{eight}}}}$ $= \dfrac{{\{ 9,\;10,\;J,\;Q,\;K\} \;{\rm{of}}\;{\rm{Spades}}\;{\rm{and}}\;{\rm{of}}\;{\rm{Clubs}}}}{{\{ 9,\;10,\;J,\;Q,\;K\} \;{\rm{per}}\;{\rm{suit}}\; \times \;4\;{\rm{suits}}}}$ $= \dfrac{{10}}{{20}}$ $= \dfrac{1}{2}$
which confirms the assertion
 $\begin{array}{l} \;{\rm{IF}}\;A\;{\rm{and}}\;B\;{\rm{are}}\;{\rm{independent}}\;{\rm{events}}\;\\ \;\;\;\;\;\;\;\;\; \Rightarrow \,\,\,P(A/B) = P(A)\\ \;\;\;\;\;\;\;\;\,{\rm{and}}\;P(B/A) = P(B) \end{array}$ $\ldots(1)$
Note that the favorable cases while calculating $P(A/B)$ are those cases in $A$that are common to $B$; the total cases are all cases in $B$. Thus
 $P(A/B) = \dfrac{{\# \;{\rm{favorable}}\;{\rm{cases}}}}{{\# \;{\rm{total}}\;{\rm{cases}}}}$ $= \dfrac{{\# \;(A \cap B)}}{{\# \;(B)}}$ $\ldots(2)$
If the entire sample space of the experiment consists of $N$ out comes, we can write $(2)$ as
 $P(A/B) = \dfrac{{\# \;\left( {A \cap B} \right)/N}}{{\# \;\left( B \right)/N}}$ $= \dfrac{{P\left( {A \cap B} \right)}}{{P\left( B \right)}}$ $\ldots(3)$
Similarly,
 $P\left( {B/A} \right) = \dfrac{{P\left( {A \cap B} \right)}}{{P\left( A \right)}}$ $\ldots(4)$
This means that if $A$ and $B$ are independent, then
 $P\left( {A/B} \right) = P(A) = \dfrac{{P\left( {A \cap B} \right)}}{{P\left( B \right)}}$ From $(1)$ and $(3)$ ${P\left( {A \cap B} \right) = P(A) \cdot P(B)}$
Thus, the probability of the intersection of two independent events is simply the product of the individual probabilities.