Binomial Example_likelihood&MLE

Example) Mathematical Statistics and Data Analysis 3ED Chapter 8. Q31.

George spins a coin three times and observed no heads. He then gives the coin to Hilary. She spins it until the first head occurs, and ends up spinning it four times total. Let $\theta $ denote the probability the coin comes up heads.

a) What is the likelihood of $\theta $?

b) What is the MLE of $\theta $  

$\triangleright$ Think First!! 

This is a binomial distribution example! $P(X=k)=\left ( \frac{n}{k} \right )p^k(1-p)^{n-k}$

PDF of binomial is $f(x|\theta)=\theta^x(1-\theta)^{1-x}$  

$\triangleright$ Solution (a)  

They spin a coin 7 times in total. (George: 3 times, Hilary 4 times) 

Let X be 1 (if the result is head), o (otherwise) 

George case (X): The likelihood is $\theta $, $L_{1}(\theta | X_{1},...,X_{n})= \prod_{i=1}^{n}f(X_{i}|\theta)=\prod \theta^{x_{i}}(1-\theta)^{n-x_{i}}$ 

                       George spins a coin three times, here n is 3. 

                       There is no head, $X_{1}=X_{2}=X_{3}=0$  

Hilary case (Y): PDF of Y: $g(Y|\theta)=\theta (1-\theta)^{y-1}$

                        The likelihood of Y: $L_{2}=(\theta|y)=\theta(1-\theta)^{y-1}$ 

                        Here Y is the number of toss required. So Y is 4.

                        (why $\theta^1$ ? b/c there is one head)    

$\rightarrow$ the likelihood of both George and Hilary $\theta $ is...

$\therefore$ $ L(\theta|X_{1},...,X_{n}, Y)= \left [ \prod_{i=1}^{n}\theta^{X_{i}}(1-\theta)^{n-x_i} \right ]\theta (1-\theta)^{y-1}$  $=\theta^{1+\sum x_{i}}(1-\theta)^{n-\sum x_{i}+y-1}$ 

$\triangleright$ Solution (b)  

$\Rightarrow$ Now log likelihood is $= \left [1+\sum _{i=1}^{n} x_{i} \right] \ln(\theta)+ \left [ n - \sum _{i=1}^{n} x_{i}+y -1 \right ] \ln (1-\theta)$

$\Rightarrow$ Take a derivative with respect to $\theta $ is $\frac{d \ln(L)}{d \theta}=\frac{1+\sum x_{i}}{\theta}+\frac{n-\sum x_{i} +y -1}{\theta - 1}$ = (set) 0

    = $\frac{1+\sum x_{i}}{\theta}=\frac{n-\sum x_{i} +y -1}{\theta - 1} \rightarrow (n+y)\theta =1+\sum x_{i} \rightarrow \theta = \frac{1+\sum_{i=1}^{n}x_{i}}{n+y}$ 

     $\star \frac{d}{dx}(\log (1-x))=\frac{1}{x-1}$ 

Now n=3, $X_{1}=X_{2}=X_{3}=0, Y=4$, 

$\therefore \theta_{MLE}= \frac{1+0+0+0}{3+4}=\frac {1}{7}$