Likelihood function:

\[L(\theta) = L(\theta; x_1, x_2, ..., x_n) = \prod_{i=1}^n f(x_i; \theta) = f(x_1; \theta) \cdot ... \cdot f(x_n; \theta)\]

It is often easier to consider

\[\text{ln}L(\theta) = \sum_{i=1}^n \text{ln}f(x_i; \theta)\]

Maximum Likelihood Estimator:

\[\hat{\theta} = \text{arg}\max L(\theta) = \text{arg}\max \text{ln} L(\theta)\]

Method of Moments:

\[E(X) = h(\theta) \qquad\qquad \text{Set} \qquad\qquad \bar{X} = h(\tilde{\theta}) \qquad\qquad \text{Solve for } \tilde{\theta}\]

Consider a single observation \(X\) of a Binomial random variable with \(n\) trials and probability of success \(p\). That is,

\[P(X = k) = {n\choose k} p^k (1 - p)^{n - k}, \qquad\qquad k = 0, 1, ..., n\]

Obtain the method of moments estimator of \(p, \: \tilde{p}\)

Binomial: \(E(X) = np\)

\(X = n\tilde{p} \qquad\qquad \Rightarrow\qquad\qquad \tilde{p} = \frac{X}{n}\)

Obtain the maximum likelihood estimator of \(p, \: \hat{p}\)

\[L(p) = {n \choose x} p^x (1 - p)^{n - x}\]

\[\text{ln} L(p) = \text{ln} {n \choose x} + x\text{ ln} p + (n - x) \text{ln}(1 - p)\]

\[\frac{d}{dp} \text{ln} L(p)= \frac{x}{p} - \frac{n - x}{1 - p} = \frac{x - xp - np + xp}{p(1-p)} = \frac{x - np}{p(1 - p)}\]