Midterm 2
Thus far, we have discussed set theory and probability distributions. However, we have not talked about how to fit distributions to data, which is an important topic in statistics. For this midterm, you will take the skills you have learned and apply them to estimating distribution parameters.
Read the following article on Maximum Likelihood Estimation (MLE), which can be accessed through UNL’s library.
Myung, In Jae. “Tutorial on Maximum Likelihood Estimation.” Journal of Mathematical Psychology, vol. 47, no. 1, Feb. 2003, pp. 90–100, https://doi.org/10.1016/S0022-2496(02)00028-7.
Complete each of the following problems. To submit this midterm, you must type your solutions using your preferred software (e.g., Word, LaTeX, or Typst) and submit your work as a PDF on Canvas. No late submissions will be accepted. For each problem that requires calculations, you must show your work in order to receive credit. No credit will be received if you do not show your work. You may include scans of handwritten work, but the final solution must be typed. Additionally, you are not allowed to work on the midterm with your classmates or use AI tools.
Questions from reading
Describe the purpose of maximum likelihood estimation. How would we use this in our course? (2-4 sentences)
What is the difference between the pdf, \(f(y|w)\), and the likelihood function, \(L(w|y)\)? (2-4 sentences)
Describe the process of maximum likelihood estimation. What steps do you need to take for the estimation, and what is the goal? (3-5 sentences)
Connecting MLE’s to our course
Recall that the exponential distribution is defined as follows: \[f(x)=\frac{1}{\beta}e^{-x/\beta};\quad x>0,\ \beta>0\] In class, we showed how to derive \(E[X]=\beta\) when \(X\sim \text{Exp}(\beta)\). Derive \(E[X^2]\) and show that \(V(X)=\beta^2\).
Suppose you take a sample of size \(n\) from a population with a known pdf, and each observation is i.i.d. (independently and identically distributed). The joint pdf for the sample is the product of the pdfs for each \(X_i\). That is, for \(X_1,X_2,\dots,X_n\sim \text{Some pdf}\), \[f(x=(x_1,x_2,\dots,x_n))=\prod_{i=1}^{n}f(x_i)=f(x_1)\times f(x_2)\times\dots\times f(x_n)\] Explain how the process of setting up the joint pdf makes sense using properties of set theory and probability discussed in our course. (2-4 sentences)
Suppose that your sample from above came from an exponential distribution. Write the log-likelihood function (using \(\log_e\)) for the observed sample and simplify the expression so that \(\sum_{i=1}^n x_i\) is present in your solution.
The MLE of \(\beta\) is \(\hat{\beta}=\bar{x}=\frac1n\sum_{i=1}^n x_i\), the sample mean. Suppose that you took a sample of 10 sensor readings from the population. Given the following values, what is the estimate of \(\beta\) using the MLE? Additionally, interpret your estimate in the context of the sensor readings.
0.36, 1.67, 0.01, 0.37, 0.19, 7.26, 2.36, 1.69, 1.16, 0.57
Suppose that your classmate is collecting data from this population using a faulty sensor. To calibrate the sensor, they need to use \(Y=1.05 X + 0.2\), where \(X\) was previously defined. What is the expected value and variance of the data collected from the faulty sensor (i.e., \(E(Y)\) and \(V(Y)\))? Use the value for \(\hat{\beta}\) that you computed in problem (7) and the properties derived in problem (4).
Suppose that your sample only took two independent sensor readings from this population, \(X_1\) and \(X_2\). What does this imply about \(\text{Corr}(X_1, X_2)\)? Briefly explain your answer using intuition (1-2 sentences), and show the mathematical process of computing \(\text{Corr}(X_1, X_2)\) by setting up each of the required integrals and calculations. You do not need to evaluate the integrals, but rather show the setup and how the resulting quantities would be used to calculate the correlation. That is, show the process when you start with \(f_{X_1}(x_1)\) and \(f_{X_2}(x_2)\). Recall that \[\text{Corr}(X,Y)=\frac{\text{Cov}(X,Y)}{\sqrt{V(X)V(Y)}}\]
Derive the CDF of the exponential distribution using your value for \(\hat{\beta}\) from problem (7) and use your CDF to calculate the probability of observing a sensor reading that is greater than 1.