Mixing
Estimating the MLE where the parameter is also the constraint
Clash Royale CLAN TAG#URR8PPP
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty margin-bottom:0;
up vote
3
down vote
favorite
Independent random variables $X_1,X_2,ldots,X_n sim f_X$ are modeled with a common density
$$f_X(x) = fracalpha(x/beta)^alpha-1beta quad quad quad textfor all 0 le x le beta.$$
Then I've calculated the log-likelihood function as
$$ l(alpha,beta)=nlog(alpha)-nalphalog(beta)+(alpha-1)sum_i=1^nlog(x_i).$$
And I've found an estimate for $alpha$ by taking the derivative
$$hatalpha=fracnlog(beta)-sum_i=1^nlog(x_i),$$
for $log(beta)nesum_i=1^nlog(x_i)$. But how can I find the MLE of $beta$, when it is also a constraint on where the function is defined?
probability estimation maximum-likelihood
edited 2 hours ago
Ben
17.2k22286
asked 3 hours ago
Nic Nic
183
New contributor
Nic Nic is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
add a comment |Â
up vote
3
down vote
favorite
Independent random variables $X_1,X_2,ldots,X_n sim f_X$ are modeled with a common density
$$f_X(x) = fracalpha(x/beta)^alpha-1beta quad quad quad textfor all 0 le x le beta.$$
Then I've calculated the log-likelihood function as
$$ l(alpha,beta)=nlog(alpha)-nalphalog(beta)+(alpha-1)sum_i=1^nlog(x_i).$$
And I've found an estimate for $alpha$ by taking the derivative
$$hatalpha=fracnlog(beta)-sum_i=1^nlog(x_i),$$
for $log(beta)nesum_i=1^nlog(x_i)$. But how can I find the MLE of $beta$, when it is also a constraint on where the function is defined?
probability estimation maximum-likelihood
edited 2 hours ago
Ben
17.2k22286
asked 3 hours ago
Nic Nic
183
New contributor
Nic Nic is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
I just realised that since the score function is decreasing in $beta$ then it should be minimized which is gotten through order statistics, so $$hatbeta=X^(n) $$
– Nic Nic
3 hours ago
Slight mistake in the pdf; check the differentiation again.
– StubbornAtom
3 hours ago
Is this is part of an assignment or homework, consider adding theself-studytag. Also read the tag wiki.
– StubbornAtom
3 hours ago
1
Forgot to add the divide by $beta$ in the pdf. It was used in calculating the score-function. It's just practice, so wasn't sure what it classified as.
– Nic Nic
3 hours ago
No problem. What you call the score function is actually the logarithm of the likelihood. Score function comes into play after you differentiate the log-likelihood.
– StubbornAtom
3 hours ago
add a comment |Â
up vote
3
down vote
favorite
up vote
3
down vote
favorite
Independent random variables $X_1,X_2,ldots,X_n sim f_X$ are modeled with a common density
$$f_X(x) = fracalpha(x/beta)^alpha-1beta quad quad quad textfor all 0 le x le beta.$$
Then I've calculated the log-likelihood function as
$$ l(alpha,beta)=nlog(alpha)-nalphalog(beta)+(alpha-1)sum_i=1^nlog(x_i).$$
And I've found an estimate for $alpha$ by taking the derivative
$$hatalpha=fracnlog(beta)-sum_i=1^nlog(x_i),$$
for $log(beta)nesum_i=1^nlog(x_i)$. But how can I find the MLE of $beta$, when it is also a constraint on where the function is defined?
probability estimation maximum-likelihood
edited 2 hours ago
Ben
17.2k22286
asked 3 hours ago
Nic Nic
183
New contributor
Nic Nic is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
Independent random variables $X_1,X_2,ldots,X_n sim f_X$ are modeled with a common density
$$f_X(x) = fracalpha(x/beta)^alpha-1beta quad quad quad textfor all 0 le x le beta.$$
Then I've calculated the log-likelihood function as
$$ l(alpha,beta)=nlog(alpha)-nalphalog(beta)+(alpha-1)sum_i=1^nlog(x_i).$$
And I've found an estimate for $alpha$ by taking the derivative
$$hatalpha=fracnlog(beta)-sum_i=1^nlog(x_i),$$
for $log(beta)nesum_i=1^nlog(x_i)$. But how can I find the MLE of $beta$, when it is also a constraint on where the function is defined?
probability estimation maximum-likelihood
probability estimation maximum-likelihood
edited 2 hours ago
Ben
17.2k22286
asked 3 hours ago
Nic Nic
183
New contributor
Nic Nic is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
edited 2 hours ago
Ben
17.2k22286
asked 3 hours ago
Nic Nic
183
New contributor
Nic Nic is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
edited 2 hours ago
Ben
17.2k22286
edited 2 hours ago
Ben
17.2k22286
edited 2 hours ago
Ben
17.2k22286
17.2k22286
asked 3 hours ago
Nic Nic
183
New contributor
Nic Nic is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
asked 3 hours ago
Nic Nic
183
asked 3 hours ago
Nic Nic
183
183
New contributor
Nic Nic is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
New contributor
Nic Nic is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
Nic Nic is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.
I just realised that since the score function is decreasing in $beta$ then it should be minimized which is gotten through order statistics, so $$hatbeta=X^(n) $$
– Nic Nic
3 hours ago
Slight mistake in the pdf; check the differentiation again.
– StubbornAtom
3 hours ago
Is this is part of an assignment or homework, consider adding theself-studytag. Also read the tag wiki.
– StubbornAtom
3 hours ago
1
Forgot to add the divide by $beta$ in the pdf. It was used in calculating the score-function. It's just practice, so wasn't sure what it classified as.
– Nic Nic
3 hours ago
No problem. What you call the score function is actually the logarithm of the likelihood. Score function comes into play after you differentiate the log-likelihood.
– StubbornAtom
3 hours ago
add a comment |Â
I just realised that since the score function is decreasing in $beta$ then it should be minimized which is gotten through order statistics, so $$hatbeta=X^(n) $$
– Nic Nic
3 hours ago
Slight mistake in the pdf; check the differentiation again.
– StubbornAtom
3 hours ago
Is this is part of an assignment or homework, consider adding theself-studytag. Also read the tag wiki.
– StubbornAtom
3 hours ago
1
Forgot to add the divide by $beta$ in the pdf. It was used in calculating the score-function. It's just practice, so wasn't sure what it classified as.
– Nic Nic
3 hours ago
No problem. What you call the score function is actually the logarithm of the likelihood. Score function comes into play after you differentiate the log-likelihood.
– StubbornAtom
3 hours ago
I just realised that since the score function is decreasing in $beta$ then it should be minimized which is gotten through order statistics, so $$hatbeta=X^(n) $$
– Nic Nic
3 hours ago
I just realised that since the score function is decreasing in $beta$ then it should be minimized which is gotten through order statistics, so $$hatbeta=X^(n) $$
– Nic Nic
3 hours ago
Slight mistake in the pdf; check the differentiation again.
– StubbornAtom
3 hours ago
Slight mistake in the pdf; check the differentiation again.
– StubbornAtom
3 hours ago
Is this is part of an assignment or homework, consider adding the
self-study tag. Also read the tag wiki.– StubbornAtom
3 hours ago
Is this is part of an assignment or homework, consider adding the
self-study tag. Also read the tag wiki.– StubbornAtom
3 hours ago
1
1
Forgot to add the divide by $beta$ in the pdf. It was used in calculating the score-function. It's just practice, so wasn't sure what it classified as.
– Nic Nic
3 hours ago
Forgot to add the divide by $beta$ in the pdf. It was used in calculating the score-function. It's just practice, so wasn't sure what it classified as.
– Nic Nic
3 hours ago
No problem. What you call the score function is actually the logarithm of the likelihood. Score function comes into play after you differentiate the log-likelihood.
– StubbornAtom
3 hours ago
No problem. What you call the score function is actually the logarithm of the likelihood. Score function comes into play after you differentiate the log-likelihood.
– StubbornAtom
3 hours ago
add a comment |Â
2 Answers
2
active
oldest
votes
up vote
2
down vote
accepted
Your density function is:
$$p_X(x|alpha,beta) = fracalphabeta Big( fracxbeta Big)^alpha-1 quad quad quad textfor 0 leqslant x leqslant beta.$$
Hence, your log-likelihood function is:
$$ell_mathbfx(alpha, beta) = n ln alpha - n alpha ln beta + (alpha-1) sum_i=1^n ln x_i quad quad quad textfor 0 leqslant x_(1) leqslant x_(n) leqslant beta.$$
The score function and Hessian matrix are given respectively by:
$$beginequation beginaligned
nabla ell_mathbfx(alpha, beta)
&= beginbmatrix
n/alpha - n ln beta + sum_i=1^n ln x_i \[6pt]
n alpha/beta \[6pt]
endbmatrix, \[10pt]
nabla^2 ell_mathbfx(alpha, beta)
&= beginbmatrix
-n/alpha^2 & n/beta \[6pt]
n/beta & - n alpha/beta^2 \[6pt]
endbmatrix.
endaligned endequation$$
The function is strictly increasing with respect to $beta$ and the function is concave (i.e., the Hessian matrix is negative definite). This means that the MLE of $beta$ occurs at the boundary point, and the MLE of $alpha$ occurs at the unique critical point. We have the estimators:
$$hatalpha = fracnsum_i=1^n (ln x_(n) - ln x_i) quad quad quad hatbeta = x_(n).$$
As you can see, when your parameter enters the density as a bound on the range of $x$ you can get a situation where the MLE occurs at the boundary of the log-likelihood. This is all just standard use of calculus techniques --- sometimes maximising values of an objective function occur at critical points and sometimes they occur at boundary points.
answered 2 hours ago
Ben
17.2k22286
Sorry, did you use the second partial derivative test here by showing that the Hessian is negative definite?
– StubbornAtom
2 hours ago
Since the MLE for $beta$ occurs at a boundary point, you only need to check SOC for the MLE of $alpha$. This confirms that there is a unique critical point value that is a local maximum.
– Ben
2 hours ago
I was under the impression that the second partial derivative test fails here since the Hessian is not totally differentiable (because $fracpartial ell_mathbf xpartial beta$ does not exist at $beta=x_(n)$). Like when support depends on parameter, we cannot use derivatives to derive MLE in single parameter problems.
– StubbornAtom
2 hours ago
You can still check SOC by looking at the "profile log-likelihood" that results from substituting $beta = x_(n)$.
– Ben
2 hours ago
And what is SOC?
– StubbornAtom
2 hours ago
 |Â
show 1 more comment
up vote
2
down vote
Looks like both $alpha$ and $beta$ are unknown here. So our parameter is $theta=(alpha,beta)$.
The population pdf is $$f_theta(x)=fracalphabeta^alphax^alpha-1mathbf1_0<x<betaquad,,alpha>0$$
So, given the sample $(x_1,x_2,ldots,x_n)$, likelihood function of $theta$ is
beginalign
L(theta)&=prod_i=1^n f_theta(x_i)
\&=left(fracalphabeta^alpharight)^nleft(prod_i=1^n x_iright)^alpha-1mathbf1_0<x_1,x_2,ldots,x_n<beta
\&=left(fracalphabeta^alpharight)^nleft(prod_i=1^n x_iright)^alpha-1mathbf1_0<x_(n)<betaquad,,alpha>0
endalign
, where $x_(n)=max_1le ile n x_i$ is the largest order statistic.
The log-likelihood is therefore $$ell(theta)=n(lnalpha-alphalnbeta)+(alpha-1)sum_i=1^n ln x_i+ln(mathbf1_0<x_(n)<beta)$$
Observe that, given the sample, the parameter space has become $$Theta=theta:alpha>0,beta>x_(n)$$
Keeping $alpha$ fixed, justify that $ell(theta)$ is maximized for the minimum value of $beta$ subject to the constraint $betain(x_(n),infty)$. In other words, as you say, $ell(theta)$ is a decreasing function of $beta$ for fixed $alpha$. Hence conclude that MLE of $beta$ as you guessed is $$hatbeta=X_(n)$$
It is now valid to derive the MLE of $alpha$ by differentiating the log-likelihood as you have done. This MLE is likely to depend on the MLE of $beta$.
Indeed,
beginalign
fracpartialellpartialalpha&=fracnalpha-nlnbeta+sum_i=1^n ln x_i
endalign
, which vanishes if and only if $$alpha=fracnnlnbeta-sum_i=1^nln x_i$$
(Since $x_i<betaimplies ln x_i<lnbetaimplies sum ln x_i<nlnbeta$, the above expression is defined.)
So our possible candidate for MLE of $alpha$ is $$hatalpha=fracnnlnhatbeta-sum_i=1^nln x_i$$
At this point, you can finish your argument saying that MLE of $theta=(alpha,beta)$ is $hattheta=(hatalpha,hatbeta)$.
But since this is a maximization problem in two variables $(alpha,beta)$, you could perhaps verify that $$ell(hattheta)ge ell (theta)$$ holds for every $theta$. This would be a bit more rigorous I think.
answered 2 hours ago
StubbornAtom
1,6671325
add a comment |Â
2 Answers
2
active
oldest
votes
2 Answers
2
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
2
down vote
accepted
Your density function is:
$$p_X(x|alpha,beta) = fracalphabeta Big( fracxbeta Big)^alpha-1 quad quad quad textfor 0 leqslant x leqslant beta.$$
Hence, your log-likelihood function is:
$$ell_mathbfx(alpha, beta) = n ln alpha - n alpha ln beta + (alpha-1) sum_i=1^n ln x_i quad quad quad textfor 0 leqslant x_(1) leqslant x_(n) leqslant beta.$$
The score function and Hessian matrix are given respectively by:
$$beginequation beginaligned
nabla ell_mathbfx(alpha, beta)
&= beginbmatrix
n/alpha - n ln beta + sum_i=1^n ln x_i \[6pt]
n alpha/beta \[6pt]
endbmatrix, \[10pt]
nabla^2 ell_mathbfx(alpha, beta)
&= beginbmatrix
-n/alpha^2 & n/beta \[6pt]
n/beta & - n alpha/beta^2 \[6pt]
endbmatrix.
endaligned endequation$$
The function is strictly increasing with respect to $beta$ and the function is concave (i.e., the Hessian matrix is negative definite). This means that the MLE of $beta$ occurs at the boundary point, and the MLE of $alpha$ occurs at the unique critical point. We have the estimators:
$$hatalpha = fracnsum_i=1^n (ln x_(n) - ln x_i) quad quad quad hatbeta = x_(n).$$
As you can see, when your parameter enters the density as a bound on the range of $x$ you can get a situation where the MLE occurs at the boundary of the log-likelihood. This is all just standard use of calculus techniques --- sometimes maximising values of an objective function occur at critical points and sometimes they occur at boundary points.
answered 2 hours ago
Ben
17.2k22286
Sorry, did you use the second partial derivative test here by showing that the Hessian is negative definite?
– StubbornAtom
2 hours ago
Since the MLE for $beta$ occurs at a boundary point, you only need to check SOC for the MLE of $alpha$. This confirms that there is a unique critical point value that is a local maximum.
– Ben
2 hours ago
I was under the impression that the second partial derivative test fails here since the Hessian is not totally differentiable (because $fracpartial ell_mathbf xpartial beta$ does not exist at $beta=x_(n)$). Like when support depends on parameter, we cannot use derivatives to derive MLE in single parameter problems.
– StubbornAtom
2 hours ago
You can still check SOC by looking at the "profile log-likelihood" that results from substituting $beta = x_(n)$.
– Ben
2 hours ago
And what is SOC?
– StubbornAtom
2 hours ago
 |Â
show 1 more comment
up vote
2
down vote
accepted
Your density function is:
$$p_X(x|alpha,beta) = fracalphabeta Big( fracxbeta Big)^alpha-1 quad quad quad textfor 0 leqslant x leqslant beta.$$
Hence, your log-likelihood function is:
$$ell_mathbfx(alpha, beta) = n ln alpha - n alpha ln beta + (alpha-1) sum_i=1^n ln x_i quad quad quad textfor 0 leqslant x_(1) leqslant x_(n) leqslant beta.$$
The score function and Hessian matrix are given respectively by:
$$beginequation beginaligned
nabla ell_mathbfx(alpha, beta)
&= beginbmatrix
n/alpha - n ln beta + sum_i=1^n ln x_i \[6pt]
n alpha/beta \[6pt]
endbmatrix, \[10pt]
nabla^2 ell_mathbfx(alpha, beta)
&= beginbmatrix
-n/alpha^2 & n/beta \[6pt]
n/beta & - n alpha/beta^2 \[6pt]
endbmatrix.
endaligned endequation$$
The function is strictly increasing with respect to $beta$ and the function is concave (i.e., the Hessian matrix is negative definite). This means that the MLE of $beta$ occurs at the boundary point, and the MLE of $alpha$ occurs at the unique critical point. We have the estimators:
$$hatalpha = fracnsum_i=1^n (ln x_(n) - ln x_i) quad quad quad hatbeta = x_(n).$$
As you can see, when your parameter enters the density as a bound on the range of $x$ you can get a situation where the MLE occurs at the boundary of the log-likelihood. This is all just standard use of calculus techniques --- sometimes maximising values of an objective function occur at critical points and sometimes they occur at boundary points.
answered 2 hours ago
Ben
17.2k22286
Sorry, did you use the second partial derivative test here by showing that the Hessian is negative definite?
– StubbornAtom
2 hours ago
Since the MLE for $beta$ occurs at a boundary point, you only need to check SOC for the MLE of $alpha$. This confirms that there is a unique critical point value that is a local maximum.
– Ben
2 hours ago
I was under the impression that the second partial derivative test fails here since the Hessian is not totally differentiable (because $fracpartial ell_mathbf xpartial beta$ does not exist at $beta=x_(n)$). Like when support depends on parameter, we cannot use derivatives to derive MLE in single parameter problems.
– StubbornAtom
2 hours ago
You can still check SOC by looking at the "profile log-likelihood" that results from substituting $beta = x_(n)$.
– Ben
2 hours ago
And what is SOC?
– StubbornAtom
2 hours ago
 |Â
show 1 more comment
up vote
2
down vote
accepted
up vote
2
down vote
accepted
Your density function is:
$$p_X(x|alpha,beta) = fracalphabeta Big( fracxbeta Big)^alpha-1 quad quad quad textfor 0 leqslant x leqslant beta.$$
Hence, your log-likelihood function is:
$$ell_mathbfx(alpha, beta) = n ln alpha - n alpha ln beta + (alpha-1) sum_i=1^n ln x_i quad quad quad textfor 0 leqslant x_(1) leqslant x_(n) leqslant beta.$$
The score function and Hessian matrix are given respectively by:
$$beginequation beginaligned
nabla ell_mathbfx(alpha, beta)
&= beginbmatrix
n/alpha - n ln beta + sum_i=1^n ln x_i \[6pt]
n alpha/beta \[6pt]
endbmatrix, \[10pt]
nabla^2 ell_mathbfx(alpha, beta)
&= beginbmatrix
-n/alpha^2 & n/beta \[6pt]
n/beta & - n alpha/beta^2 \[6pt]
endbmatrix.
endaligned endequation$$
The function is strictly increasing with respect to $beta$ and the function is concave (i.e., the Hessian matrix is negative definite). This means that the MLE of $beta$ occurs at the boundary point, and the MLE of $alpha$ occurs at the unique critical point. We have the estimators:
$$hatalpha = fracnsum_i=1^n (ln x_(n) - ln x_i) quad quad quad hatbeta = x_(n).$$
As you can see, when your parameter enters the density as a bound on the range of $x$ you can get a situation where the MLE occurs at the boundary of the log-likelihood. This is all just standard use of calculus techniques --- sometimes maximising values of an objective function occur at critical points and sometimes they occur at boundary points.
answered 2 hours ago
Ben
17.2k22286
Your density function is:
$$p_X(x|alpha,beta) = fracalphabeta Big( fracxbeta Big)^alpha-1 quad quad quad textfor 0 leqslant x leqslant beta.$$
Hence, your log-likelihood function is:
$$ell_mathbfx(alpha, beta) = n ln alpha - n alpha ln beta + (alpha-1) sum_i=1^n ln x_i quad quad quad textfor 0 leqslant x_(1) leqslant x_(n) leqslant beta.$$
The score function and Hessian matrix are given respectively by:
$$beginequation beginaligned
nabla ell_mathbfx(alpha, beta)
&= beginbmatrix
n/alpha - n ln beta + sum_i=1^n ln x_i \[6pt]
n alpha/beta \[6pt]
endbmatrix, \[10pt]
nabla^2 ell_mathbfx(alpha, beta)
&= beginbmatrix
-n/alpha^2 & n/beta \[6pt]
n/beta & - n alpha/beta^2 \[6pt]
endbmatrix.
endaligned endequation$$
The function is strictly increasing with respect to $beta$ and the function is concave (i.e., the Hessian matrix is negative definite). This means that the MLE of $beta$ occurs at the boundary point, and the MLE of $alpha$ occurs at the unique critical point. We have the estimators:
$$hatalpha = fracnsum_i=1^n (ln x_(n) - ln x_i) quad quad quad hatbeta = x_(n).$$
As you can see, when your parameter enters the density as a bound on the range of $x$ you can get a situation where the MLE occurs at the boundary of the log-likelihood. This is all just standard use of calculus techniques --- sometimes maximising values of an objective function occur at critical points and sometimes they occur at boundary points.
answered 2 hours ago
Ben
17.2k22286
edited 2 hours ago
answered 2 hours ago
Ben
17.2k22286
answered 2 hours ago
Ben
17.2k22286
answered 2 hours ago
Ben
17.2k22286
17.2k22286
Sorry, did you use the second partial derivative test here by showing that the Hessian is negative definite?
– StubbornAtom
2 hours ago
Since the MLE for $beta$ occurs at a boundary point, you only need to check SOC for the MLE of $alpha$. This confirms that there is a unique critical point value that is a local maximum.
– Ben
2 hours ago
I was under the impression that the second partial derivative test fails here since the Hessian is not totally differentiable (because $fracpartial ell_mathbf xpartial beta$ does not exist at $beta=x_(n)$). Like when support depends on parameter, we cannot use derivatives to derive MLE in single parameter problems.
– StubbornAtom
2 hours ago
You can still check SOC by looking at the "profile log-likelihood" that results from substituting $beta = x_(n)$.
– Ben
2 hours ago
And what is SOC?
– StubbornAtom
2 hours ago
 |Â
show 1 more comment
Sorry, did you use the second partial derivative test here by showing that the Hessian is negative definite?
– StubbornAtom
2 hours ago
Since the MLE for $beta$ occurs at a boundary point, you only need to check SOC for the MLE of $alpha$. This confirms that there is a unique critical point value that is a local maximum.
– Ben
2 hours ago
I was under the impression that the second partial derivative test fails here since the Hessian is not totally differentiable (because $fracpartial ell_mathbf xpartial beta$ does not exist at $beta=x_(n)$). Like when support depends on parameter, we cannot use derivatives to derive MLE in single parameter problems.
– StubbornAtom
2 hours ago
You can still check SOC by looking at the "profile log-likelihood" that results from substituting $beta = x_(n)$.
– Ben
2 hours ago
And what is SOC?
– StubbornAtom
2 hours ago
Sorry, did you use the second partial derivative test here by showing that the Hessian is negative definite?
– StubbornAtom
2 hours ago
Sorry, did you use the second partial derivative test here by showing that the Hessian is negative definite?
– StubbornAtom
2 hours ago
Since the MLE for $beta$ occurs at a boundary point, you only need to check SOC for the MLE of $alpha$. This confirms that there is a unique critical point value that is a local maximum.
– Ben
2 hours ago
Since the MLE for $beta$ occurs at a boundary point, you only need to check SOC for the MLE of $alpha$. This confirms that there is a unique critical point value that is a local maximum.
– Ben
2 hours ago
I was under the impression that the second partial derivative test fails here since the Hessian is not totally differentiable (because $fracpartial ell_mathbf xpartial beta$ does not exist at $beta=x_(n)$). Like when support depends on parameter, we cannot use derivatives to derive MLE in single parameter problems.
– StubbornAtom
2 hours ago
I was under the impression that the second partial derivative test fails here since the Hessian is not totally differentiable (because $fracpartial ell_mathbf xpartial beta$ does not exist at $beta=x_(n)$). Like when support depends on parameter, we cannot use derivatives to derive MLE in single parameter problems.
– StubbornAtom
2 hours ago
You can still check SOC by looking at the "profile log-likelihood" that results from substituting $beta = x_(n)$.
– Ben
2 hours ago
You can still check SOC by looking at the "profile log-likelihood" that results from substituting $beta = x_(n)$.
– Ben
2 hours ago
And what is SOC?
– StubbornAtom
2 hours ago
And what is SOC?
– StubbornAtom
2 hours ago
 |Â
show 1 more comment
up vote
2
down vote
Looks like both $alpha$ and $beta$ are unknown here. So our parameter is $theta=(alpha,beta)$.
The population pdf is $$f_theta(x)=fracalphabeta^alphax^alpha-1mathbf1_0<x<betaquad,,alpha>0$$
So, given the sample $(x_1,x_2,ldots,x_n)$, likelihood function of $theta$ is
beginalign
L(theta)&=prod_i=1^n f_theta(x_i)
\&=left(fracalphabeta^alpharight)^nleft(prod_i=1^n x_iright)^alpha-1mathbf1_0<x_1,x_2,ldots,x_n<beta
\&=left(fracalphabeta^alpharight)^nleft(prod_i=1^n x_iright)^alpha-1mathbf1_0<x_(n)<betaquad,,alpha>0
endalign
, where $x_(n)=max_1le ile n x_i$ is the largest order statistic.
The log-likelihood is therefore $$ell(theta)=n(lnalpha-alphalnbeta)+(alpha-1)sum_i=1^n ln x_i+ln(mathbf1_0<x_(n)<beta)$$
Observe that, given the sample, the parameter space has become $$Theta=theta:alpha>0,beta>x_(n)$$
Keeping $alpha$ fixed, justify that $ell(theta)$ is maximized for the minimum value of $beta$ subject to the constraint $betain(x_(n),infty)$. In other words, as you say, $ell(theta)$ is a decreasing function of $beta$ for fixed $alpha$. Hence conclude that MLE of $beta$ as you guessed is $$hatbeta=X_(n)$$
It is now valid to derive the MLE of $alpha$ by differentiating the log-likelihood as you have done. This MLE is likely to depend on the MLE of $beta$.
Indeed,
beginalign
fracpartialellpartialalpha&=fracnalpha-nlnbeta+sum_i=1^n ln x_i
endalign
, which vanishes if and only if $$alpha=fracnnlnbeta-sum_i=1^nln x_i$$
(Since $x_i<betaimplies ln x_i<lnbetaimplies sum ln x_i<nlnbeta$, the above expression is defined.)
So our possible candidate for MLE of $alpha$ is $$hatalpha=fracnnlnhatbeta-sum_i=1^nln x_i$$
At this point, you can finish your argument saying that MLE of $theta=(alpha,beta)$ is $hattheta=(hatalpha,hatbeta)$.
But since this is a maximization problem in two variables $(alpha,beta)$, you could perhaps verify that $$ell(hattheta)ge ell (theta)$$ holds for every $theta$. This would be a bit more rigorous I think.
answered 2 hours ago
StubbornAtom
1,6671325
add a comment |Â
up vote
2
down vote
Looks like both $alpha$ and $beta$ are unknown here. So our parameter is $theta=(alpha,beta)$.
The population pdf is $$f_theta(x)=fracalphabeta^alphax^alpha-1mathbf1_0<x<betaquad,,alpha>0$$
So, given the sample $(x_1,x_2,ldots,x_n)$, likelihood function of $theta$ is
beginalign
L(theta)&=prod_i=1^n f_theta(x_i)
\&=left(fracalphabeta^alpharight)^nleft(prod_i=1^n x_iright)^alpha-1mathbf1_0<x_1,x_2,ldots,x_n<beta
\&=left(fracalphabeta^alpharight)^nleft(prod_i=1^n x_iright)^alpha-1mathbf1_0<x_(n)<betaquad,,alpha>0
endalign
, where $x_(n)=max_1le ile n x_i$ is the largest order statistic.
The log-likelihood is therefore $$ell(theta)=n(lnalpha-alphalnbeta)+(alpha-1)sum_i=1^n ln x_i+ln(mathbf1_0<x_(n)<beta)$$
Observe that, given the sample, the parameter space has become $$Theta=theta:alpha>0,beta>x_(n)$$
Keeping $alpha$ fixed, justify that $ell(theta)$ is maximized for the minimum value of $beta$ subject to the constraint $betain(x_(n),infty)$. In other words, as you say, $ell(theta)$ is a decreasing function of $beta$ for fixed $alpha$. Hence conclude that MLE of $beta$ as you guessed is $$hatbeta=X_(n)$$
It is now valid to derive the MLE of $alpha$ by differentiating the log-likelihood as you have done. This MLE is likely to depend on the MLE of $beta$.
Indeed,
beginalign
fracpartialellpartialalpha&=fracnalpha-nlnbeta+sum_i=1^n ln x_i
endalign
, which vanishes if and only if $$alpha=fracnnlnbeta-sum_i=1^nln x_i$$
(Since $x_i<betaimplies ln x_i<lnbetaimplies sum ln x_i<nlnbeta$, the above expression is defined.)
So our possible candidate for MLE of $alpha$ is $$hatalpha=fracnnlnhatbeta-sum_i=1^nln x_i$$
At this point, you can finish your argument saying that MLE of $theta=(alpha,beta)$ is $hattheta=(hatalpha,hatbeta)$.
But since this is a maximization problem in two variables $(alpha,beta)$, you could perhaps verify that $$ell(hattheta)ge ell (theta)$$ holds for every $theta$. This would be a bit more rigorous I think.
answered 2 hours ago
StubbornAtom
1,6671325
add a comment |Â
up vote
2
down vote
up vote
2
down vote
Looks like both $alpha$ and $beta$ are unknown here. So our parameter is $theta=(alpha,beta)$.
The population pdf is $$f_theta(x)=fracalphabeta^alphax^alpha-1mathbf1_0<x<betaquad,,alpha>0$$
So, given the sample $(x_1,x_2,ldots,x_n)$, likelihood function of $theta$ is
beginalign
L(theta)&=prod_i=1^n f_theta(x_i)
\&=left(fracalphabeta^alpharight)^nleft(prod_i=1^n x_iright)^alpha-1mathbf1_0<x_1,x_2,ldots,x_n<beta
\&=left(fracalphabeta^alpharight)^nleft(prod_i=1^n x_iright)^alpha-1mathbf1_0<x_(n)<betaquad,,alpha>0
endalign
, where $x_(n)=max_1le ile n x_i$ is the largest order statistic.
The log-likelihood is therefore $$ell(theta)=n(lnalpha-alphalnbeta)+(alpha-1)sum_i=1^n ln x_i+ln(mathbf1_0<x_(n)<beta)$$
Observe that, given the sample, the parameter space has become $$Theta=theta:alpha>0,beta>x_(n)$$
Keeping $alpha$ fixed, justify that $ell(theta)$ is maximized for the minimum value of $beta$ subject to the constraint $betain(x_(n),infty)$. In other words, as you say, $ell(theta)$ is a decreasing function of $beta$ for fixed $alpha$. Hence conclude that MLE of $beta$ as you guessed is $$hatbeta=X_(n)$$
It is now valid to derive the MLE of $alpha$ by differentiating the log-likelihood as you have done. This MLE is likely to depend on the MLE of $beta$.
Indeed,
beginalign
fracpartialellpartialalpha&=fracnalpha-nlnbeta+sum_i=1^n ln x_i
endalign
, which vanishes if and only if $$alpha=fracnnlnbeta-sum_i=1^nln x_i$$
(Since $x_i<betaimplies ln x_i<lnbetaimplies sum ln x_i<nlnbeta$, the above expression is defined.)
So our possible candidate for MLE of $alpha$ is $$hatalpha=fracnnlnhatbeta-sum_i=1^nln x_i$$
At this point, you can finish your argument saying that MLE of $theta=(alpha,beta)$ is $hattheta=(hatalpha,hatbeta)$.
But since this is a maximization problem in two variables $(alpha,beta)$, you could perhaps verify that $$ell(hattheta)ge ell (theta)$$ holds for every $theta$. This would be a bit more rigorous I think.
answered 2 hours ago
StubbornAtom
1,6671325
Looks like both $alpha$ and $beta$ are unknown here. So our parameter is $theta=(alpha,beta)$.
The population pdf is $$f_theta(x)=fracalphabeta^alphax^alpha-1mathbf1_0<x<betaquad,,alpha>0$$
So, given the sample $(x_1,x_2,ldots,x_n)$, likelihood function of $theta$ is
beginalign
L(theta)&=prod_i=1^n f_theta(x_i)
\&=left(fracalphabeta^alpharight)^nleft(prod_i=1^n x_iright)^alpha-1mathbf1_0<x_1,x_2,ldots,x_n<beta
\&=left(fracalphabeta^alpharight)^nleft(prod_i=1^n x_iright)^alpha-1mathbf1_0<x_(n)<betaquad,,alpha>0
endalign
, where $x_(n)=max_1le ile n x_i$ is the largest order statistic.
The log-likelihood is therefore $$ell(theta)=n(lnalpha-alphalnbeta)+(alpha-1)sum_i=1^n ln x_i+ln(mathbf1_0<x_(n)<beta)$$
Observe that, given the sample, the parameter space has become $$Theta=theta:alpha>0,beta>x_(n)$$
Keeping $alpha$ fixed, justify that $ell(theta)$ is maximized for the minimum value of $beta$ subject to the constraint $betain(x_(n),infty)$. In other words, as you say, $ell(theta)$ is a decreasing function of $beta$ for fixed $alpha$. Hence conclude that MLE of $beta$ as you guessed is $$hatbeta=X_(n)$$
It is now valid to derive the MLE of $alpha$ by differentiating the log-likelihood as you have done. This MLE is likely to depend on the MLE of $beta$.
Indeed,
beginalign
fracpartialellpartialalpha&=fracnalpha-nlnbeta+sum_i=1^n ln x_i
endalign
, which vanishes if and only if $$alpha=fracnnlnbeta-sum_i=1^nln x_i$$
(Since $x_i<betaimplies ln x_i<lnbetaimplies sum ln x_i<nlnbeta$, the above expression is defined.)
So our possible candidate for MLE of $alpha$ is $$hatalpha=fracnnlnhatbeta-sum_i=1^nln x_i$$
At this point, you can finish your argument saying that MLE of $theta=(alpha,beta)$ is $hattheta=(hatalpha,hatbeta)$.
But since this is a maximization problem in two variables $(alpha,beta)$, you could perhaps verify that $$ell(hattheta)ge ell (theta)$$ holds for every $theta$. This would be a bit more rigorous I think.
answered 2 hours ago
StubbornAtom
1,6671325
edited 2 hours ago
answered 2 hours ago
StubbornAtom
1,6671325
answered 2 hours ago
StubbornAtom
1,6671325
answered 2 hours ago
StubbornAtom
1,6671325
1,6671325
add a comment |Â
add a comment |Â
Nic Nic is a new contributor. Be nice, and check out our Code of Conduct.
Nic Nic is a new contributor. Be nice, and check out our Code of Conduct.
Nic Nic is a new contributor. Be nice, and check out our Code of Conduct.
Nic Nic is a new contributor. Be nice, and check out our Code of Conduct.
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f374396%2festimating-the-mle-where-the-parameter-is-also-the-constraint%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
I just realised that since the score function is decreasing in $beta$ then it should be minimized which is gotten through order statistics, so $$hatbeta=X^(n) $$
– Nic Nic
3 hours ago
Slight mistake in the pdf; check the differentiation again.
– StubbornAtom
3 hours ago
Is this is part of an assignment or homework, consider adding the
self-studytag. Also read the tag wiki.– StubbornAtom
3 hours ago
1
Forgot to add the divide by $beta$ in the pdf. It was used in calculating the score-function. It's just practice, so wasn't sure what it classified as.
– Nic Nic
3 hours ago
No problem. What you call the score function is actually the logarithm of the likelihood. Score function comes into play after you differentiate the log-likelihood.
– StubbornAtom
3 hours ago