Mixing
Can a proper prior and exponentiated likelihood lead to an improper posterior?
Clash Royale CLAN TAG#URR8PPP
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty margin-bottom:0;
up vote
3
down vote
favorite
(This question is inspired by this comment from Xi'an.)
It is well known that if the prior distribution $pi(theta)$ is proper and the likelihood $L(theta | x)$ is well-defined, then the posterior distribution $pi(theta|x)propto pi(theta) L(theta|x)$ is proper almost surely.
In some cases, we use instead a tempered or exponentiated likelihood, leading to a pseudo-posterior
$$tildepi(theta|x)propto pi(theta) L(theta|x)^alpha$$
for some $alpha>0$ (for example, this can have computational advantages).
In this setting, is it possible to have a proper prior but an improper pseudo-posterior?
bayesian prior posterior
asked 3 hours ago
Robin Ryder
985413
add a comment |Â
up vote
3
down vote
favorite
(This question is inspired by this comment from Xi'an.)
It is well known that if the prior distribution $pi(theta)$ is proper and the likelihood $L(theta | x)$ is well-defined, then the posterior distribution $pi(theta|x)propto pi(theta) L(theta|x)$ is proper almost surely.
In some cases, we use instead a tempered or exponentiated likelihood, leading to a pseudo-posterior
$$tildepi(theta|x)propto pi(theta) L(theta|x)^alpha$$
for some $alpha>0$ (for example, this can have computational advantages).
In this setting, is it possible to have a proper prior but an improper pseudo-posterior?
bayesian prior posterior
asked 3 hours ago
Robin Ryder
985413
Actually, a few minutes later, I would consider it unlikely since the divergence of the prior x likelihood product is reduced when considering the prior x likelihood^ α product... Any tern going to infinity is going there more slowly! And terms going to zero more slowly are controlled by the proper prior. My bet is thus that this is impossible. (warning: I have been known to be wrong!)
– Xi'an
3 hours ago
add a comment |Â
up vote
3
down vote
favorite
up vote
3
down vote
favorite
(This question is inspired by this comment from Xi'an.)
It is well known that if the prior distribution $pi(theta)$ is proper and the likelihood $L(theta | x)$ is well-defined, then the posterior distribution $pi(theta|x)propto pi(theta) L(theta|x)$ is proper almost surely.
In some cases, we use instead a tempered or exponentiated likelihood, leading to a pseudo-posterior
$$tildepi(theta|x)propto pi(theta) L(theta|x)^alpha$$
for some $alpha>0$ (for example, this can have computational advantages).
In this setting, is it possible to have a proper prior but an improper pseudo-posterior?
bayesian prior posterior
asked 3 hours ago
Robin Ryder
985413
(This question is inspired by this comment from Xi'an.)
It is well known that if the prior distribution $pi(theta)$ is proper and the likelihood $L(theta | x)$ is well-defined, then the posterior distribution $pi(theta|x)propto pi(theta) L(theta|x)$ is proper almost surely.
In some cases, we use instead a tempered or exponentiated likelihood, leading to a pseudo-posterior
$$tildepi(theta|x)propto pi(theta) L(theta|x)^alpha$$
for some $alpha>0$ (for example, this can have computational advantages).
In this setting, is it possible to have a proper prior but an improper pseudo-posterior?
bayesian prior posterior
bayesian prior posterior
asked 3 hours ago
Robin Ryder
985413
asked 3 hours ago
Robin Ryder
985413
edited 2 hours ago
asked 3 hours ago
Robin Ryder
985413
asked 3 hours ago
Robin Ryder
985413
asked 3 hours ago
Robin Ryder
985413
985413
Actually, a few minutes later, I would consider it unlikely since the divergence of the prior x likelihood product is reduced when considering the prior x likelihood^ α product... Any tern going to infinity is going there more slowly! And terms going to zero more slowly are controlled by the proper prior. My bet is thus that this is impossible. (warning: I have been known to be wrong!)
– Xi'an
3 hours ago
add a comment |Â
Actually, a few minutes later, I would consider it unlikely since the divergence of the prior x likelihood product is reduced when considering the prior x likelihood^ α product... Any tern going to infinity is going there more slowly! And terms going to zero more slowly are controlled by the proper prior. My bet is thus that this is impossible. (warning: I have been known to be wrong!)
– Xi'an
3 hours ago
Actually, a few minutes later, I would consider it unlikely since the divergence of the prior x likelihood product is reduced when considering the prior x likelihood^ α product... Any tern going to infinity is going there more slowly! And terms going to zero more slowly are controlled by the proper prior. My bet is thus that this is impossible. (warning: I have been known to be wrong!)
– Xi'an
3 hours ago
Actually, a few minutes later, I would consider it unlikely since the divergence of the prior x likelihood product is reduced when considering the prior x likelihood^ α product... Any tern going to infinity is going there more slowly! And terms going to zero more slowly are controlled by the proper prior. My bet is thus that this is impossible. (warning: I have been known to be wrong!)
– Xi'an
3 hours ago
add a comment |Â
1 Answer
1
active
oldest
votes
up vote
3
down vote
accepted
For $alpha leq 1$, perhaps this is an argument to show that it is impossible to construct such a posterior?
We'd like to find out if it's possible for $int tilde pi(theta|x)dtheta = infty$.
On the RHS:
$$ int pi(theta) L^alpha(theta|x) dtheta = E_theta(L^alpha(theta|x))$$
If $alpha leq 1$, $x^alpha$ is a concave function, so by the Jensen inequality:
$$ E_theta(L^alpha(theta|x)) leq E^alpha_theta(L(theta|x)) = m(x)^alpha < infty $$
... where $m(x)$ as Xi'an pointed out, is the normalising constant (the evidence).
edited 1 hour ago
Xi'an
50.4k686335
answered 2 hours ago
InfProbSciX
1347
Neat, thanks. I like that you are using the fact that for $alpha=1$ the posterior is proper.
– Robin Ryder
55 mins ago
add a comment |Â
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
3
down vote
accepted
For $alpha leq 1$, perhaps this is an argument to show that it is impossible to construct such a posterior?
We'd like to find out if it's possible for $int tilde pi(theta|x)dtheta = infty$.
On the RHS:
$$ int pi(theta) L^alpha(theta|x) dtheta = E_theta(L^alpha(theta|x))$$
If $alpha leq 1$, $x^alpha$ is a concave function, so by the Jensen inequality:
$$ E_theta(L^alpha(theta|x)) leq E^alpha_theta(L(theta|x)) = m(x)^alpha < infty $$
... where $m(x)$ as Xi'an pointed out, is the normalising constant (the evidence).
edited 1 hour ago
Xi'an
50.4k686335
answered 2 hours ago
InfProbSciX
1347
Neat, thanks. I like that you are using the fact that for $alpha=1$ the posterior is proper.
– Robin Ryder
55 mins ago
add a comment |Â
up vote
3
down vote
accepted
For $alpha leq 1$, perhaps this is an argument to show that it is impossible to construct such a posterior?
We'd like to find out if it's possible for $int tilde pi(theta|x)dtheta = infty$.
On the RHS:
$$ int pi(theta) L^alpha(theta|x) dtheta = E_theta(L^alpha(theta|x))$$
If $alpha leq 1$, $x^alpha$ is a concave function, so by the Jensen inequality:
$$ E_theta(L^alpha(theta|x)) leq E^alpha_theta(L(theta|x)) = m(x)^alpha < infty $$
... where $m(x)$ as Xi'an pointed out, is the normalising constant (the evidence).
edited 1 hour ago
Xi'an
50.4k686335
answered 2 hours ago
InfProbSciX
1347
Neat, thanks. I like that you are using the fact that for $alpha=1$ the posterior is proper.
– Robin Ryder
55 mins ago
add a comment |Â
up vote
3
down vote
accepted
up vote
3
down vote
accepted
For $alpha leq 1$, perhaps this is an argument to show that it is impossible to construct such a posterior?
We'd like to find out if it's possible for $int tilde pi(theta|x)dtheta = infty$.
On the RHS:
$$ int pi(theta) L^alpha(theta|x) dtheta = E_theta(L^alpha(theta|x))$$
If $alpha leq 1$, $x^alpha$ is a concave function, so by the Jensen inequality:
$$ E_theta(L^alpha(theta|x)) leq E^alpha_theta(L(theta|x)) = m(x)^alpha < infty $$
... where $m(x)$ as Xi'an pointed out, is the normalising constant (the evidence).
edited 1 hour ago
Xi'an
50.4k686335
answered 2 hours ago
InfProbSciX
1347
For $alpha leq 1$, perhaps this is an argument to show that it is impossible to construct such a posterior?
We'd like to find out if it's possible for $int tilde pi(theta|x)dtheta = infty$.
On the RHS:
$$ int pi(theta) L^alpha(theta|x) dtheta = E_theta(L^alpha(theta|x))$$
If $alpha leq 1$, $x^alpha$ is a concave function, so by the Jensen inequality:
$$ E_theta(L^alpha(theta|x)) leq E^alpha_theta(L(theta|x)) = m(x)^alpha < infty $$
... where $m(x)$ as Xi'an pointed out, is the normalising constant (the evidence).
edited 1 hour ago
Xi'an
50.4k686335
answered 2 hours ago
InfProbSciX
1347
edited 1 hour ago
Xi'an
50.4k686335
edited 1 hour ago
Xi'an
50.4k686335
edited 1 hour ago
Xi'an
50.4k686335
50.4k686335
answered 2 hours ago
InfProbSciX
1347
answered 2 hours ago
InfProbSciX
1347
answered 2 hours ago
InfProbSciX
1347
1347
Neat, thanks. I like that you are using the fact that for $alpha=1$ the posterior is proper.
– Robin Ryder
55 mins ago
add a comment |Â
Neat, thanks. I like that you are using the fact that for $alpha=1$ the posterior is proper.
– Robin Ryder
55 mins ago
Neat, thanks. I like that you are using the fact that for $alpha=1$ the posterior is proper.
– Robin Ryder
55 mins ago
Neat, thanks. I like that you are using the fact that for $alpha=1$ the posterior is proper.
– Robin Ryder
55 mins ago
add a comment |Â
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f370919%2fcan-a-proper-prior-and-exponentiated-likelihood-lead-to-an-improper-posterior%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Actually, a few minutes later, I would consider it unlikely since the divergence of the prior x likelihood product is reduced when considering the prior x likelihood^ α product... Any tern going to infinity is going there more slowly! And terms going to zero more slowly are controlled by the proper prior. My bet is thus that this is impossible. (warning: I have been known to be wrong!)
– Xi'an
3 hours ago