Mixing
Correct understanding of De Finetti`s representation theorem
Clash Royale CLAN TAG#URR8PPP
.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty margin-bottom:0;
up vote
3
down vote
favorite
I am currently interestend in understanding De Finetti`s representation theorem. As I am only familiar with Frequentist thinking I have some problems to understand its meaning. I have already read the great answer by Zen at: What is so cool about de Finetti's representation theorem?, but there are still a couple things that I don't understand.
Recall that De Finetti's Representation Theorem says that $X_i_i=1^infty$ is exchangeable if and only if there is a random variable $Theta:Omegato [0,1]$, with distribution $mu_theta$, such that
$$p(X_1=x_1, ..., X_n=x_n)=int_[0,1]theta^sum_i=1^nx_i(1-theta)^n-sum_i=1^nx_idmu_Theta$$
Furthermore De Finetti`s strong law of lare numbers says that:
$$overlineX_n=frac1nsum_i=1^nX_itoTheta quad a.s. text for n to infty $$
How how am I to interprete this random variable $Theta$ and it`s realisation $theta_T=Theta(omega)$. Can I think of this parameter as being randomly drawn once from a box of tickets, therefore being constant but unknown to the observer?
Why isn't this parameter always the same (or is it the same, e.g. in the case of repeated coin tosses)? Can you give an example or an intuitive explanation that clarifies this idea? Or framed differently what does $mu_Theta$ stand for? Is this Parameter just a construction in our head which - if we knew it - would enable us to construct the beliefs we would have, if we had seen all the outcomes?
bayesian random-variable parameterization
asked 5 hours ago
0rangetree
389112
add a comment |Â
up vote
3
down vote
favorite
I am currently interestend in understanding De Finetti`s representation theorem. As I am only familiar with Frequentist thinking I have some problems to understand its meaning. I have already read the great answer by Zen at: What is so cool about de Finetti's representation theorem?, but there are still a couple things that I don't understand.
Recall that De Finetti's Representation Theorem says that $X_i_i=1^infty$ is exchangeable if and only if there is a random variable $Theta:Omegato [0,1]$, with distribution $mu_theta$, such that
$$p(X_1=x_1, ..., X_n=x_n)=int_[0,1]theta^sum_i=1^nx_i(1-theta)^n-sum_i=1^nx_idmu_Theta$$
Furthermore De Finetti`s strong law of lare numbers says that:
$$overlineX_n=frac1nsum_i=1^nX_itoTheta quad a.s. text for n to infty $$
How how am I to interprete this random variable $Theta$ and it`s realisation $theta_T=Theta(omega)$. Can I think of this parameter as being randomly drawn once from a box of tickets, therefore being constant but unknown to the observer?
Why isn't this parameter always the same (or is it the same, e.g. in the case of repeated coin tosses)? Can you give an example or an intuitive explanation that clarifies this idea? Or framed differently what does $mu_Theta$ stand for? Is this Parameter just a construction in our head which - if we knew it - would enable us to construct the beliefs we would have, if we had seen all the outcomes?
bayesian random-variable parameterization
asked 5 hours ago
0rangetree
389112
add a comment |Â
up vote
3
down vote
favorite
up vote
3
down vote
favorite
I am currently interestend in understanding De Finetti`s representation theorem. As I am only familiar with Frequentist thinking I have some problems to understand its meaning. I have already read the great answer by Zen at: What is so cool about de Finetti's representation theorem?, but there are still a couple things that I don't understand.
Recall that De Finetti's Representation Theorem says that $X_i_i=1^infty$ is exchangeable if and only if there is a random variable $Theta:Omegato [0,1]$, with distribution $mu_theta$, such that
$$p(X_1=x_1, ..., X_n=x_n)=int_[0,1]theta^sum_i=1^nx_i(1-theta)^n-sum_i=1^nx_idmu_Theta$$
Furthermore De Finetti`s strong law of lare numbers says that:
$$overlineX_n=frac1nsum_i=1^nX_itoTheta quad a.s. text for n to infty $$
How how am I to interprete this random variable $Theta$ and it`s realisation $theta_T=Theta(omega)$. Can I think of this parameter as being randomly drawn once from a box of tickets, therefore being constant but unknown to the observer?
Why isn't this parameter always the same (or is it the same, e.g. in the case of repeated coin tosses)? Can you give an example or an intuitive explanation that clarifies this idea? Or framed differently what does $mu_Theta$ stand for? Is this Parameter just a construction in our head which - if we knew it - would enable us to construct the beliefs we would have, if we had seen all the outcomes?
bayesian random-variable parameterization
asked 5 hours ago
0rangetree
389112
I am currently interestend in understanding De Finetti`s representation theorem. As I am only familiar with Frequentist thinking I have some problems to understand its meaning. I have already read the great answer by Zen at: What is so cool about de Finetti's representation theorem?, but there are still a couple things that I don't understand.
Recall that De Finetti's Representation Theorem says that $X_i_i=1^infty$ is exchangeable if and only if there is a random variable $Theta:Omegato [0,1]$, with distribution $mu_theta$, such that
$$p(X_1=x_1, ..., X_n=x_n)=int_[0,1]theta^sum_i=1^nx_i(1-theta)^n-sum_i=1^nx_idmu_Theta$$
Furthermore De Finetti`s strong law of lare numbers says that:
$$overlineX_n=frac1nsum_i=1^nX_itoTheta quad a.s. text for n to infty $$
How how am I to interprete this random variable $Theta$ and it`s realisation $theta_T=Theta(omega)$. Can I think of this parameter as being randomly drawn once from a box of tickets, therefore being constant but unknown to the observer?
Why isn't this parameter always the same (or is it the same, e.g. in the case of repeated coin tosses)? Can you give an example or an intuitive explanation that clarifies this idea? Or framed differently what does $mu_Theta$ stand for? Is this Parameter just a construction in our head which - if we knew it - would enable us to construct the beliefs we would have, if we had seen all the outcomes?
bayesian random-variable parameterization
bayesian random-variable parameterization
asked 5 hours ago
0rangetree
389112
asked 5 hours ago
0rangetree
389112
edited 5 hours ago
asked 5 hours ago
0rangetree
389112
asked 5 hours ago
0rangetree
389112
asked 5 hours ago
0rangetree
389112
389112
add a comment |Â
add a comment |Â
1 Answer
1
active
oldest
votes
up vote
3
down vote
accepted
What is the proper interpretation of the parameter? The natural interpretation of the parameter $Theta$ comes from the law-of-large numbers, which you have stated in your question. This says that the following equivalence holds almost surely:
$$Theta = lim_n rightarrow infty frac1n sum_i=1^n X_i.$$
From this equation we see that $Theta$ is the limiting proportion of positive outcomes ($X_i=1$) in the observeable sequence.$^dagger$ Since you are familiar with frequentist thinking, you will recognise this as the frequentist definition of the probability of a positive outcome in the sequence. In fact, there is a good argument to be made that de Finetti's theorem is a statement of the equivalence that is used in frequentist theory as the definition of probability.
To give an applied example of this interpretation, suppose that the outcomes $X_i$ are indicators for a sequence of coin flips, indicating an outcome of heads. In this case the observable sequence $X_1,X_2,X_3,...$ is an exchangeable sequence of coin flips and the parameter $Theta$ is the long-run limiting proportion of heads in the sequence.
In your question you also ask if the parameter is "just a construction in our heads". Well, it is no more of a construction than the notion that there is an infinite sequence of outcomes in the first place. If we are willing to assume that there is an infinite sequence of observable outcomes (which is itself arguably a mere hypothetical construction) then that sequence has a limiting proportion of positive outcomes$^dagger$, and that limiting proportion is the parameter.
$^dagger$ There is a slight technicality here, since the limiting proportion is a Cesaro limit that does not exist for all possible sequences. The parameter can be defined as the Banach limit, which always exists; for discussion, see O'Neill (2009).
Is the parameter an unknown constant? In classical frequentist analysis, the parameters are treated as "unknown constants". There is no such thing as an unknown constant in Bayesian statistics, since all unknown quantities are treated as random variables with a prior distribution. Hence, if you are going to treat $Theta$ as an unknown constant, you are using frequentist analysis, not Bayesian analysis.
Why isn't the parameter always the same? As shown above, the parameter $Theta$ represents the limiting proportion of positive outcomes in the sequence. Clearly, one can imagine that there are different sequences of exchangeable outcomes where the limiting proportions of positive outcomes are different, and hence, the parameter would not be the same across different sequences. For example, a particular exchangeable sequence of coin-tosses might have a limiting proportion of heads of $Theta = 0.5$, meaning that we are dealing with a "fair" coin. Alternatively, we might have an exchangeable sequence of coin-tosses with a limiting proportion of heads of $Theta = 0.51$, meaning that we are dealing with coin that is "biased" towards heads.
What does $mu_Theta$ stand for? This is the prior probability measure for the parameter (equivalent to its prior distribution). In Bayesian analysis, this measure represents our beliefs about the unknown parameter prior to seeing the data. After we see the data we then update this to a posterior belief, represented by the posterior probability measure (equivalently a posterior distribution).
answered 4 hours ago
Ben
14.5k12176
add a comment |Â
1 Answer
1
active
oldest
votes
1 Answer
1
active
oldest
votes
active
oldest
votes
active
oldest
votes
up vote
3
down vote
accepted
What is the proper interpretation of the parameter? The natural interpretation of the parameter $Theta$ comes from the law-of-large numbers, which you have stated in your question. This says that the following equivalence holds almost surely:
$$Theta = lim_n rightarrow infty frac1n sum_i=1^n X_i.$$
From this equation we see that $Theta$ is the limiting proportion of positive outcomes ($X_i=1$) in the observeable sequence.$^dagger$ Since you are familiar with frequentist thinking, you will recognise this as the frequentist definition of the probability of a positive outcome in the sequence. In fact, there is a good argument to be made that de Finetti's theorem is a statement of the equivalence that is used in frequentist theory as the definition of probability.
To give an applied example of this interpretation, suppose that the outcomes $X_i$ are indicators for a sequence of coin flips, indicating an outcome of heads. In this case the observable sequence $X_1,X_2,X_3,...$ is an exchangeable sequence of coin flips and the parameter $Theta$ is the long-run limiting proportion of heads in the sequence.
In your question you also ask if the parameter is "just a construction in our heads". Well, it is no more of a construction than the notion that there is an infinite sequence of outcomes in the first place. If we are willing to assume that there is an infinite sequence of observable outcomes (which is itself arguably a mere hypothetical construction) then that sequence has a limiting proportion of positive outcomes$^dagger$, and that limiting proportion is the parameter.
$^dagger$ There is a slight technicality here, since the limiting proportion is a Cesaro limit that does not exist for all possible sequences. The parameter can be defined as the Banach limit, which always exists; for discussion, see O'Neill (2009).
Is the parameter an unknown constant? In classical frequentist analysis, the parameters are treated as "unknown constants". There is no such thing as an unknown constant in Bayesian statistics, since all unknown quantities are treated as random variables with a prior distribution. Hence, if you are going to treat $Theta$ as an unknown constant, you are using frequentist analysis, not Bayesian analysis.
Why isn't the parameter always the same? As shown above, the parameter $Theta$ represents the limiting proportion of positive outcomes in the sequence. Clearly, one can imagine that there are different sequences of exchangeable outcomes where the limiting proportions of positive outcomes are different, and hence, the parameter would not be the same across different sequences. For example, a particular exchangeable sequence of coin-tosses might have a limiting proportion of heads of $Theta = 0.5$, meaning that we are dealing with a "fair" coin. Alternatively, we might have an exchangeable sequence of coin-tosses with a limiting proportion of heads of $Theta = 0.51$, meaning that we are dealing with coin that is "biased" towards heads.
What does $mu_Theta$ stand for? This is the prior probability measure for the parameter (equivalent to its prior distribution). In Bayesian analysis, this measure represents our beliefs about the unknown parameter prior to seeing the data. After we see the data we then update this to a posterior belief, represented by the posterior probability measure (equivalently a posterior distribution).
answered 4 hours ago
Ben
14.5k12176
add a comment |Â
up vote
3
down vote
accepted
What is the proper interpretation of the parameter? The natural interpretation of the parameter $Theta$ comes from the law-of-large numbers, which you have stated in your question. This says that the following equivalence holds almost surely:
$$Theta = lim_n rightarrow infty frac1n sum_i=1^n X_i.$$
From this equation we see that $Theta$ is the limiting proportion of positive outcomes ($X_i=1$) in the observeable sequence.$^dagger$ Since you are familiar with frequentist thinking, you will recognise this as the frequentist definition of the probability of a positive outcome in the sequence. In fact, there is a good argument to be made that de Finetti's theorem is a statement of the equivalence that is used in frequentist theory as the definition of probability.
To give an applied example of this interpretation, suppose that the outcomes $X_i$ are indicators for a sequence of coin flips, indicating an outcome of heads. In this case the observable sequence $X_1,X_2,X_3,...$ is an exchangeable sequence of coin flips and the parameter $Theta$ is the long-run limiting proportion of heads in the sequence.
In your question you also ask if the parameter is "just a construction in our heads". Well, it is no more of a construction than the notion that there is an infinite sequence of outcomes in the first place. If we are willing to assume that there is an infinite sequence of observable outcomes (which is itself arguably a mere hypothetical construction) then that sequence has a limiting proportion of positive outcomes$^dagger$, and that limiting proportion is the parameter.
$^dagger$ There is a slight technicality here, since the limiting proportion is a Cesaro limit that does not exist for all possible sequences. The parameter can be defined as the Banach limit, which always exists; for discussion, see O'Neill (2009).
Is the parameter an unknown constant? In classical frequentist analysis, the parameters are treated as "unknown constants". There is no such thing as an unknown constant in Bayesian statistics, since all unknown quantities are treated as random variables with a prior distribution. Hence, if you are going to treat $Theta$ as an unknown constant, you are using frequentist analysis, not Bayesian analysis.
Why isn't the parameter always the same? As shown above, the parameter $Theta$ represents the limiting proportion of positive outcomes in the sequence. Clearly, one can imagine that there are different sequences of exchangeable outcomes where the limiting proportions of positive outcomes are different, and hence, the parameter would not be the same across different sequences. For example, a particular exchangeable sequence of coin-tosses might have a limiting proportion of heads of $Theta = 0.5$, meaning that we are dealing with a "fair" coin. Alternatively, we might have an exchangeable sequence of coin-tosses with a limiting proportion of heads of $Theta = 0.51$, meaning that we are dealing with coin that is "biased" towards heads.
What does $mu_Theta$ stand for? This is the prior probability measure for the parameter (equivalent to its prior distribution). In Bayesian analysis, this measure represents our beliefs about the unknown parameter prior to seeing the data. After we see the data we then update this to a posterior belief, represented by the posterior probability measure (equivalently a posterior distribution).
answered 4 hours ago
Ben
14.5k12176
add a comment |Â
up vote
3
down vote
accepted
up vote
3
down vote
accepted
What is the proper interpretation of the parameter? The natural interpretation of the parameter $Theta$ comes from the law-of-large numbers, which you have stated in your question. This says that the following equivalence holds almost surely:
$$Theta = lim_n rightarrow infty frac1n sum_i=1^n X_i.$$
From this equation we see that $Theta$ is the limiting proportion of positive outcomes ($X_i=1$) in the observeable sequence.$^dagger$ Since you are familiar with frequentist thinking, you will recognise this as the frequentist definition of the probability of a positive outcome in the sequence. In fact, there is a good argument to be made that de Finetti's theorem is a statement of the equivalence that is used in frequentist theory as the definition of probability.
To give an applied example of this interpretation, suppose that the outcomes $X_i$ are indicators for a sequence of coin flips, indicating an outcome of heads. In this case the observable sequence $X_1,X_2,X_3,...$ is an exchangeable sequence of coin flips and the parameter $Theta$ is the long-run limiting proportion of heads in the sequence.
In your question you also ask if the parameter is "just a construction in our heads". Well, it is no more of a construction than the notion that there is an infinite sequence of outcomes in the first place. If we are willing to assume that there is an infinite sequence of observable outcomes (which is itself arguably a mere hypothetical construction) then that sequence has a limiting proportion of positive outcomes$^dagger$, and that limiting proportion is the parameter.
$^dagger$ There is a slight technicality here, since the limiting proportion is a Cesaro limit that does not exist for all possible sequences. The parameter can be defined as the Banach limit, which always exists; for discussion, see O'Neill (2009).
Is the parameter an unknown constant? In classical frequentist analysis, the parameters are treated as "unknown constants". There is no such thing as an unknown constant in Bayesian statistics, since all unknown quantities are treated as random variables with a prior distribution. Hence, if you are going to treat $Theta$ as an unknown constant, you are using frequentist analysis, not Bayesian analysis.
Why isn't the parameter always the same? As shown above, the parameter $Theta$ represents the limiting proportion of positive outcomes in the sequence. Clearly, one can imagine that there are different sequences of exchangeable outcomes where the limiting proportions of positive outcomes are different, and hence, the parameter would not be the same across different sequences. For example, a particular exchangeable sequence of coin-tosses might have a limiting proportion of heads of $Theta = 0.5$, meaning that we are dealing with a "fair" coin. Alternatively, we might have an exchangeable sequence of coin-tosses with a limiting proportion of heads of $Theta = 0.51$, meaning that we are dealing with coin that is "biased" towards heads.
What does $mu_Theta$ stand for? This is the prior probability measure for the parameter (equivalent to its prior distribution). In Bayesian analysis, this measure represents our beliefs about the unknown parameter prior to seeing the data. After we see the data we then update this to a posterior belief, represented by the posterior probability measure (equivalently a posterior distribution).
answered 4 hours ago
Ben
14.5k12176
What is the proper interpretation of the parameter? The natural interpretation of the parameter $Theta$ comes from the law-of-large numbers, which you have stated in your question. This says that the following equivalence holds almost surely:
$$Theta = lim_n rightarrow infty frac1n sum_i=1^n X_i.$$
From this equation we see that $Theta$ is the limiting proportion of positive outcomes ($X_i=1$) in the observeable sequence.$^dagger$ Since you are familiar with frequentist thinking, you will recognise this as the frequentist definition of the probability of a positive outcome in the sequence. In fact, there is a good argument to be made that de Finetti's theorem is a statement of the equivalence that is used in frequentist theory as the definition of probability.
To give an applied example of this interpretation, suppose that the outcomes $X_i$ are indicators for a sequence of coin flips, indicating an outcome of heads. In this case the observable sequence $X_1,X_2,X_3,...$ is an exchangeable sequence of coin flips and the parameter $Theta$ is the long-run limiting proportion of heads in the sequence.
In your question you also ask if the parameter is "just a construction in our heads". Well, it is no more of a construction than the notion that there is an infinite sequence of outcomes in the first place. If we are willing to assume that there is an infinite sequence of observable outcomes (which is itself arguably a mere hypothetical construction) then that sequence has a limiting proportion of positive outcomes$^dagger$, and that limiting proportion is the parameter.
$^dagger$ There is a slight technicality here, since the limiting proportion is a Cesaro limit that does not exist for all possible sequences. The parameter can be defined as the Banach limit, which always exists; for discussion, see O'Neill (2009).
Is the parameter an unknown constant? In classical frequentist analysis, the parameters are treated as "unknown constants". There is no such thing as an unknown constant in Bayesian statistics, since all unknown quantities are treated as random variables with a prior distribution. Hence, if you are going to treat $Theta$ as an unknown constant, you are using frequentist analysis, not Bayesian analysis.
Why isn't the parameter always the same? As shown above, the parameter $Theta$ represents the limiting proportion of positive outcomes in the sequence. Clearly, one can imagine that there are different sequences of exchangeable outcomes where the limiting proportions of positive outcomes are different, and hence, the parameter would not be the same across different sequences. For example, a particular exchangeable sequence of coin-tosses might have a limiting proportion of heads of $Theta = 0.5$, meaning that we are dealing with a "fair" coin. Alternatively, we might have an exchangeable sequence of coin-tosses with a limiting proportion of heads of $Theta = 0.51$, meaning that we are dealing with coin that is "biased" towards heads.
What does $mu_Theta$ stand for? This is the prior probability measure for the parameter (equivalent to its prior distribution). In Bayesian analysis, this measure represents our beliefs about the unknown parameter prior to seeing the data. After we see the data we then update this to a posterior belief, represented by the posterior probability measure (equivalently a posterior distribution).
answered 4 hours ago
Ben
14.5k12176
answered 4 hours ago
Ben
14.5k12176
answered 4 hours ago
Ben
14.5k12176
answered 4 hours ago
Ben
14.5k12176
14.5k12176
add a comment |Â
add a comment |Â
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
StackExchange.ready(
function ()
StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fstats.stackexchange.com%2fquestions%2f367135%2fcorrect-understanding-of-de-finettis-representation-theorem%23new-answer', 'question_page');
);
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Post as a guest
Sign up or log in
StackExchange.ready(function ()
StackExchange.helpers.onClickDraftSave('#login-link');
);
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
Sign up using Google
Sign up using Facebook
Sign up using Email and Password
