Mfcttrf

I read the following paragraph on Wikipedia recently:

Bayesian intervals treat their bounds as fixed and the estimated parameter as a random variable, whereas frequentist confidence intervals treat their bounds as random variables and the parameter as a fixed value.

However, I am not sure whether this is true. My interpretation of the credible interval was that it encapsulated our own uncertainty about the true value of the estimated parameter but that the estimated parameter itself did have some kind of 'true' value.

This is slightly different to saying that the estimated parameter is a 'random variable'. Am I wrong?

Consider the situation in which you have $n = 20$ observations of a binary (2-coutcome) process. Often the two possible outcomes on each trial are called Success and Failure.

Frequentist confidence interval. Suppose you observe $x = 15$ successes in the $n = 20$ trials. View the number $X$ of Successes as a random variable $X sim mathsfBinom(n=20; p),$ where the success probability $p$ is an unknown constant. The Wald 95% frequentist confidence interval
is based on $hat p = 15/20 = 0.75,$ an estimate of $p.$
Using a normal approximation, this CI is of the form $hat p pm 1.96sqrthat p(1-hat p)/n$ or
$(0.560, 0.940).$ [The somewhat improved Agresti-Coull
style of 95% CI is $(0.526, 0.890).]$

A common interpretation is that the procedure that
produces such an interval will produce lower and upper confidence limits that include the true value of $p$ in 95% of instances over the long run. [The advantage of the Agresti-Coull interval is that the long run proportion of such inclusions is nearer to 95% than for the Wald interval.]

Bayesian credible interval. The Bayesian approach
begins by treating $p$ as a random variable. Prior to seeing data, if we have no prior experience with the kind binomial experiment being conducted or no personal
opinion as to the distribution of $p,$ we may choose
the 'flat' or 'noninformative' uniform distribution,
saying $p sim mathsfUnif(0, 1) equiv
mathsfBeta(1,1).$

Then, given 15 successes in 20 binomial trials, we find the posterior distribution of $p$ as
the product of the prior distribution and the binomial likelihood function.

$$f(p|x) propto p^1-1(1-p)^1-1 times
p^15(1-p)^5 propto
p^16-1(1-p)^6-1,$$
where the symbol $propto$ (read 'proportional to')
indicates that we are omitting 'norming' constant
factors of the distributions, which do not contain $p.$
Without the norming factor, a density function or PMF
is called the 'kernel' of the distribution.

Here we recognize that the kernel of the posterior distribution is that of the distribution $mathsfBeta(16, 6).$ Then a 95% Bayesian posterior interval
or credible interval is found by cutting 2.5% from each tail of the posterior distribution. Here is the result from R:
$(0.528,0.887).$ [For information about beta distributions, see Wikipedia.]

qbeta(c(.025,.975), 16, 6)
[1] 0.5283402 0.8871906

If we believed the prior to be reasonable and believe that
the 20-trial binomial experiment was fairly conducted,
then logically we must expect the Bayesian
interval estimate to give useful information about
the experiment at hand---with no reference to a hypothetical long-run future.

Notice that this Bayesian credible interval
is numerically similar to the Agresti-Coull confidence interval. However, as you point out,
the interpretations of the two types of interval estimates (frequentist and Bayesian) are not the same.

Informative prior. Before we saw the data, if we had reason to believe
that $p approx 2/3,$ then we might have chosen the
distribution $mathsfBeta(8,4)$ as the prior distribution. [This distribution has mean 2/3, standard deviation about 0.35, and puts about 95% of its
probability in the interval $(0.39, 0.89).$]

qbeta(c(.025,.975), 8,4)
[1] 0.3902574 0.8907366

In that case, multiplying the prior by the likelihood gives the posterior kernel of $mathsfBeta(23,7),$
so that the 95% Bayesian credible interval is
$(0.603, 0.897).$ The posterior distribution is a melding of the information in the prior and the likelihood, which are in rough agreement, so the resulting Bayesian interval
estimate is shorter than than the interval from
the flat prior.

qbeta(c(.025,.975), 23,7)
[1] 0.6027531 0.8970164

Notes: (1) The beta prior and binomial likelihood function
are 'conjugage`, that is, mathematically compatible in a way that allows us to find the posterior distribution without computation. Sometimes, there does not seem to
be a prior distribution that is conjugate with the likelihood. The it may be necessary to use numerical integration to find the posterior distribution.

(2) A Bayesian credible interval from an noninformative prior essentially depends on the likelihood function. Also, much of frequentist inference depends of the likelihood function. Thus is is not
a surprise that a Bayesian credible interval from a flat prior may be numerically similar to a frequentist confidence interval based on the same likelihood.

Your interpretation is correct. In my opinion that particular passage in the Wikipedia article obfuscates a simple concept with opaque technical language. The initial passage is much clearer: "is an interval within which an unobserved parameter value falls with a particular subjective probability".

The technical term "random variable" is misleading, especially from a Bayesian point of view. It's still used just out of tradition; take a look at Shafer's intriguing historical study When to call a variable random about its origins. From a Bayesian point of view, "random" simply means "unknown" or "uncertain" (for whatever reason), and "variable" is a misnomer for "quantity" or "value". For example, when we try to assess our uncertainty about the speed of light $c$ from a measurement or experiment, we speak of $c$ as a "random variable"; but it's obviously not "random" (and what does "random" mean?), nor is it "variable" – in fact, it's a constant. It's just a physical constant whose exact value we're uncertain about.

See § 16.4 (and other places) in Jaynes's book for an illuminating discussion of this topic.

In frequentist theory the term "random variable" may have a different meaning though. I'm not an expert in this theory, so I won't try to define it there. I think there's some literature around that shows that frequentist confidence intervals and Bayesian intervals can be quite different; see for example Confidence intervals vs Bayesian intervals or https://www.ncbi.nlm.nih.gov/pubmed/6830080.