bayesian probability

 

  • [29] Personal probabilities and objective methods for constructing priors Following the work on expected utility theory of Ramsey and von Neumann, decision-theorists have
    accounted for rational behavior using a probability distribution for the agent.

  • These theorists and their successors have suggested several methods for constructing “objective” priors (Unfortunately, it is not clear how to assess the relative “objectivity”
    of the priors proposed under these methods): • Maximum entropy • Transformation group analysis • Reference analysis Each of these methods contributes useful priors for “regular” one-parameter problems, and each prior can handle some challenging
    statistical models (with “irregularity” or several parameters).

  • [7]: 97–98  Bayesian methodology Bayesian methods are characterized by concepts and procedures as follows: • The use of random variables, or more generally unknown quantities,[8]
    to model all sources of uncertainty in statistical models including uncertainty resulting from lack of information (see also aleatoric and epistemic uncertainty).

  • The term Bayesian derives from the 18th-century mathematician and theologian Thomas Bayes, who provided the first mathematical treatment of a non-trivial problem of statistical
    data analysis using what is now known as Bayesian inference.

  • Personal probabilities are problematic for science and for some applications where decision-makers lack the knowledge or time to specify an informed probability-distribution
    (on which they are prepared to act).

  • • While for the frequentist, a hypothesis is a proposition (which must be either true or false) so that the frequentist probability of a hypothesis is either 0 or 1, in Bayesian
    statistics, the probability that can be assigned to a hypothesis can also be in a range from 0 to 1 if the truth value is uncertain.

  • To meet the needs of science and of human limitations, Bayesian statisticians have developed “objective” methods for specifying prior probabilities.

  • [17] While frequentist statistics remains strong (as demonstrated by the fact that much of undergraduate teaching is based on it [18]), Bayesian methods are widely accepted
    and used, e.g., in the field of machine learning.

  • [27] Decision theory approach[edit] A decision-theoretic justification of the use of Bayesian inference (and hence of Bayesian probabilities) was given by Abraham Wald, who
    proved that every admissible statistical procedure is either a Bayesian procedure or a limit of Bayesian procedures.

  • Johann Pfanzagl completed the Theory of Games and Economic Behavior by providing an axiomatization of subjective probability and utility, a task left uncompleted by von Neumann
    and Oskar Morgenstern: their original theory supposed that all the agents had the same probability distribution, as a convenience.

  • [30] Pfanzagl’s axiomatization was endorsed by Oskar Morgenstern: “Von Neumann and I have anticipated … [the question whether probabilities] might, perhaps more typically,
    be subjective and have stated specifically that in the latter case axioms could be found from which could derive the desired numerical utility together with a number for the probabilities (cf.

  • Indeed, some Bayesians have argued the prior state of knowledge defines the (unique) prior probability-distribution for “regular” statistical problems; cf.

  • History The term Bayesian derives from Thomas Bayes (1702–1761), who proved a special case of what is now called Bayes’ theorem in a paper titled “An Essay towards solving
    a Problem in the Doctrine of Chances”.

  • [12] In the 20th century, the ideas of Laplace developed in two directions, giving rise to objective and subjective currents in Bayesian practice.

  • [11] Early Bayesian inference, which used uniform priors following Laplace’s principle of insufficient reason, was called “inverse probability” (because it infers backwards
    from observations to parameters, or from effects to causes).

  • Bayesian probability is an interpretation of the concept of probability, in which, instead of frequency or propensity of some phenomenon, probability is interpreted as reasonable
    expectation[1] representing a state of knowledge[2] or as quantification of a personal belief.

  • [39] Thus, the Bayesian statistician needs either to use informed priors (using relevant expertise or previous data) or to choose among the competing methods for constructing
    “objective” priors.

  • [21] Other axiomatizations have been suggested by various authors with the purpose of making the theory more rigorous.

  • For objectivists, who interpret probability as an extension of logic, probability quantifies the reasonable expectation that everyone (even a “robot”) who shares the same
    knowledge should share in accordance with the rules of Bayesian statistics, which can be justified by Cox’s theorem.

  • [35] This work demonstrates that Bayesian-probability propositions can be falsified, and so meet an empirical criterion of Charles S. Peirce, whose work inspired Ramsey.

  • [19] Justification of Bayesian probabilities The use of Bayesian probabilities as the basis of Bayesian inference has been supported by several arguments, such as Cox axioms,
    the Dutch book argument, arguments based on decision theory and de Finetti’s theorem.

  • [38] Since individuals act according to different probability judgments, these agents’ probabilities are “personal” (but amenable to objective study).

  • Axiomatic approach[edit] Richard T. Cox showed that Bayesian updating follows from several axioms, including two functional equations and a hypothesis of differentiability.

 

Works Cited

[‘1. Cox, R.T. (1946). “Probability, Frequency, and Reasonable Expectation”. American Journal of Physics. 14 (1): 1–10. Bibcode:1946AmJPh..14….1C. doi:10.1119/1.1990764.
2. ^ Jump up to:a b Jaynes, E.T. (1986). “Bayesian Methods: General Background”.
In Justice, J. H. (ed.). Maximum-Entropy and Bayesian Methods in Applied Statistics. Cambridge: Cambridge University Press. CiteSeerX 10.1.1.41.1055.
3. ^ Jump up to:a b c de Finetti, Bruno (2017). Theory of Probability: A critical introductory
treatment. Chichester: John Wiley & Sons Ltd. ISBN 9781119286370.
4. ^ Hailperin, Theodore (1996). Sentential Probability Logic: Origins, Development, Current Status, and Technical Applications. London: Associated University Presses. ISBN 0934223459.
5. ^
Howson, Colin (2001). “The Logic of Bayesian Probability”. In Corfield, D.; Williamson, J. (eds.). Foundations of Bayesianism. Dordrecht: Kluwer. pp. 137–159. ISBN 1-4020-0223-8.
6. ^ Paulos, John Allen (5 August 2011). “The Mathematics of Changing
Your Mind [by Sharon Bertsch McGrayne]”. Book Review. New York Times. Archived from the original on 2022-01-01. Retrieved 2011-08-06.
7. ^ Jump up to:a b Stigler, Stephen M. (March 1990). The history of statistics. Harvard University Press. ISBN
9780674403413.
8. ^ Jump up to:a b Dupré, Maurice J.; Tipler, Frank J. (2009). “New axioms for rigorous Bayesian probability”. Bayesian Analysis. 4 (3): 599–606. CiteSeerX 10.1.1.612.3036. doi:10.1214/09-BA422.
9. ^ Jump up to:a b Cox, Richard
T. (1961). The algebra of probable inference (Reprint ed.). Baltimore, MD; London, UK: Johns Hopkins Press; Oxford University Press [distributor]. ISBN 9780801869822.
10. ^ McGrayne, Sharon Bertsch (2011). The Theory that Would not Die. [https://archive.org/details/theorythatwouldn0000mcgr/page/10
10 ], p. 10, at Google Books.
11. ^ Stigler, Stephen M. (1986). “Chapter 3”. The History of Statistics. Harvard University Press. ISBN 9780674403406.
12. ^ Jump up to:a b Fienberg, Stephen. E. (2006). “When did Bayesian Inference become “Bayesian”?”
(PDF). Bayesian Analysis. 1 (1): 5, 1–40. doi:10.1214/06-BA101. Archived from the original (PDF) on 10 September 2014.
13. ^ Harris, Marshall Dees (1959). “Recent developments of the so-called Bayesian approach to statistics”. Agricultural Law Center.
Legal-Economic Research. University of Iowa: 125 (fn. #52), 126. The works of Wald, Statistical Decision Functions (1950) and Savage, The Foundation of Statistics (1954) are commonly regarded starting points for current Bayesian approaches
14. ^
Annals of the Computation Laboratory of Harvard University. Vol. 31. 1962. p. 180. This revolution, which may or may not succeed, is neo-Bayesianism. Jeffreys tried to introduce this approach, but did not succeed at the time in giving it general appeal.
15. ^
Kempthorne, Oscar (1967). The Classical Problem of Inference—Goodness of Fit. Fifth Berkeley Symposium on Mathematical Statistics and Probability. p. 235. It is curious that even in its activities unrelated to ethics, humanity searches for a religion.
At the present time, the religion being ‘pushed’ the hardest is Bayesianism.
16. ^ Bernardo, J.M. (2005). “Reference analysis”. Bayesian Thinking – Modeling and Computation. Handbook of Statistics. Vol. 25. pp. 17–90. doi:10.1016/S0169-7161(05)25002-2.
ISBN 9780444515391.
17. ^ Wolpert, R.L. (2004). “A conversation with James O. Berger”. Statistical Science. 9: 205–218. doi:10.1214/088342304000000053.
18. ^ Bernardo, José M. (2006). A Bayesian mathematical statistics primer (PDF). ICOTS-7. Bern.
Archived (PDF) from the original on 2022-10-09.
19. ^ Bishop, C.M. (2007). Pattern Recognition and Machine Learning. Springer.
20. ^ Smith, C. Ray; Erickson, Gary (1989). “From Rationality and Consistency to Bayesian Probability”. In Skilling,
John (ed.). Maximum Entropy and Bayesian Methods. Dordrecht: Kluwer. pp. 29–44. doi:10.1007/978-94-015-7860-8_2. ISBN 0-7923-0224-9.
21. ^ Halpern, J. (1999). “A counterexample to theorems of Cox and Fine” (PDF). Journal of Artificial Intelligence
Research. 10: 67–85. doi:10.1613/jair.536. S2CID 1538503. Archived (PDF) from the original on 2022-10-09.
22. ^ Hacking (1967), Section 3, page 316
23. ^ Hacking (1988, page 124)
24. ^ Skyrms, Brian (1 January 1987). “Dynamic Coherence and Probability
Kinematics”. Philosophy of Science. 54 (1): 1–20. CiteSeerX 10.1.1.395.5723. doi:10.1086/289350. JSTOR 187470. S2CID 120881078.
25. ^ Joyce, James (30 September 2003). “Bayes’ Theorem”. The Stanford Encyclopedia of Philosophy. stanford.edu.
26. ^
Fuchs, Christopher A.; Schack, Rüdiger (1 January 2012). Ben-Menahem, Yemima; Hemmo, Meir (eds.). Probability in Physics. The Frontiers Collection. Springer Berlin Heidelberg. pp. 233–247. arXiv:1103.5950. doi:10.1007/978-3-642-21329-8_15. ISBN 9783642213281.
S2CID 119215115.
27. ^ van Frassen, Bas (1989). Laws and Symmetry. Oxford University Press. ISBN 0-19-824860-1.
28. ^ Wald, Abraham (1950). Statistical Decision Functions. Wiley.
29. ^ Bernardo, José M.; Smith, Adrian F.M. (1994). Bayesian Theory.
John Wiley. ISBN 0-471-92416-4.
30. ^ Pfanzagl (1967, 1968)
31. ^ Morgenstern (1976, page 65)
32. ^ Galavotti, Maria Carla (1 January 1989). “Anti-Realism in the Philosophy of Probability: Bruno de Finetti’s Subjectivism”. Erkenntnis. 31 (2/3):
239–261. doi:10.1007/bf01236565. JSTOR 20012239. S2CID 170802937.
33. ^ Jump up to:a b c Galavotti, Maria Carla (1 December 1991). “The notion of subjective probability in the work of Ramsey and de Finetti”. Theoria. 57 (3): 239–259. doi:10.1111/j.1755-2567.1991.tb00839.x.
ISSN 1755-2567.
34. ^ Jump up to:a b Dokic, Jérôme; Engel, Pascal (2003). Frank Ramsey: Truth and Success. Routledge. ISBN 9781134445936.
35. ^ Davidson et al. (1957)
36. ^ Thornton, Stephen (7 August 2018). “Karl Popper”. Stanford Encyclopedia
of Philosophy. Metaphysics Research Lab, Stanford University.
37. ^ Popper, Karl (2002) [1959]. The Logic of Scientific Discovery (2nd ed.). Routledge. p. 57. ISBN 0-415-27843-0 – via Google Books. (translation of 1935 original, in German).
38. ^
Peirce & Jastrow (1885)
39. ^ Jump up to:a b Bernardo, J. M. (2005). “Reference Analysis”. In Dey, D.K.; Rao, C. R. (eds.). Handbook of Statistics (PDF). Vol. 25. Amsterdam: Elsevier. pp. 17–90. Archived (PDF) from the original on 2022-10-09.

Photo credit: https://www.flickr.com/photos/jolives/3171577977/’]