Bayesians and non-Bayesians, often called frequentists, seem to be perpetually at logger- heads on fundamental questions of statistical inference. This paper takes as agnostic a stand as is possible for a practising frequentist, and tries to elicit a Bayesian answer to questions of interest to frequentists. The argument is based on my presentation at a debate organised by the Rimini Centre for Economic Analysis, between me as the frequentist "advocate", and Christian Robert on the Bayesian side.Bayesian methods, bootstrap, Bahadur-Savage result

In this paper we discuss several aspects of simulation based Bayesian econometric inference. We start at an elementary level on basic concepts of Bayesian analysis; evaluating integrals by simulation methods is a crucial ingredient in Bayesian inference. Next, the most popular and well-known simulation techniques are discussed, the MetropolisHastings algorithm and Gibbs sampling (being the most popular Markov chain Monte Carlo methods) and importance sampling. After that, we discuss two recently developed sampling methods: adaptive radial based direction sampling [ARDS], which makes use of a transformation to radial coordinates, and neural network sampling, which makes use of a neural network approximation to the posterior distribution of interest. Both methods are especially useful in cases where the posterior distribution is not well-behaved, in the sense of having highly non-elliptical shapes. The simulation techniques are illustrated in several example models, such as a model for the real US GNP and models for binary data of a US recession indicator.

Relative risk (RR) is a preferred measure for investigating associations in clinical and epidemiological studies with dichotomous outcomes. However, if the outcome of interest is rare, it frequently occurs that no events are observed in one of the comparison groups. In this case, many of the standard methods used to obtain confidence intervals (CIs) for the RRs are not feasible, even in studies with strong statistical evidence of an association. Different strategies for solving this challenge have been suggested in the literature. This paper, which uses both mathematical arguments and statistical simulations, aims to present, compare, and discuss the different statistical approaches to obtain CIs for RRs in the case of no events in one of the comparison groups. Moreover, we compare these frequentist methods with Bayesian approaches to determine credibility intervals (CrIs) for the RRs. Our results indicate that most of the suggested approaches can be used to obtain CIs (or CrIs) for RRs in the case of no events, although one-sided intervals obtained by methods based on deliberate, probabilistic considerations should be preferred over ad hoc methods. In addition, we demonstrate that Bayesian approaches can be used to obtain CrIs in these situations. Thus, it is possible to obtain statistical inference for the RR, even in studies with no events in one of the comparison groups, and CIs for the RRs should always be provided. However, it is important to note that the obtained intervals are sensitive to the method chosen in the case of small sample sizes.

Elaboration of Bayesian phylogenetic inference methods has continued at pace in recent years with major new advances in nearly all aspects of the joint modelling of evolutionary data. It is increasingly appreciated that some evolutionary questions can only be adequately answered by combining evidence from multiple independent sources of data, including genome sequences, sampling dates, phenotypic data, radiocarbon dates, fossil occurrences, and biogeographic range information among others. Including all relevant data into a single joint model is very challenging both conceptually and computationally. Advanced computational software packages that allow robust development of compatible (sub-)models which can be composed into a full model hierarchy have played a key role in these developments. Developing such software frameworks is increasingly a major scientific activity in its own right, and comes with specific challenges, from practical software design, development and engineering challenges to statistical and conceptual modelling challenges. BEAST 2 is one such computational software platform, and was first announced over 4 years ago. Here we describe a series of major new developments in the BEAST 2 core platform and model hierarchy that have occurred since the first release of the software, culminating in the recent 2.5 release. Author summary Bayesian phylogenetic inference methods have undergone considerable development in recent years, and joint modelling of rich evolutionary data, including genomes, phenotypes and fossil occurrences is increasingly common. Advanced computational software packages that allow robust development of compatible (sub-)models which can be composed into a full model hierarchy have played a key role in these developments. Developing scientific software is increasingly crucial to advancement in many fields of biology. The challenges range from practical software development and engineering, distributed team coordination, conceptual development and statistical modelling, to validation and testing. BEAST 2 is one such computational software platform for phylogenetics, population genetics and phylodynamics, and was first announced over 4 years ago. Here we describe the full range of new tools and models available on the BEAST 2.5 platform, which expand joint evolutionary inference in many new directions, especially for joint inference over multiple data types, non-tree models and complex phylodynamics.

Surrogate models are statistical or conceptual approximations for more complex simulation models. In this context, it is crucial to propagate the uncertainty induced by limited simulation budget and surrogate approximation error to predictions, inference, and subsequent decision-relevant quantities. However, quantifying and then propagating the uncertainty of surrogates is usually limited to special analytic cases or is otherwise computationally very expensive. In this paper, we propose a framework enabling a scalable, Bayesian approach to surrogate modeling with thorough uncertainty quantification, propagation, and validation. Specifically, we present three methods for Bayesian inference with surrogate models given measurement data. This is a task where the propagation of surrogate uncertainty is especially relevant, because failing to account for it may lead to biased and/or overconfident estimates of the parameters of interest. We showcase our approach in three detailed case studies for linear and nonlinear real-world modeling scenarios. Uncertainty propagation in surrogate models enables more reliable and safe approximation of expensive simulators and will therefore be useful in various fields of applications.

Abstract Bootstrapping is a simulation-based method to provide statistical inference, or to measure accuracy, on estimators of interest, such as random field parameters, i.e., mean and variance of a physical quantity at each location within a random field, and auto-correlation coefficient of the quantities among any two different locations of the field. Statistical inference may be performed using analytical methods together with multiple sets of complete measurement data. However, measurement data may be incomplete sometimes, due to, for examples, sensor failure, storage capacity in collecting measurement data, or cost/difficulty in increasing measurement density. In such a case, it is challenging to perform statistical inference of auto-correlation coefficients, particularly between locations where no data are measured directly. This study aims to develop an innovative and robust bootstrap method to perform statistical inference on auto-correlation coefficient in one dimensional random fields, e.g., time series or variations of a quantity along a spatial direction such as depth, from multiple sets of incomplete and sparse measurements. The proposed approach is based on a Bayesian compressive sampling (BCS)-Karhunen–Loeve (KL) expansion random field generator, which has been used to generate random field samples from sparse measurement data. The proposed bootstrap method is illustrated and validated through both numerical examples and real measurement data. The results obtained from the proposed bootstrap method using multiples sets of incomplete measurement data are consistent with those obtained from analytical methods using multiple sets of complete measurement data.

The AEGISS (Ascertainment and Enhancement of Disease Surveillance and Statistics) project uses spatio-temporal statistical methods to identify anomalies in the incidence of gastrointestinal infections in the UK. The focus of this paper is the modelling of temporal variation in incidence using data from the Southampton area in southern England. We identified and fitted a hierarchical stochastic model for the time series of daily incident cases to enable probabilistic prediction of temporal variation in risk, and demonstrated the resulting gains in predictive accuracy by comparison with a conventional analysis based on an over-dispersed Poisson log-linear regression model. We used Bayesian methods of inference in order to incorporate parameter uncertainty in our predictive inference of risk. Incorporation of our model in the overall spatio-temporal model, will contribute to the accurate and timely prediction of unusually high food-poisoning incidence, and thus to the identification and prevention of future outbreaks

The Bayesian analytical framework is clinically intuitive, characterized by the incorporation of previous evidence into the analysis, and allowing an estimation treatment effects and their associated uncertainties. The application of Bayesian statistical inference is not new to the cardiovascular field, as illustrated by various recent randomized trials that applied a primary Bayesian analysis. Given the guideline-shaping character of trials, a thorough understanding of the concepts and technical details of Bayesian statistical methodology is of utmost importance to the modern practicing cardiovascular physician. Therefore, this Review aims to present a step-by-step guide to interpreting and performing a Bayesian (re-)analysis of cardiovascular clinical trials, while highlighting the main advantages of Bayesian inference for the clinical reader. After an introduction of the concepts of frequentist and Bayesian statistical inference and reasons to apply Bayesian methods, key steps for performing a Bayesian analysis are presented, including: the verification of the clinical appropriateness of the research question, the quality and completeness of the trial design, as well as the adequate elicitation of the prior (i.e., ones belief towards a certain treatment before the current evidence becomes available), identification of the likelihood, and their combination into a posterior distribution. Examination of this posterior distribution offers the possibility of not only determining the probability of treatment superiority, but also the probability of exceeding any chosen minimal clinically important difference. Multiple priors should be transparently prespecified, limiting post-hoc manipulations. Using this guide, three cardiovascular randomized controlled trials are re-analysed, demonstrating the clarity and versatility of Bayesian inference.

Federated learning faces huge challenges from model overfitting due to the lack of data and statistical diversity among clients. To address these challenges, this paper proposes a novel personalized federated learning method via Bayesian variational inference named pFedBayes. To alleviate the overfitting, weight uncertainty is introduced to neural networks for clients and the server. To achieve personalization, each client updates its local distribution parameters by balancing its construction error over private data and its KL divergence with global distribution from the server. Theoretical analysis gives an upper bound of averaged generalization error and illustrates that the convergence rate of the generalization error is minimax optimal up to a logarithmic factor. Experiments show that the proposed method outperforms other advanced personalized methods on personalized models, e.g., pFedBayes respectively outperforms other SOTA algorithms by 1.25%, 0.42% and 11.71% on MNIST, FMNIST and CIFAR-10 under non-i.i.d. limited data.

We review Bayesian inference for dynamic latent variable models using the data augmentation principle. We detail the difficulties of stimulating dynamic latent variables in a Gibbs sampler. We propose an alternative specification of the dynamic disequilibrium model which leads to a simple simulation procedure and renders Bayesian inference fully operational. Identification issues are discussed. We conduct a specification search using the posterior deviance criterion of Spiegelhalter, Best, Carlin, and van der Linde (2002) for a disequilibrium model of the Polish credit market.Latent variables, Disequilibrium models, Bayesian inference, Gibbs sampler, Credit rationing

We investigate solution methods for large-scale inverse problems governed by partial differential equations (PDEs) via Bayesian inference. The Bayesian framework provides a statistical setting to infer uncertain parameters from noisy measurements. To quantify posterior uncertainty, we adopt Markov Chain Monte Carlo (MCMC) approaches for generating samples. To increase the efficiency of these approaches in high-dimension, we make use of local information about gradient and Hessian of the target potential, also via Hamiltonian Monte Carlo (HMC). Our target application is inferring the field of soil permeability processing observations of pore pressure, using a nonlinear PDE poromechanics model for predicting pressure from permeability. We compare the performance of different sampling approaches in this and other settings. We also investigate the effect of dimensionality and non-gaussianity of distributions on the performance of different sampling methods.

Discrete Bayesian Networks (BNs) have been very successful as a framework both for inference and for expressing certain causal hypotheses. In this paper we present a class of graphical models called the chain event graph (CEG) models, that generalises the class of discrete BN models. This class is suited for representing conditional independence and sample space structures of asymmetric models. It retains many useful properties of discrete BNs, in particular admitting conjugate estimation. It provides a flexible and expressive framework for representing and analysing the implications of causal hypotheses, expressed in terms of the effects of a manipulation of the generating underlying system.We prove that, as for a BN, identifiability analyses of causal effects can be performed through examining the topology of the CEG graph, leading to theorems analogous to the Backdoor theorem for the BN

This work proposes a Bayesian inference method for the reduced-order modeling of time-dependent systems. Informed by the structure of the governing equations, the task of learning a reduced-order model from data is posed as a Bayesian inverse problem with Gaussian prior and likelihood. The resulting posterior distribution characterizes the operators defining the reduced-order model, hence the predictions subsequently issued by the reduced-order model are endowed with uncertainty. The statistical moments of these predictions are estimated via a Monte Carlo sampling of the posterior distribution. Since the reduced models are fast to solve, this sampling is computationally efficient. Furthermore, the proposed Bayesian framework provides a statistical interpretation of the regularization term that is present in the deterministic operator inference problem, and the empirical Bayes approach of maximum marginal likelihood suggests a selection algorithm for the regularization hyperparameters. The proposed method is demonstrated on two examples: the compressible Euler equations with noise-corrupted observations, and a single-injector combustion process. offline cost. To evaluate the probabilistic ROM in the online stage, draw samples from the posterior distribution of ˆ O and solve the corresponding ROM realizations, then calculate the statistics of these ROM solutions.

The CLAssification and Data Analysis Group (CLADAG) of the Italian Statistical Society recently organised a competition, the 'Young Researcher Data Mining Prize' sponsored by the SAS Institute. This paper was the winning entry and in it we detail our approach to the problem proposed and our results. The main methods used are linear regression, mixture models, Bayesian autoregressive and Bayesian dynamic models

The three main approaches in statistical inference – classical statistics, Bayesian and likelihood – are in current use in phylogeny research. The three approaches are discussed and compared, with particular emphasis on theoretical properties illustrated by simple thought-experiments. The methods are problematic on axiomatic grounds (classical statistics), extra-mathematical grounds relating to the use of a prior (Bayesian inference) or practical grounds (likelihood). This essay aims to increase understanding of these limits among those with an interest in phylogeny.Peer reviewe

The modelling of many real-world problems relies on computationally heavy simulations of randomly interacting individuals or agents. However, the values of the parameters that underlie the interactions between agents are typically poorly known, and hence they need to be inferred from macroscopic observations of the system. Since statistical inference rests on repeated simulations to sample the parameter space, the high computational expense of these simulations can become a stumbling block. In this paper, we compare two ways to mitigate this issue in a Bayesian setting through the use of machine learning methods: One approach is to construct lightweight surrogate models to substitute the simulations used in inference. Alternatively, one might altogether circumvent the need for Bayesian sampling schemes and directly estimate the posterior distribution. We focus on stochastic simulations that track autonomous agents and present two case studies: tumour growths and the spread of infectious diseases. We demonstrate that good accuracy in inference can be achieved with a relatively small number of simulations, making our machine learning approaches orders of magnitude faster than classical simulation-based methods that rely on sampling the parameter space. However, we find that while some methods generally produce more robust results than others, no algorithm offers a one-size-fits-all solution when attempting to infer model parameters from observations. Instead, one must choose the inference technique with the specific real-world application in mind. The stochastic nature of the considered real-world phenomena poses an additional challenge that can become insurmountable for some approaches. Overall, we find machine learning approaches that create direct inference machines to be promising for real-world applications. We present our findings as general guidelines for modelling practitioners.

Background Although null hypothesis significance testing (NHST) is the agreed gold standard in medical decision making and the most widespread inferential framework used in medical research, it has several drawbacks. Bayesian methods can complement or even replace frequentist NHST, but these methods have been underutilised mainly due to a lack of easy-to-use software. JASP is an open-source software for common operating systems, which has recently been developed to make Bayesian inference more accessible to researchers, including the most common tests, an intuitive graphical user interface and publication-ready output plots. This article provides a non-technical introduction to Bayesian hypothesis testing in JASP by comparing traditional tests and statistical methods with their Bayesian counterparts. Results The comparison shows the strengths and limitations of JASP for frequentist NHST and Bayesian inference. Specifically, Bayesian hypothesis testing via Bayes factors can complement and even replace NHST in most situations in JASP. While p -values can only reject the null hypothesis, the Bayes factor can state evidence for both the null and the alternative hypothesis, making confirmation of hypotheses possible. Also, effect sizes can be precisely estimated in the Bayesian paradigm via JASP. Conclusions Bayesian inference has not been widely used by now due to the dearth of accessible software. Medical decision making can be complemented by Bayesian hypothesis testing in JASP, providing richer information than single p -values and thus strengthening the credibility of an analysis. Through an easy point-and-click interface researchers used to other graphical statistical packages like SPSS can seemlessly transition to JASP and benefit from the listed advantages with only few limitations.

Polya trees fix partitions and use random probabilities in order to construct random probability measures. With quantile pyramids we instead fix probabilities and use random partitions. For nonparametric Bayesian inference we use a prior which supports piecewise linear quantile functions, based on the need to work with a finite set of partitions, yet we show that the limiting version of the prior exists. We also discuss and investigate an alternative model based on the so-called substitute likelihood, Both approaches factorize in a convenient way leading to relatively straightforward analysis via MCMC, since analytic summaries of posterior distributions are too complicated. We give conditions securing the existence of an absolute continuous quantile process, and discuss consistency and approximate normality for the sequence of posterior distributions. Illustrations are included