Smug Bayesians

[oharar]

You mention the problems with estimating CIs in regularised models. Of course, we Bayesians can be smug about this, and point to MCMC. An exponential prior is equivalent to a LASSO, but we can use other priors too. The PC priors approach (https://arxiv.org/abs/1403.4630) is an extension of this idea.

[BertvanderVeen]

It is straightforward to show with simulations that there are cases that the CI in regularised model seem to provide nominal coverage. The trick would be in determining when that happens? Would that be possible based on the unpenalized effect size and magnitude of the tuning parameter?

[bbolker]

Continuing in the Bayes/freq crossover tradition: we know that Bayesian models are well-calibrated, by definition; that is, if we draw parameters from the prior (and the sampling/estimation method isn't broken), then we will get exact coverage. It's a little slippery to define this properly in frequentist-world, but my take has always been that if the form of the penalization implicitly matches the effect size spectrum then things will work well.

Talts, Sean, Michael Betancourt, Daniel Simpson, Aki Vehtari, and Andrew Gelman. “Validating Bayesian Inference Algorithms with Simulation-Based Calibration.” ArXiv:1804.06788 [Stat], October 21, 2020. http://arxiv.org/abs/1804.06788.

[BertvanderVeen]

It might be worth having a look at Wood (2006) on this: https://onlinelibrary.wiley.com/doi/full/10.1111/j.1467-842X.2006.00450.x. Mgcv by default makes use of this 'Bayesian estimated covariance matrix of the parameter estimators'. See also https://onlinelibrary.wiley.com/doi/full/10.1111/j.1467-9469.2011.00760.x.