Måns Thulin: When a 95% confidence interval really is a 1% interval – Challenges (and solutions) when analyzing discrete data

When analyzing discrete data, such as data from the binomial and Poisson distributions or contingency tables, we are faced with many challenges that don't arise for continuous data. In this talk, I will explain what these challenges are and how the discreteness of the underlying distributions cause confidence intervals and hypothesis tests to behave in undesirable ways. We will see examples where a 95% confidence interval turns out to be a 1% confidence interval, situations where an alternative definition of the p-value yields better-than-optimal test, a setting where a 94% interval is wider than the corresponding 95% interval and finally discuss how we can get rid of most of these problems using coin-tossing. If time permits, we will also have a look at some connections between frequentist, Bayesian and fiducial confidence intervals for binomial proportions.