Preface

Statistical analysis and machine learning have often differed in emphasis: the former has traditionally sought to understand data, while the latter has placed greater emphasis on prediction. Understanding data is a worthwhile goal in its own right. However, predictions derive their value from the decisions they inform—a point that is easily lost on the machine learning community, with its focus on benchmarks measuring predictive performance. Indeed, a weather forecast of rain tomorrow, for example, is useful only insofar as it helps us decide whether to carry an umbrella or cancel a picnic.

Once we put decisions front and centre, it becomes natural to ask whether they are rational and, ideally, optimal. Bayesian decision theory provides a principled answer to this question. From this perspective, Bayesian deep learning (BDL) is not primarily about approximating posterior distributions over model parameters, or even posterior predictive distributions. These are means to a more important end: approximating Bayes-optimal decisions—the decisions that an ideal Bayesian agent would make. Deep neural networks are among the most powerful and flexible models we have discovered so far, and using them to approximate such decisions therefore seems to be one of our best available avenues.

One might argue that a sufficiently capable language model, pretrained on the entire internet, will eventually learn to make Bayes-optimal decisions implicitly. Perhaps it will, but current systems do not yet give us sufficient reason to rely on this alone. If we want control over these approximations, and insight into their quality, the most direct route is to perform Bayesian inference within neural network models and draw on the advances made by Bayesian statistics over the past two centuries. This is the basis for BDL, and what this book is about.

While parts of the book develop a principal route through the landscape of Bayesian methods, other chapters cover foundational and mathematical aspects. Scalability asks how Bayesian reasoning can survive the size of contemporary models and datasets. The applications of the book demonstrate how widely these points reach, and topical developments showcase possible future routes BDL approaches may take. This book is consequently both a technical reference and an invitation to view deep learning through a wider lens. Our hope is that readers come away seeing BDL not as a specialised corner of machine learning, but as a framework for building powerful models while remaining explicit about evidence, uncertainty, and the decisions that give predictions their purpose.

Before presenting the content of the book, the topics it covers, and the people who contributed to it, we would like to emphasise that this book is not intended as a comprehensive survey of the field. Many important subjects receive only limited attention or are not covered at all, such as predictive Bayes or connections between Bayesian inference and optimisation duality. Likewise, many researchers whose work has shaped Bayesian deep learning are not represented among the authors. The book should therefore rather be understood as a snapshot of a broad and rapidly developing field at a particular moment in time. Its selection of topics and contributors reflects the scope and circumstances of this collaborative project, and is therefore necessarily biased. Hence, it should not be interpreted as a judgement about the relative importance of these topics. We still hope, nonetheless, that the book will serve as a useful reference for researchers, educators, and practitioners, as well as provide a foundation for future work in Bayesian deep learning.