# Data Science

Data-driven models, inverse problems, uncertainty quantification

### Data-driven dimensionality reduction in probabilistic prediction

High-dimensional noisy time series associated with the underlying complex dynamics often contain redundant information and can be compactly represented by a dynamical process on a low-dimensional manifold; this is commonly referred to as the ‘manifold hypothesis’ and is related to the concentration of measure phenomenon in high-dimensional data sets. Due to the linear character of classical dimension reduction methods, such as the Principal Component Analysis, they are ill-suited to recover the nonlinear structure of the underlying state manifold. Robust and accurate dynamical predictions, based on reduced-order models extracted from empirical data, require a systematic understanding of how to combine manifold learning methods with analysis & probability techniques for extracting dynamical features from noisy or incomplete time-dependent data. This project will approach this problem from the probabilistic/stochastic viewpoint. For details please contact Michal Branicki (m.branicki@ed.ac.uk).

### Bayesian ensemble/mean field data assimilation for forecasting complex, high-dimensional dynamics

Forecasting (i.e., predicting the state) of a dynamical system associated with the physical reality is often hindered by inadequate knowledge of the underlying dynamics and/or its initial state. In many practical settings, such as in atmosphere-ocean prediction or autonomous navigation, this is compensated by access to some empirical data obtained from a near real-time measurements of the true dynamics that can be used improve the dynamical model and the subsequent state estimates. However, high dimensionality of the state space, partial and noisy observations, and so-called model error pose critical challenges to DA in applications. Fundamental problems of stability, accuracy and well-posedness of DA, particularly for infinite-dimensional dynamical systems governed by PDEs, continue to pose serious challenges to rigorous analysis of DA schemes, and are far from being resolved. This project will focus on selected issues related to Bayesian DA approaches in high-dimensional dynamical systems based on interacting ensembles of predictions which used in applications but whose performance is theoretically challenging yet important to understand. For details please contact Michal Branicki (m.branicki@ed.ac.uk).

### Geometry of information flow & uncertainty quantification in deep neural networks

Reliable implementation of machine and deep learning techniques on neural networks requires a fundamental understanding of how and why these algorithms work, and it necessitates determining application-adapted network architecture to give robust predictions/estimates. Innovative network architectures and training algorithms for specific applications in artificial intelligence, autonomous adaptation, and learning continue to be developed at a fast pace. However, there is no rigorous framework which allows for a systematic understanding of the many successes of machine learning based on neural networks. Probability theory and information geometry furnish the analysis of neural networks with suitable tools that allow to uncover and analyse the hidden geometry of information flow within a given network architecture. This project will make inroads into understanding the learning capacity of deep neural network architectures, and quantifying uncertainty in the network output through the prism of graph-based and/or mean-field systems processing finite, uncertain, and incomplete information. For details please contact Michal Branicki (m.branicki@ed.ac.uk).

### Stability of deep learning algorithms

Many commentators are asking whether current artificial intelligence solutions are sufficiently robust, resilient, and trustworthy; and how such issues should be quantified and addressed. Numerical analysts have the tools to contribute to the debate. One apsect of interest is the common practice of using low precision floating point formats to speed up computation time. Here rounding error analysis, along with new developments in stochastic rounding, can add insights into the expected accuracy. More generally, it is known that deep learning algorithms are susceptible to adversarial attacks: these are deliberate, targeted perturbations to input data that have a dramatic effect on the output; for example, a traffic "Stop" sign on the roadside can be misinterpreted as a speed limit sign when minimal graffiti is added. More generally, malicious perturbations can be made to the weights and biases that parameterize a network. The vulnerability of systems to such interventions raises questions around security, privacy and ethics, and there has been a rapid escalation of attack and defence strategies. Questions of particular interest to mathematicians include: Under realistic assumptions, do adversarial examples always exist with high probability? If so, can they always be computed? Is it easier to construct successful attacks when the data space is high dimensional? What is the trade-off between stability and accuracy in a deep neural network? Informal enquiries can be made to Des Higham (d.j.higham@ed.ac.uk).

### Data-driven models in molecular dynamics

Multiscale modelling plays an essential role in molecular simulation as the range of scales involved precludes the use of a single, unified system of equations. The most accurate model is quantum mechanics which describes the evolution of a system of nuclei and electrons. When a modest-sized quantum system is discretized for numerical solution, there results an unimaginably large number of equations which can swamp even the most powerful computer systems. A classical model based on potential energy functions for the interaction of atomic nuclei provides a much simplified description, but one that precludes many important effects (breakage of bonds, quantum tunnelling, etc.). Even the classical description must be further 'coarse-grained' to provide an effective scheme for large scale or slow-developing processes that would otherwise remain inaccessible in computer simulation. In a multiscale model, different models are unified by the use of bridging algorithms, numerical and analytical averaging, and reliance on the principles of statistical mechanics. In this project, the goal is to use experimental data in place of simulation data to capture complex local processes and low-level interactions in a molecular system . A system is no longer viewed as being described by a single inter-molecular potential energy surface, but rather by a collection of surfaces which can be locally determined, on-the-fly, from tabulated data. The resulting procedures will engender methodological changes in order to retain statistical properties that are relevant for the simulator. This project has aspects of molecular dynamics, computational statistical mechanics and quantum mechanics. It further relates to machine learning and has applications in materials modelling. Informal enquiries can be made to Ben Leimkuhler (b.leimkuhler@ed.ac.uk).

### Techniques for Uncertainty Quantification

Data assimilation techniques can be used to combine a numerical model with observations -- the numerical model captures the physics of the problem, while the observations provide information about the real system. However observations have associated errors, and these errors lead to uncertainty in state estimates. This project will study the application of uncertainty quantification techniques to the study of geophysical fluid flows, including techniques based on algorithmic differentiation or Monte Carlo methods. Informal enquiries can be made to James Maddison (j.r.maddison@ed.ac.uk).

### Studying transient turbulence with convolutional neural networks and Koopman operator theory

Deep convolutional neural networks (CNNs) can be exceptional image classifiers. This ability to identify and extract patterns from complex, high-dimensional data means that they are also a powerful tool for analysing nonlinear partial differential equations (PDEs). For example, CNNs have been used recently to generate robust predictive models for chaotic dynamics and to design new numerical methods for solving the Navier-Stokes equations on very coarse grids. These advances rely on the fact that, given sufficient training data, CNNs develop an accurate internal representation of the inertial manifold of solutions to the governing PDE. This project will make use of these new methods along with modern dynamical systems theory (the Koopman operator) to study a variety of statistically steady and transient turbulent flows: In transient flows, the Koopman operator allows us to generalise ideas from statistically-steady dynamical systems and describe the evolution in terms of nonlinear coherent states (Koopman modes); in forced turbulence, CNNs can be employed to generate low-order models based on simple invariant solutions of the governing equations which capture the multiscale nature of the flow. The project can be very flexible within these areas – it could involve the development of both new theoretical tools (Koopman decompositions for self-similar dynamics) or focus on the neural network architectures needed to identify Koopman modes or multiscale simple invariant solutions. The student will have access to GPU computing facilities for network training and there will also be opportunity to interact with collaborators in Cambridge and Harvard depending on their interests. Please contact Jacob Page (jacob.page@ed.ac.uk) for informal enquiries.

### PDE-constrained optimization in scientific processes

A vast number of important and challenging applications in mathematics and engineering are governed by inverse problems. One crucial class of these problems, which has significant applicability to real-world processes, including those of fluid flow, chemical and biological mechanisms, medical imaging, and others, is that of PDE-constrained optimization. However, whereas such problems can typically be written in a precise form, generating accurate numerical solutions on the discrete level is a highly non-trivial task, due to the dimension and complexity of the matrix systems involved. In order to tackle practical problems, it is essential to devise strategies for storing and working with systems of huge dimensions, which result from fine discretizations of the PDEs in space and time variables. In this project, "all-at-once" solvers coupled with appropriate preconditioning techniques will be derived for these systems, in such a way that one may achieve fast and robust convergence in theory and in practice. Informal enquiries can be made to John Pearson (j.pearson@ed.ac.uk). This project is related to the EPSRC Fellowship http://gow.epsrc.ac.uk/NGBOViewGrant.aspx?GrantRef=EP/M018857/1.

### Numerical analysis of Bayesian inverse problems

In areas as diverse as climate modelling, geosciences and medicine, mathematical models and computer simulations are routinely used to inform decisions and assess risk. However, the parameters appearing in the mathematical models are often unknown, and have to be estimated from measurements. This project is concerned with the inverse problem of determining the unknown parameters in the model, given some measurements of the output of the model. In the Bayesian framework, the solution to this inverse problem is the probability distribution of the unknown parameters, conditioned on the observed outputs. Combining ideas from numerical analysis, statistics and stochastic analysis, this project will address questions related to the error introduced in the distribution of the parameters, when the mathematical model is approximated by a numerical method. Informal enquiries can be made to Aretha Teckentrup (A.Teckentrup@ed.ac.uk).

### Optimal construction of statistical interpolants

Many problems in science and engineering involve an unknown complex process, which it is not possible to observe fully and accurately. The goal is then to reconstruct the unknown process, given a small number of direct or indirect observations. Mathematically, this problem can be reformulated as reconstructing a function from limited information available, such as a small number of function evaluations. Statistical approaches, such as interpolation or regression using Gaussian processes, provide us with a best guess of the unknown function, as well as a measure of how confident we are in our reconstruction. Combining ideas from machine learning, numerical analysis and statistics, this project will address questions related to optimal reconstructions, such as the optimal choice of the location of the function evaluations used for the reconstruction. Informal enquiries can be made to Aretha Teckentrup (A.Teckentrup@ed.ac.uk).

### Sampling methods in uncertainty quantification

For details on the range of potential topics in this area please contact Aretha Teckentrup (A.Teckentrup@ed.ac.uk).

### Stochastic differential equations, sampling and big data

For details on the range of potential topics in this area please contact Konstantinos Zygalakis (kzygalakis@ed.ac.uk)