EPSRC Institutional Sponsorship Collaborative Event

Workshop on Inverse Problems

17 March 2016

International Centre for Mathematical Sciences, 15 South College Street, Edinburgh, EH8 9AA

The aim is collaborative work for the identification of problems in the application areas which
would benefit from advanced statistical analysis and modelling in order to obtain new insights.

The objectives are the encouragement of creative thinking, the initial development of projects around the challenges for the research areas and the creation of internal research days in the application areas in order to assist with the development of collaborative links across the university, other TATI partners, other HEIs and to industry.

Expected outcome: development of collaborative interdisciplinary projects; particularly those that may potentially feed into the Alan Turing Institute programmes.

Statistical inverse problem refers to inference for an unknown object (e.g. a function or an image) which is observed indirectly, and Bayesian modelling is predominantly used to solve it as a principled way to incorporate available a priori information. Mathematical and statistical challenges are to create efficient and reliable computational methods as well as to specify a family of prior distributions that leads to the smallest possible error of the solution. Dynamical linear and nonlinear inverse problems are a novel challenge in statistics and require novel techniques, both modelling and computational.

The workshop attendance is free but registration is required for catering purposes. If you would like to come to the meeting: please register here.

Programme

9:15 -9:20 - Introduction

9:20-10:05 - Felix Lucka (UCL) Recent Advances in Bayesian Inference for Biomedical Imaging

10:05-10:20 - discussion

10:20-11:05 - Mikael Kuusela (EPFL, Switzerland) Shape-constrained uncertainty quantification in unfolding elementary particle spectra at the Large Hadron Collider

11:05-11:20 - discussion

11:20-11:45 - coffee break

11:45-12:45 - Eduard Kontar (University of Glasgow) Inverse problems in solar physics: from observables to physical parameters

12:45-1:00pm - discussion

1:00-2:00pm - Lunch

2:00-3:00pm - Yves Rozenholc (Universite Paris Descartes, France) Estimation in the Laplace convolution model and application to tumoral blood flow quantification

3:00-3:15pm - discussion

3:15-4:00pm - Anna Simoni (CREST, France) Adaptive Bayesian estimation in indirect Gaussian sequence space models

4:00-4:15pm - discussion

4:15-4:30pm - coffee break

4:30-5:00pm - Pol Moreno (University of Edinburgh) Vision-as-inverse-graphics for detailed scene understanding

5:00-5:15pm - discussion and closing

ABSTRACTS

Felix Lucka (UCL) Recent Advances in Bayesian Inference for Biomedical Imaging

In the first part of the talk, we give an overview on Bayesian inference as a framework for solving ill-posed inverse problems. Its increasing popularity will be illustrated by highlighting several recent developments such as posterior uncertainty quantification, infinite dimensional Bayesian inversion, sparsity priors, dynamic Bayesian inversion and new posterior sampling techniques. Motivated by biomedical imaging applications, we then focus on Bayesian inference for linear inverse problems with sparsity-promoting prior distributions. We first present a collection of own contributions for priors with convex, but non-smooth energies, including the development of efficient Markov chain Monte Carlo (MCMC) samplers, new theoretical insights into the relationship between MAP and CM estimates and computational results such as the inversion of experimental computed tomography (CT) data with total variation (TV) and Besov space priors. Then, we discuss hierarchical Bayesian modeling leading to priors with non-convex energies and their application to EEG/MEG source localization. Finally, we close by some general comments on Bayesian inversion.

Mikael Kuusela (EPFL, Switzerland) Shape-constrained uncertainty quantification in unfolding elementary particle spectra at the Large Hadron Collider

The high energy physics unfolding problem is an important statistical inverse problem in data analysis at the Large Hadron Collider (LHC) at CERN. The goal of unfolding is to make nonparametric inferences about a particle spectrum from measurements smeared by the finite resolution of the particle detectors. Previous unfolding methods use ad hoc discretization and regularization, resulting in confidence intervals that can have significantly lower coverage than expected. Instead of regularizing using a roughness penalty or early stopping, we impose physically justified shape constraints: positivity, monotonicity, and convexity. We quantify the uncertainty by constructing a nonparametric confidence set for the true spectrum consisting of all spectra that satisfy the shape constraints and predict observations within an appropriately calibrated level of fit to the data. Projecting that set produces simultaneous confidence intervals for all functionals of the spectrum, including averages within bins. The confidence intervals have guaranteed frequentist finite-sample coverage in the important and challenging class of unfolding problems for steeply falling particle spectra. We demonstrate that the method is effective using simulations that mimic unfolding the inclusive jet transverse momentum spectrum at the LHC. The shape-constrained intervals provide usefully tight conservative confidence intervals, while the conventional methods suffer from severe undercoverage. Joint work with Philip B. Stark (UC Berkeley).

Yves Rozenholc (Universite Paris Descartes, France) Estimation in the Laplace convolution model and application to tumoral blood flow quantification

I will introduce briefly how blood flow quantification related to Laplace convolution model may lead to tumoral biomarkers for personalization of treatment and follow up. Then I will present some results on Laplace deconvolution that we have developed with Marianna Pensky to tackle this problem from a statistical point of view.

Anna Simoni (CREST, France) Adaptive Bayesian estimation in indirect Gaussian sequence space models

In an indirect Gaussian sequence space model lower and upper bounds are derived for the concentration rate of the posterior distribution of the parameter of interest shrinking to the parameter value \theta_0 that generates the data. While this establishes posterior consistency, however, the concentration rate depends on both \theta_0 and a tuning parameter which enters the prior distribution. We first provide an oracle optimal choice of the tuning parameter, i.e., optimized for each \theta_0 separately. The optimal choice of the prior distribution allows us to derive an oracle optimal concentration rate of the associated posterior distribution. Moreover, for a given class of parameters and a suitable choice of the tuning parameter, we show that the resulting uniform concentration rate over the given class is optimal in a minimax sense. Finally, we construct a hierarchical prior that is adaptive. This means that, given a parameter \theta_0 or a class of parameters, respectively, the posterior distribution contracts at the oracle rate or at the minimax rate over the class. Notably, the hierarchical prior does not depend neither on \theta_0 nor on the given class. Moreover, convergence of the fully data-driven Bayes estimator at the oracle or at the minimax rate is established. Joint work with Jan Johannes and Rudolf Schenk.

Pol Moreno (University of Edinburgh) Vision-as-inverse-graphics for detailed scene understanding

A long-standing view of computer vision is that it is the inverse of a computer graphics problem. That is, the goal of computer vision is to infer the objects present in a scene, their positions and poses, the illuminant etc. In the language of machine learning, the object identities, poses, illuminant etc are latent variables which must be inferred in order to understand the scene.

We have developed a stochastic scene generator and used it to render these scenes to produce images. We then train a recognition model to infer the relevant latent variables. The great advantage of using synthetic data is that there is ready access to the relevant latent variables, and that large quantities of data can be easily generated for training the recognition models. Joint work with Chris Williams.

----------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Workshop on Inverse Problems

International Centre for Mathematical Sciences, 15 South College Street, Edinburgh

Two half-day workshop, 30 November and 1 December 2015

Programme

Monday 30 November

1:30-1:55pm - arrival

1:55pm Welcome

2:00-3:00pm - Mike Christie (Petroleum Institute, Heriot Watt University) Inverse Problems in Reservoir Simulation

3:00-3:15 - discussion

3:15-3:45pm - coffee break

3:45-4:45pm - Martin Benning (University of Cambridge) Inversion of non-linear image formation models in MRI

4:45-5pm - discussion

5:00-5:30pm - Short talks:

Dimitris Kamilis (School of Engineering, UoE) Bayesian estimation in electromagnetic imaging

Natalia Bochkina (School of Mathematics, UoE) Nonregular Bayesian inverse problems

Jenovah Rodrigues (School of Mathematics, UoE) Bayesian inverse problems and Computerised tomography

5:30-6pm - discussion

Tuesday 1 December

1 -1:30pm - lunch

1:30 -2:30pm - Jacek Gondzio (School of Mathematics, UoE) Using second-order information in big data optimization

2:30-2:45 - discussion

2:45-3:30pm - Richard Essery (School of GeoSciences, UoE) Inverse problems in snow physics

3:30-4:00pm - coffee break

4:00 -5:00pm - Subramanian Ramamoorthy (School of Informatics, UoE) Activity forecasting and estimation as inverse problems

5:00-5:30pm - Short talks

Jonathan Gair (BioSS and School of Mathematics, UoE) Challenges in inference for gravitational wave experiments

Murat Uney (School of Engineering, UoE) Multiple input multiple output (MIMO) sensing in dynamic environments

Ian Main (School of GeoSciences, UoE) How big can an earthquake be?

5:30 - 6:00pm - discussion

Abstracts

Martin Benning (University of Cambridge)

Inversion of non-linear image formation models in MRI

We are presenting a modification of a preconditioned variant of the well-known Alternating Direction Method of Multipliers (ADMM) algorithm that aims at solving convex optimisation problems with nonlinear operator constraints. We further show equivalence under specific circumstances to the recently proposed Nonlinear Primal-Dual Hybrid Gradient (NL-PDHGM) method with extrapolation step on the dual instead of the primal variable. Subsequently, the algorithm is demonstrated to handle nonlinear inverse problems arising in industrial and medical imaging, namely velocity-encoded and parallel Magnetic Resonance Imaging (MRI).

Mike Christie (Petroleum Institute, Heriot Watt University)

Inverse Problems in Reservoir Simulation

This talk will look at the inverse problems in the oil industry and how modern stochastic sampling and MCMC techniques can be deployed to obtain effective uncertainty quantification for reservoir simulation.

The uncertainties in reservoir modelling arise principally from our lack of knowledge of the subsurface properties. Data is typically known precisely only at limited numbers of spatial locations, and can maybe inferred through analysis of seismic responses more widely.

The codes used to model flow in oil reservoirs are often large commercial black box codes, where the user has no access to the code internals and is therefore unable to obtain gradients. This can limit the use of some of the more modern and efficient MCMC algorithms. In the talk we will show examples of model calibration and posterior sampling for real reservoir models, using techniques such as Hamiltonian Monte Carlo, population MCMC, and Multi-Level HMC.

Richard Essery (School of GeoSciences, UoE)

Inverse problems in snow physics

Snow on the ground is a complex porous medium with properties that can vary greatly in space and time. In addition to the bulk properties of water in solid, liquid and vapour phases, the bicontinuous structures of the ice lattice and the pore space have important influences on the thermodynamic, hydraulic, radiative, mechanical, electrical and chemical properties of snow. Knowledge of these properties are required for applications including predicting climate feedbacks, permafrost stability, water resources, avalanche and flood risk, and surface processing of pollutants in Arctic haze. This talk will review emerging methods that invert models of electrical potential generation and microwave radiation scattering in snow to obtain estimates of water fluxes and storage from indirect measurements.

Jacek Gondzio (joint work K. Fountoulakis) (School of Mathematics, UoE)

Using second-order information in big data optimization

We address sparse approximation problems which arise frequently in statistics, machine learning and signal/image reconstruction. Although many very simple instances of such problems are solved using the first-order methods, the less trivial instances defy algorithms which use only the first-order information. In this talk we argue that one should use an approximate second-order information when solving the underlying optimization problems. We develop an inexact Newton method and boost its convergence by applying appropriate preconditioning techniques which exploit two particular features of such problems:

(i) sparsity of the solution, and

(ii) near-orthogonality of the matrices involved.

The latter originates from the restricted isometry properties frequently assumed in signal/image processing problems. Spectral analysis of the preconditioners and their practical efficiency when solving linear systems in the Newton Conjugate Gradient method will be presented. Numerical results of solving L1-regularization problems of unprecedented sizes will be presented.

[1] Fountoulakis and Gondzio, Second-order Method for Strongly Convex $\ell_1$-regularization problems, Mathematical Programming (published on-line, February 2015). DOI: 10.1007/s10107-015-0875-4

[2] Fountoulakis and Gondzio, Performance of First- and Second-Order Methods for Big Data Optimization, Technical Report ERGO-15-005, March 2015. http://www.maths.ed.ac.uk/~gondzio/reports/trillion.html

Subramanian Ramamoorthy (School of Informatics, University of Edinburgh)

Activity forecasting and estimation as inverse problems

Many intelligent systems have the need to make predictions about the current and future actions of other agents, e.g., the human user. The problem is one of going from observed traces of a goal-directed behaviour to beliefs about the underlying plans and dynamics that generated these.

I will begin by describing the formulation of activity inference as an inverse planning problem, in which observers invert a probabilistic graphical model of goal-dependent plans to infer agents’ goals. While the basic formulation is attractive, and cognitively plausible, many realistic applications require more flexibility, including the ability to incorporate evidence drawn at multiple scales. I will outline work on combining topology-based clustering of trajectories with particle filter methods to achieve multiscale particle filters.

Time permitting, I will also discuss other problem domains and sources of data on which we are interested in applying these methods.