SeqDAIS

Fri, 01 Jun 2018 00:00:00 +0000

The capacity of supervised learning systems to generalize to new examples is inherently limited by the collected data. If it is not an accurate reﬂection of the population, then the system will not perform well. In particular, if the collected data is biased, which means that one observes certain examples more often than normal, then the system can be misled into thinking that certain outcomes are also more likely to occur. Although the system might make accurate diagnoses for new patients arriving to that particular hospital, it will make inaccurate diagnoses for patients arriving to 1a diﬀerent hospital. The diﬀerences between patient populations might be due to regional diﬀerences such as exercise culture, but the fact that one observes for instance older patients more than normal means that data collected from that hospital is biased with respect to the total human population.

Handling biased samples from populations is a problem that has long been studied in statistics and econometrics. Although a number of techniques have been proposed to improve supervised learning systems trained on biased data, things have changed with the tremendous increase in computational power in the last 20 years. The ﬁeld has advanced to the point where we ask the question whether it is possible to generalize to particular target populations as well. Can we adapt a supervised learning system trained on adult human heart disease patients to make accurate decisions for infant heart disease patients?

Work from the last 10 years has looked at incorporating unlabeled data from these target populations. With this additional information, systems can recognize changes in data properties, ﬁnd correspondences between populations and adapt their decisions accordingly. Successful adaptation is deﬁned as an improvement over the performance of the original system. Nonetheless, the analysis of this problem is not complete, and it is not clear which conditions have to be fulﬁlled in order for the system to perform well. It seems that in cases where it is diﬃcult to describe how two populations relate to each other, adaptive systems suﬀer from high variability. They are highly uncertain about their decisions and often wrong. However, it seems that the more similar the populations are, the likelier it is that the system adapts well. It would, for example, be easier for the system to adapt to predict heart disease in adolescents based on data from adults, then it would for the system to adapt to infants. However, here might lie a potentially crucial insight: can we design a system that ﬁrst adapts to the closest population and only then adapts to the ﬁnal target population? In other words, a system that sequentially adapts?

This position is supported by a Niels Stensen Fellowship grant, offered by the Niels Stensen Stichting.

DAPR

Mon, 01 Sep 2014 00:00:00 +0000

Artificial intelligence, in particular machine learning and pattern recognition, is concerned with teaching computer systems to perform tasks. Tasks such as autonomous driving, recognizing tumors in medical images, or detecting suspicious packages in airports. Such systems learn by observing examples, i.e. data, and forming a mathematical description of what types of variations occur, i.e. a statistical model. For new input, the system computes the most likely output and makes a decision accordingly. As a scientific field, it is situated between statistics and and algorithmics. As a technology, it has become a very powerful tool due to the massive amounts of data being collected and the drop in the cost of computation.

However, obtaining enough data is still very difficult. There are often substantial financial, operational or ethical considerations in collecting data. The majority of research in machine learning deals with constraints on the amount, the labeling and the types of data that are available. One such constraint is that it is only possible to collect labeled data from one population, or domain, but the goal is to make decisions for another domain. It is unclear under which conditions this will be possible, which inspires the research question: when and how can a classification algorithm generalize from a source domain to a target domain?

My research has looked at different approaches to domain adaptation. Firstly, we have asked some critical questions on whether the standard approaches to model validation still hold in the context of different domains. As a result, we have proposed a means to reduce uncertainty in the validation risk estimator, but that does not solve the problem completely. Secondly, we modeled the transfer from source to target domain using parametric families of distributions, which works well in simple contexts such as feature dropout at test time. Thirdly, we looked at a more practical problem: tissue classifiers trained on data from one MRI scanner degrade when applied to data from another scanner due to acquisition-based variations. We tackled this problem by learning a representation for which detrimental variations are minimized while maintaining tissue contrast. Finally, considering that many approaches fail in practice because their assumptions are not met, we designed a parameter estimator that never performs worse than the naive non-adaptive classifier.

Theory | Wouter M. Kouw

SeqDAIS

DAPR