Image Reconstruction by Domain-transform Manifold Learning

Peng Liu May 27, 2018

Abstract

Image reconstruction is essential for imaging applications across the physical and life sciences, including optical and radar systems, magnetic resonance imaging, X-ray computed tomography, positron emission tomography, ultrasound imaging, and radio astronomy^1,2,3. During image acquisition, the sensor encodes an intermediate representation of an object in the sensor domain, which is subsequently reconstructed into an image by an inversion of the encoding function. Image reconstruction is challenging because analytic knowledge of the exact inverse transform may not exist a priori, especially in the presence of sensor non-idealities and noise. Thus, the standard reconstruction approach involves approximating the inverse function with multiple ad-hoc stages in a signal processing chain^4,5, the composition of which depends on the details of each acquisition strategy, and often requires expert parameter tuning to optimize reconstruction performance. Here we present a unified framework for image reconstruction—automated transform by manifold approximation (AUTOMAP)—which recasts image reconstruction as a data-driven supervised learning task that allows a mapping between the sensor and the image domain to emerge from an appropriate corpus of training data. We implement AUTOMAP with a deep neural network and exhibit its flexibility in learning reconstruction transforms for various magnetic resonance imaging acquisition strategies, using the same network architecture and hyperparameters. We further demonstrate that manifold learning during training results in sparse representations of domain transforms along low-dimensional data manifolds, and observe superior immunity to noise and a reduction in reconstruction artifacts compared with conventional handcrafted reconstruction methods. In addition to improving the reconstruction performance of existing acquisition methodologies, we anticipate that AUTOMAP and other learned reconstruction approaches will accelerate the development of new acquisition strategies across imaging modalities.

https://arxiv.org/pdf/1704.08841.pdf

Authors: Bo Zhu1,2,3, Jeremiah Z. Liu, Bruce R. Rosen1,2, Matthew S. Rosen1,2,3*

What learned from this Nature paper:

This is the first work to use deep learning to reconstruct images from sensor-domain instead of image domain, which is the so-called conventional way.
The concern to this work is about the capability of generalization. The conventional methods typically are more general to any particular case than deep learning.
The question is how we can make a safety deep-learning based image reconstructor?
The second question is what kind of other image reconstruction applications can tolerate that learning-based models gradually improve their capability of generalization.

I may have some ideas in RocMind. Let’s keep thinking this work.

Transductive Unbiased Embedding for Zero-Shot Learning

Peng Liu May 27, 2018

Abstract

Most existing Zero-Shot Learning (ZSL) methods have the strong bias problem, in which instances of unseen (target) classes tend to be categorized as one of the seen (source) classes. So they yield poor performance after being deployed in the generalized ZSL settings. In this paper, we propose a straightforward yet effective method named Quasi-Fully Supervised Learning (QFSL) to alleviate the bias problem. Our method follows the way of transductive learning, which assumes that both the labeled source images and unlabeled target images are available for training. In the semantic embedding space, the labeled source images are mapped to several fixed points specified by the source categories, and the unlabeled target images are forced to be mapped to other points specified by the target categories. Experiments conducted on AwA2, CUB and SUN datasets demonstrate that our method outperforms existing state-ofthe-art approaches by a huge margin of 9.3 ∼ 24.5% following generalized ZSL settings, and by a large margin of 0.2 ∼ 16.2% following conventional ZSL settings.

https://arxiv.org/pdf/1803.11320.pdf

Jie Song1 , Chengchao Shen1 , Yezhou Yang2 , Yang Liu3 , and Mingli Song1 1College of Computer Science and Technology, Zhejiang University, Hangzhou, China 2Arizona State University, Tempe, USA 3Alibaba Group, Hangzhou, China

What learned from this work:

I am a new guy to this topic, but I am preparing myself to get into it now.

The key point of this work is to propose a sort of new sub-topic named Quasi-Fully Supervised Learning, QFSL. Most previous works are constrained in the Conventional settings, which assumes that all test images are from target classes (unlabeled), however, this work changes this assumption to be Generalized settings, which is test images should be from both target and source classes.
One concern or thought is what kind of answers it would give when most test images are not belonging to any class in both target and source. Could the issue collapse this model? How can we have the model to learn how to learn?

The following points only belong to me: