Menu Close

Image and Video Processing: From Mars to Hollywood with a Stop at the Hospital from Pro. Guillermo Sapiro

Peng Liu

This is a basic image process course and very interesting to me. I go through most of the chapters, including Introduction to image and video processing, Spatial processing, Image restoration, Segmentation, Sparse modeling, and compressed sensing, and Medical Imaging. I skip three lectures: Image and video compression, Geometric PDEs, and Image and video inpainting. https://www.coursera.org/learn/image-processing/home/welcome

The most attractive topics are using image processing to discover the HIV cell structure and enhance deep brain simulation.

In addition, Hough transform is a very basic and powerful algorithm for image segmentation and discovering structures.

Sparse modeling and compressed sensing is another really important lecture I have to dig deeper later on.

Read more →

AmbientGAN: Generative models from lossy measurements

Peng Liu

Motivation:

How can we train our generative models with partial, noisy observations”

Why do we care?

In many settings, it is expensive or even impossible to obtain fully-observed samples, but economical to obtain partial, noisy samples.

Proposes in this paper:

  • AmbientGAN: train the discriminator not on the raw day domain but on the measurement domain
  • Propose the way to train the generative model with a noisy, corrupted, or missing data without any clean images
  • Prove that it is theoretically possible to recover the original true data distribution even though the measurement process is not invertible

MODEL ARCHITECTURE

 

 

 

 

Generative Adversarial Networks

 

Limitation of GAN

  • Require Good (or fully observed ) training samples

Related work:

  • Compressed sensing attempts to address this problem by exploiting the models of the data structure, sparsity
  • Bora et al. ICML 2017 “Compressed Sensing using Generative Models”
  • Compressed sensing is a very promising study and can give amazing results (need to go deeper)

Chicken and Egg

  • they have proposed it is possible to solve the problem with a small number of measurements by using Generative models
  • what if it is even not possible to gather the good data in the first place?
  • How can we collect enough data to train a generative model to start with?

 

Measurements?

 

 

 

 

 

 

 

 

 

 

 

 

Results

 

Conclusion

  • it is possible to train the generator without fully-observed data
  • In theory, it is possible to find the true distribution by training the generator when the measurement process is invertible and differentiable
  • Empirically, it is possible to recover the good data distribution even though the measurement process is not clearly known.

Possible Applications?

  • OCR
  • Gayoung’s webtoon data
  • Adding Reconstructionloss and Cyclic loss
  • Learnable f(.) by FC
  • etc.

Qestions to consider:

  • Cycle-GAN v.s Ambient-GAN?

Source: https://www.slideshare.net/thinkingfactory/introduction-to-ambient-gan

 

 

Read more →

Benchmarking Neural Network Robustness To Common Corruptions and Perturbations

Peng Liu

Paper: https://openreview.net/pdf?id=HJz6tiCqYm

Summary

This paper observes that a major flaw in common image-classification networks is their lack of robustness to common corruptions and perturbations. The authors develop and publish two variants of the ImageNet validation dataset, one for corruptions and one for perturbations. They then propose metrics for evaluating several common networks on their new datasets and find that robustness has not improved much from AlexNet to ResNet. They do, however, find several ways to improve performance including using larger networks, using ResNeXt, and using adversarial logit pairing.


Quality: The datasets and metrics are very thoroughly treated, and are the key contribution of the paper.

Some questions: What happens if you combine ResNeXt with ALP or histogram equalization? Or any other combinations? Is ALP equally beneficial across all networks? Are there other useful adversarial defenses?

Clarity: The novel validation sets and reasoning for them are well-explained, as are the evaluation metrics. Some explanation of adversarial logit pairing would be welcome, and some intuition (or speculation) as to why it is so effective at improving robustness.

Originality: Although adversarial robustness is a relatively popular subject, I am not aware of any other work presenting datasets of corrupted/perturbed images.

Significance: The paper highlights a significant weakness in many image-classification networks, provides a benchmark, and identifies ways to improve robustness. It would be improved by more thorough testing, but that is less important than the dataset, metrics and basic benchmarking provided.

Question: Why do authors do not recommend training on the new datasets?

Read more →