( 참고 : 패스트 캠퍼스 , 한번에 끝내는 컴퓨터비전 초격차 패키지 )
Data Augmentation (2)
Categories of Data Augmentation
- (1) Rule-based
- (2) GAN-based
- (3) AutoML-based
(2) GAN-based
for more natural data augmentation
- ex) medical image augmentation
- ex) SinGAN : Single Image GAN
- ex) GANGealing : GAN-Supervised Alignment
a) Medical Image Augmentation
GAN-based Synthetic Medical Image Augmentation for increased CNN Performance in Liver Lesion Classification, Frid-Adar et al., NIPS 2017
- evaluate based on classification ( 3-class )
b) SinGAN
SinGAN : Learning a Generative Model from a Single Natural Image, Shaham et al., ICCV 2019, best paper award
-
UNconditional GAN with single Image
-
use multi-scale pyramid & Fully-convolutional GANs
- GLOBAL & LOCAL information with pyramid
-
manipulation task
-
result :
- generator : small capacity ( to not memorize the training data )
- ( 3x3 conv - BN - LeakyReLU ) x 5
- Loss Function : Adversarial Loss + Reconstruction Loss
c) GANGealing
GAN-Supervised Dense Visual Alignment, Peebles et al., CVPR 2022
- Supervision with images generated by GAN
- architecture :
- (1) pre-trained GAN ( LEFT )
- (2) pre-trained conditional GAN ( RIGHT )
- (3) STN (Spatial Transformer) \(\rightarrow\) update this one!
- Step 1) generate images with (1) GAN
- Step 2) generate images with (2) GAN, given certain condition \(c\)
- Step 3) Align images from Step 1), using STN
- Step 4) Loss ( images from step2, images from step 3 )
Cons :
- only applicable in class-specific GAN
- unapplicable when high pose variation