How to augment vision data

Data augmentation is important to train models with good generalization ability, especially when the size of your dataset is limited. FastAI.jl gives you high-level helpers to use data augmentation in vision learning tasks, but also allows directly using DataAugmentation.jl , the underlying data augmentation library.

By default, the only augmentation that will be used in computer vision tasks is a random crop, meaning that after images, keypoints and masks are resized to a similar size a random portion will be cropped during training. We can demonstrate this on the image classification task.


			
			
			
			using
			
			 

	
			FastAI
			,
			
			 
			FastVision
			,
			
			 
			FastMakie
			
			

			
			import
			
			 
			CairoMakie
			;
			 
			
			
			CairoMakie
			.
			
			activate!
			(
			
			type
			=
			
			"
			png
			"
			)
			

			

			
			
			data
			,
			 
			blocks
			 
			=
			 
			
			load
			(
			
			

	
			datarecipes
			(
			)
			[
			
			"
			imagenette2-160
			"
			]
			)
			

			
			task
			 
			=
			 
			

	
			BlockTask
			(
			
    
			blocks
			,
			
    
			
			(
			
        
			

	
			ProjectiveTransforms
			(
			
			(
			128
			,
			 
			128
			)
			)
			,
			
        
			

	
			ImagePreprocessing
			(
			)
			,
			
        
			

	
			OneHot
			(
			)
			
    
			)
			

			)
			

			
			
			xs
			,
			 
			ys
			 
			=
			 
			
			

	
			FastAI
			.
			

	
			makebatch
			(
			task
			,
			 
			data
			,
			 
			
			fill
			(
			4
			,
			 
			3
			)
			)
			

			

	
			showbatch
			(
			task
			,
			 
			
			(
			xs
			,
			 
			ys
			)
			)

			LoadError("string", 14, ArgumentError("The attribute `textsize` has been renamed to `fontsize` in Makie v0.19. Please change all occurrences of `textsize` to `fontsize` or revert back to an earlier version."))

Most learning tasks let you pass additional augmentations as keyword arguments. For example, ImageClassification takes the aug_projection and aug_image arguments. FastAI.jl provides the augs_projection helper to quickly construct a set of projective data augmentations.


			
			
			
			task2
			 
			=
			 
			

	
			BlockTask
			(
			
    
			blocks
			,
			
    
			
			(
			
        
			

	
			ProjectiveTransforms
			(
			
			(
			128
			,
			 
			128
			)
			,
			 
			
			augmentations
			=
			

	
			augs_projection
			(
			)
			)
			,
			
        
			

	
			ImagePreprocessing
			(
			)
			,
			
        
			

	
			OneHot
			(
			)
			
    
			)
			

			)
			

			
			
			xs2
			,
			 
			ys2
			 
			=
			 
			
			

	
			FastAI
			.
			

	
			makebatch
			(
			task2
			,
			 
			data
			,
			 
			
			fill
			(
			4
			,
			 
			3
			)
			)
			

			

	
			showbatch
			(
			task2
			,
			 
			
			(
			xs2
			,
			 
			ys2
			)
			)

			LoadError("string", 10, ArgumentError("The attribute `textsize` has been renamed to `fontsize` in Makie v0.19. Please change all occurrences of `textsize` to `fontsize` or revert back to an earlier version."))

Likewise, there is an augs_lighting helper that adds contrast and brightness augmentation:


			
			
			
			task3
			 
			=
			 
			

	
			BlockTask
			(
			
    
			blocks
			,
			
    
			
			(
			
        
			

	
			ProjectiveTransforms
			(
			
			(
			128
			,
			 
			128
			)
			,
			 
			
			augmentations
			=
			

	
			augs_projection
			(
			)
			)
			,
			
        
			

	
			ImagePreprocessing
			(
			
			augmentations
			=
			

	
			augs_lighting
			(
			)
			)
			,
			
        
			

	
			OneHot
			(
			)
			
    
			)
			

			)
			

			
			
			xs3
			,
			 
			ys3
			 
			=
			 
			
			

	
			FastAI
			.
			

	
			makebatch
			(
			task3
			,
			 
			data
			,
			 
			
			fill
			(
			4
			,
			 
			3
			)
			)
			

			

	
			showbatch
			(
			task3
			,
			 
			
			(
			xs3
			,
			 
			ys3
			)
			)

			LoadError("string", 10, ArgumentError("The attribute `textsize` has been renamed to `fontsize` in Makie v0.19. Please change all occurrences of `textsize` to `fontsize` or revert back to an earlier version."))