.FastAIDiscriminativeLRs

			DiscriminativeLRs(paramgroups, factors)

Use different learning rates based on paramgroups. factors maps each group to a factor that the learning rate is multiplied by, so for a parameter x the factor is get(factors, getgroup(paramgroups, x), 1).

See ParamGroups.

Examples

Combining with regular gradient descent, but only training a part of the model.

			
			
			
			
			using
			
			 

	
			Flux
			.

	
			Optimise
			:
			
			 

	
			Descent
			,
			
			 

	
			Optimiser
			

			

			
			model
			 
			=
			 
			

	
			Chain
			(
			

	
			Dense
			(
			3
			,
			 
			5
			)
			,
			 
			

	
			Dense
			(
			5
			,
			 
			3
			)
			)
			

			
			paramgroups
			 
			=
			 
			

	
			ParamGroups
			(
			

	
			IndexGrouper
			(
			
			[
			1
			,
			 
			2
			]
			)
			,
			 
			model
			)
			

			

			
			dlro
			 
			=
			 
			

			DiscriminativeLRs
			(
			paramgroups
			,
			 
			
			Dict
			(
			
			1
			 
			=>
			 
			0.
			,
			 
			
			2
			 
			=>
			 
			1.
			)
			)
			

			
			o
			 
			=
			 
			

	
			Optimiser
			(
			dlro
			,
			 
			

	
			Descent
			(
			0.1
			)
			)