AdaBelief(η = 0.001, β::Tuple = (0.9, 0.999), ϵ = 1.0e-8)

The AdaBelief optimiser is a variant of the well-known Adam optimiser.

Parameters

• Learning rate ( η): Amount by which gradients are discounted before updating the weights.

• Decay of momentums ( β::Tuple): Exponential decay for the first (β1) and the second (β2) momentum estimate.

Examples

			
			
			
			opt
			 
			=
			 
			

			AdaBelief
			(
			)
			

			

			
			opt
			 
			=
			 
			

			AdaBelief
			(
			0.001
			,
			 
			
			(
			0.9
			,
			 
			0.8
			)
			)