transformer_encoder(planes, depth, nheads; mlp_ratio = 4.0, dropout = 0.)

Transformer as used in the base ViT architecture. (reference).

Arguments

• planes: number of input channels
• depth: number of attention blocks
• nheads: number of attention heads
• mlp_ratio: ratio of MLP layers to the number of input channels
• dropout: dropout rate