PyTorch Image Models

• What's New
• Introduction
• Models
• Features
• Results
• Getting Started (Documentation)
• Train, Validation, Inference Scripts
• Awesome PyTorch Resources
• Licenses
• Citing

What's New

July 10, 2026

• Improve optimizer torch.compile and tensor learning-rate support.
• Extend NaFlexViT patch-layout (for NaFlex-CLAP), and forward_intermediates (NaFlex dict input) support.
• Harden pickle loading and improve custom-label inference.
• Release 1.0.28

May 27, 2026

• Add model defs and pretrained weights for EUPE ViT (DINOv3-style) and ConvNeXt models. See the Efficient Universal Perception Encoder paper.
• Add TIPSv2 model defs and pretrained weights for (DINOv2-style) ViTs. See the TIPSv2 paper.

May 8, 2026

• Release 1.0.27

April 23, 2026

• Add Gemma4 ViT encoders w/ NaFlex pipeline support (variable aspect/size per image). Thanks Yonghye Kwon
• Support DINOv3 weights in NaFlexVit. Thanks Yonghye Kwon
• Some improvements to Muon fallback (AdamW/NadamW) lr behavior

March 23, 2026

• Improve pickle checkpoint handling security. Default all loading to weights_only=True, add safe_global for ArgParse.
• Improve attention mask handling for core ViT/EVA models & layers. Resolve bool masks, pass is_causal through for SSL tasks.
• Fix class & register token uses with ViT and no pos embed enabled.
• Add Patch Representation Refinement (PRR) as a pooling option in ViT. Thanks Sina (https://github.com/sinahmr).
• Improve consistency of output projection / MLP dimensions for attention pooling layers.
• Hiera model F.SDPA optimization to allow Flash Attention kernel use.
• Caution added to SGDP optimizer.
• Release 1.0.26. First maintenance release since my departure from Hugging Face.

Feb 23, 2026

• Add token distillation training support to distillation task wrappers
• Remove some torch.jit usage in prep for official deprecation
• Caution added to AdamP optimizer
• Call reset_parameters() even if meta-device init so that buffers get init w/ hacks like init_empty_weights
• Tweak Muon optimizer to work with DTensor/FSDP2 (clamp_ instead of clamp_min_, alternate NS branch for DTensor)
• Release 1.0.25

Jan 21, 2026

• Compat Break: Fix oversight w/ QKV vs MLP bias in ParallelScalingBlock (& DiffParallelScalingBlock)
  • Does not impact any trained timm models but could impact downstream use.

Jan 5 & 6, 2026

• Release 1.0.24
• Add new benchmark result csv files for inference timing on all models w/ RTX Pro 6000, 5090, and 4090 cards w/ PyTorch 2.9.1
• Fix moved module error in deprecated timm.models.layers import path that impacts legacy imports
• Release 1.0.23

Dec 30, 2025

• Add better NAdaMuon trained dpwee, dwee, dlittle (differential) ViTs with a small boost over previous runs
  • https://huggingface.co/timm/vit_dlittle_patch16_reg1_gap_256.sbb_nadamuon_in1k (83.24% top-1)
  • https://huggingface.co/timm/vit_dwee_patch16_reg1_gap_256.sbb_nadamuon_in1k (81.80% top-1)
  • https://huggingface.co/timm/vit_dpwee_patch16_reg1_gap_256.sbb_nadamuon_in1k (81.67% top-1)
• Add a ~21M param timm variant of the CSATv2 model at 512x512 & 640x640
  • https://huggingface.co/timm/csatv2_21m.sw_r640_in1k (83.13% top-1)
  • https://huggingface.co/timm/csatv2_21m.sw_r512_in1k (82.58% top-1)
• Factor non-persistent param init out of __init__ into a common method that can be externally called via init_non_persistent_buffers() after meta-device init.

Dec 12, 2025

• Add CSATV2 model (thanks https://github.com/gusdlf93) -- a lightweight but high res model with DCT stem & spatial attention. https://huggingface.co/Hyunil/CSATv2
• Add AdaMuon and NAdaMuon optimizer support to existing timm Muon impl. Appears more competitive vs AdamW with familiar hparams for image tasks.
• End of year PR cleanup, merge aspects of several long open PR
  • Merge differential attention (DiffAttention), add corresponding DiffParallelScalingBlock (for ViT), train some wee vits
    • https://huggingface.co/timm/vit_dwee_patch16_reg1_gap_256.sbb_in1k
    • https://huggingface.co/timm/vit_dpwee_patch16_reg1_gap_256.sbb_in1k
  • Add a few pooling modules, LsePlus and SimPool
  • Cleanup, optimize DropBlock2d (also add support to ByobNet based models)
• Bump unit tests to PyTorch 2.9.1 + Python 3.13 on upper end, lower still PyTorch 1.13 + Python 3.10

Dec 1, 2025

• Add lightweight task abstraction, add logits and feature distillation support to train script via new tasks.
• Remove old APEX AMP support

Nov 4, 2025

• Fix LayerScale / LayerScale2d init bug (init values ignored), introduced in 1.0.21. Thanks https://github.com/Ilya-Fradlin
• Release 1.0.22

Oct 31, 2025 🎃

• Update imagenet & OOD variant result csv files to include a few new models and verify correctness over several torch & timm versions
• EfficientNet-X and EfficientNet-H B5 model weights added as part of a hparam search for AdamW vs Muon (still iterating on Muon runs)

Oct 16-20, 2025

• Add an impl of the Muon optimizer (based on https://github.com/KellerJordan/Muon) with customizations
  • extra flexibility and improved handling for conv weights and fallbacks for weight shapes not suited for orthogonalization
  • small speedup for NS iterations by reducing allocs and using fused (b)add(b)mm ops
  • by default uses AdamW (or NAdamW if nesterov=True) updates if muon not suitable for parameter shape (or excluded via param group flag)
  • like torch impl, select from several LR scale adjustment fns via adjust_lr_fn
  • select from several NS coefficient presets or specify your own via ns_coefficients
• First 2 steps of 'meta' device model initialization supported
  • Fix several ops that were breaking creation under 'meta' device context
  • Add device & dtype factory kwarg support to all models and modules (anything inherting from nn.Module) in timm
• License fields added to pretrained cfgs in code
• Release 1.0.21

Sept 21, 2025

• Remap DINOv3 ViT weight tags from lvd_1689m -> lvd1689m to match (same for sat_493m -> sat493m)
• Release 1.0.20

Sept 17, 2025

• DINOv3 (https://arxiv.org/abs/2508.10104) ConvNeXt and ViT models added. ConvNeXt models were mapped to existing timm model. ViT support done via the EVA base model w/ a new RotaryEmbeddingDinoV3 to match the DINOv3 specific RoPE impl
  • HuggingFace Hub: https://huggingface.co/collections/timm/timm-dinov3-68cb08bb0bee365973d52a4d
• MobileCLIP-2 (https://arxiv.org/abs/2508.20691) vision encoders. New MCI3/MCI4 FastViT variants added and weights mapped to existing FastViT and B, L/14 ViTs.
• MetaCLIP-2 Worldwide (https://arxiv.org/abs/2507.22062) ViT encoder weights added.
• SigLIP-2 (https://arxiv.org/abs/2502.14786) NaFlex ViT encoder weights added via timm NaFlexViT model.
• Misc fixes and contributions

July 23, 2025

• Add set_input_size() method to EVA models, used by OpenCLIP 3.0.0 to allow resizing for timm based encoder models.
• Release 1.0.18, needed for PE-Core S & T models in OpenCLIP 3.0.0
• Fix small typing issue that broke Python 3.9 compat. 1.0.19 patch release.

July 21, 2025

• ROPE support added to NaFlexViT. All models covered by the EVA base (eva.py) including EVA, EVA02, Meta PE ViT, timm SBB ViT w/ ROPE, and Naver ROPE-ViT can be now loaded in NaFlexViT when use_naflex=True passed at model creation time
• More Meta PE ViT encoders added, including small/tiny variants, lang variants w/ tiling, and more spatial variants.
• PatchDropout fixed with NaFlexViT and also w/ EVA models (regression after adding Naver ROPE-ViT)
• Fix XY order with grid_indexing='xy', impacted non-square image use in 'xy' mode (only ROPE-ViT and PE impacted).

July 7, 2025

• MobileNet-v5 backbone tweaks for improved Google Gemma 3n behaviour (to pair with updated official weights)
  • Add stem bias (zero'd in updated weights, compat break with old weights)
  • GELU -> GELU (tanh approx). A minor change to be closer to JAX
• Add two arguments to layer-decay support, a min scale clamp and 'no optimization' scale threshold
• Add 'Fp32' LayerNorm, RMSNorm, SimpleNorm variants that can be enabled to force computation of norm in float32
• Some typing, argument cleanup for norm, norm+act layers done with above
• Support Naver ROPE-ViT (https://github.com/naver-ai/rope-vit) in eva.py, add RotaryEmbeddingMixed module for mixed mode, weights on HuggingFace Hub

|model |img_size|top1 |top5 |param_count| |--------------------------------------------------|--------|------|------|-----------| |vit_large_patch16_rope_mixed_ape_224.naver_in1k |224 |84.84 |97.122|304.4 | |vit_large_patch16_rope_mixed_224.naver_in1k |224 |84.828|97.116|304.2 | |vit_large_patch16_rope_ape_224.naver_in1k |224 |84.65 |97.154|304.37 | |vit_large_patch16_rope_224.naver_in1k |224 |84.648|97.122|304.17 | |vit_base_patch16_rope_mixed_ape_224.naver_in1k |224 |83.894|96.754|86.59 | |vit_base_patch16_rope_mixed_224.naver_in1k |224 |83.804|96.712|86.44 | |vit_base_patch16_rope_ape_224.naver_in1k |224 |83.782|96.61 |86.59 | |vit_base_patch16_rope_224.naver_in1k |224 |83.718|96.672|86.43 | |vit_small_patch16_rope_224.naver_in1k |224 |81.23 |95.022|21.98 | |vit_small_patch16_rope_mixed_224.naver_in1k |224 |81.216|95.022|21.99 | |vit_small_patch16_rope_ape_224.naver_in1k |224 |81.004|95.016|22.06 | |vit_small_patch16_rope_mixed_ape_224.naver_in1k |224 |80.986|94.976|22.06 |

• Some cleanup