BiliSakura/PAE-diffusers

PAE Diffusers Checkpoints

Converted PAE (Prior-Aligned Autoencoder) tokenizer checkpoints in standard Hub custom-pipeline layout.

PAE is a VAE-free latent framework. Each variant splits into dedicated components:

Variant	VFM backbone (decoder config)	Latent dim	Input size
pae-dinov2-large-d32	DINOv2-L (with registers)	32	224
pae-dinov3-large-d32	DINOv3-ViT-L/16	32	256
pae-mae-large-d32	MAE-L	32	256
pae-siglip2-so400m-d32	SigLIP2-SO400M	32	256

Each variant directory is a self-contained Diffusers repo. Each component subfolder ships one Python file:

model_index.json
pipeline.py
scheduler/scheduling_flow_match_pae.py
transformers/transformer_lightning_dit.py
decoder/decoder_pae.py
decoder/diffusion_pytorch_model.safetensors

Usage

from pathlib import Path
import torch
from diffusers import DiffusionPipeline

model_dir = Path("/home/czy/local/models/BiliSakura/PAE-diffusers/pae-dinov2-large-d32").resolve()
pipe = DiffusionPipeline.from_pretrained(
    str(model_dir),
    local_files_only=True,
    custom_pipeline=str(model_dir / "pipeline.py"),
    trust_remote_code=True,
    torch_dtype=torch.bfloat16,
).to("cuda")

print(pipe.get_label_ids("golden retriever"))