Tadpole
Paper GitHub Hugging Face
NeurIPS 2026

Learn about the world
before predicting it.

Tadpole is a foundation model for three-dimensional PDEs. It learns a reusable encoding of physical structure through autoencoding and then adapts it for reconstruction, dynamics, and generation.

Qiang Liu · Felix Koehler · Benjamin Holzschuh · Nils Thuerey Technical University of Munich · MCML

Read the paper View code Download weights
Tadpole logo

The starting point

Physics foundation models usually begin with a difficult question.

“Given the past, what happens next?”

That is a natural objective, but the same state can evolve differently under different equations, boundary conditions, and parameters. In 3D, the simulations needed to learn all those possibilities are also enormous.

Transient Simulation of the 3D KS equation. The KS equation represents chaotic behavior of evolving a system from the past into the future.
01

Data is heavy

High-resolution 3D trajectories quickly overwhelm storage and data-loading pipelines.

02

Worlds differ

Every system brings its own variables, spatial scale, and governing dynamics.

03

Models specialize

Training only for time evolution can bind a representation to a single kind of task.

The Tadpole idea

Ask an easier, more universal question first.

Instead ofWhat happens next?
FirstWhat does a physical state look like?

Tadpole is pre-trained as a variational autoencoder. It encodes a single 3D state into a compact latent representation, then reconstructs the original field. This makes pre-training about spatial structure, not about committing to one system’s evolution.

3D state u
Encode qφ(z | u)
Latent sampling z = μ + σ ⊙ ε
Decode pθ(u | z)
Reconstruction û
Variational autoencoder Tadpole compresses a 3D physical state into a sampled latent representation, then reconstructs the field.

Online learning

202 TB of experience.
Not 202 TB on disk.

GPU solvers generate synthetic PDE data while the network trains. A multi-stage buffer keeps simulation, communication, and learning moving in parallel, removing the need to precompute and repeatedly load a giant static dataset.

202 TBequivalent training data
1.8×end-to-end speedup*
7pre-training PDE systems
0 TBlocal dataset storage
01 · Simulator TorchFSM logo TorchFSM GPU-based PDE simulation
Generating live
02 · Simulation node FIFO Buffer Fresh simulation states
inout
Intra-node TCP/IP
03 · Training process FIFO Buffer Received PDE states
inout
04 · Local cache MFU Buffer Frequently reused samples
05 · Model training GPU training jobs Cross-node DDP
Cross-node DDP
Node AGPU · Rank 0
Node BGPU · Rank 1
Node NGPU · Rank N
Online pre-training pipeline Simulation, communication, caching, and distributed model training run concurrently—keeping GPUs supplied without a precomputed dataset.

*Measured against offline training in the authors’ high-speed SSD environment.

A shared visual language

One model,
many physical worlds.

Tadpole splits variables into individual fields and learns from random 3D crops. At inference, the same model can process different channel counts and domain sizes, including resolutions far beyond those seen in pre-training.

Zero-shot reconstruction visualization for isotropic turbulence
01 Isotropic Turbulence [4 × 1024 × 1024 × 1024]
Zero-shot reconstruction visualization for turbulent channel flow
02 Turbulent Channel Flow [3 × 96 × 96 × 192]
Zero-shot reconstruction visualization for a transitional boundary layer
03 Transitional Boundary Layer [4 × 224 × 224 × 224]
Zero-shot reconstruction visualization for magnetohydrodynamics
04 Magnetohydrodynamics [10 × 512 × 512 × 512]
Zero-shot reconstruction

Visualizations of Tadpole-B zero-shot reconstruction on different datasets (up: ground truth, down: zero-shot reconstruction).

64³

crop size during pre-training

1024³

largest evaluated spatial resolution

What the representation becomes

Learn once. Adapt three ways.

Tadpole-DFT adds latent transformations, restored skip connections, and LoRA to learn temporal evolution efficiently.

ut
LoRA
EEncoder
× γ0 × γ1 × γi
LoRA
DDecoder
ut+Δt zt
SSub-network

Use the pre-trained model directly on unseen systems, or fine-tune it for sharper domain-specific reconstructions.

ut
LoRA
EEncoder
LoRA
DDecoder
ût

Train a flow-matching model in the compact latent space, then decode new samples into full 3D physical fields.

ut
LoRA
EEncoder
zt
Generative Model Latent flow matching

Dynamics fine-tuning

Teach the representation how time moves.

Tadpole-DFT preserves the pre-trained backbone while adding only the capacity needed for dynamics. It performs best among the models compared and achieves superior performance compared to the model trained from scratch.

01 Downstream dynamics performance

TBL — One-step

NRMSEES (×10−2)
Tadpole-L
Tadpole-B
Tadpole-S
Walrus
MORPH-S
DPOT-S

Iso — One-step

NRMSEES (×10−2)
Tadpole-L
Walrus
Tadpole-B
Tadpole-S
DPOT-S
MORPH-S

Trainable Parameters

Trainable Params (M)
Tadpole-S
Tadpole-B
Tadpole-L
MORPH-S
DPOT-S
Walrus
Dynamics results Performance of Tadpole on two distinct downstream dynamics tasks TBL and Iso.
02 Pre-training improvements

TBL — One-step

NRMSEES / NRMSEESScratch
DFT LORA64
DFT LORA32
FPFT
DFT LORA16

Iso — One-step

NRMSEES / NRMSEESScratch
DFT LORA64
DFT LORA32
FPFT
DFT LORA16

Trainable Parameters

Trainable Params (M)
DFT LORA16
DFT LORA32
DFT LORA64
FPFT
Pre-training gains Performance improvements on the dynamics test from pre-training of Tadpole.

Summary

From simulation
to a reusable model.

  1. 01

    Learn the spatial structure

    A variational autoencoder compresses and reconstructs each state through a shared latent space.

  2. 02

    Simulate live

    GPU-based pseudo-spectral solvers continuously generate diverse 3D PDE states.

  3. 03

    Split and crop

    Variables become single-channel samples; random local crops unify different systems and sizes.

  4. 04

    Adapt efficiently

    Zero-shot inference, LoRA, and task-specific modules turn the representation into downstream tools.

Reference

Cite Tadpole

Read the paper View code Download weights 
@inproceedings{liu2026tadpole,
  title     = {Tadpole: Autoencoders as Foundation Models
               for 3D PDEs with Online Learning},
  author    = {Qiang Liu and Felix Koehler and Benjamin Holzschuh and Nils Thuerey},
  booktitle = {Advances in Neural Information Processing Systems},
  year      = {2026}
}