← Back to glossary

Physics Foundation Model

Domain Also:Physics Large Model / Scientific Foundation Model Updated 2026-11-04
Definition

General-purpose models pre-trained on large cross-task, cross-physics scientific corpora that adapt to specific downstream problems through fine-tuning or prompting. The goal is to turn one-off per-task modeling into a reusable train-once, deliver-many engineering pattern.

What it is

A physics foundation model is a general-purpose model pre-trained on scientific data. The logic mirrors that of language foundation models: learn a general representation on large cross-task corpora first, then adapt to specific downstream problems through fine-tuning, in-context learning or conditioning inputs.

The differences lie in the data and the objective. Training data is not web text but simulations and experiments spanning multiple physical fields, scales and operating regimes; and the learning target is not merely to fit a scalar field but to retain physical consistency - conservation relations, symmetries and geometric relationships - structurally.

Key points

  • Joint multi-task pre-training is the core: tasks with different meshes, geometries,

materials and loads go into one model that shares an underlying physical representation. The payoff is not only data efficiency but also cross-regime extrapolation - physical laws seen during training transfer to geometries and loads never seen.

  • Context conditioning changes how adaptation works: the model takes boundary conditions,

geometry or regime descriptions as part of its input, so one set of weights serves many downstream tasks, turning "train once per task" into "train once, plug in many".

  • Geometry and symmetry must be designed in: equivariant structures (such as E(3)-equivariant

message passing) markedly improve data efficiency and extrapolation stability on atomistic and rotation-invariant field problems, something a plain Transformer architecture does not discover automatically.

  • Physics constraints should enter during pre-training: keeping conservation residuals or

equation constraints in the loss is far more effective than fixing things up with post processing afterwards.

Common pitfalls / boundaries

  • A foundation model is not just a bigger single-task model. If the downstream task is far from

the pre-training distribution, fine-tuning may yield less than a small model purpose-built for that task.

  • Data scale is not data quality. Scientific data often has missing labels, inconsistent units

and mismatched meshes, and careless cleaning teaches the model false correlations.

  • Generality has a cost. One large model is usually less accurate on a specific task than a

dedicated small one; industrial deployments commonly combine a foundation model, task-specific fine-tuning and a classical solver as a fallback.

  • Evaluate with held-out operating regimes and geometries. Testing only on an

in-distribution test set cannot reveal the value of a foundation model.

Related terms

Related resources