PDE Residual Loss
Definition
The squared residual obtained by substituting the network output into the PDE, used as a training loss term. It is the core mechanism of PINNs: it lets the network be constrained by the governing equation even with no labeled solution data.
What it is
Given a PDE N[u] = 0, substitute the network output u_theta to obtain r = N[u_theta] and use L_res = mean(r^2) over collocation points. Derivatives come exactly from automatic differentiation, so no mesh differencing is needed — this is why PINNs work mesh-free.
Key points
- Loss terms need weighting: data, residual and boundary terms can differ by orders of magnitude, and bad weights make one term vanish entirely.
- Collocation sampling: uniform random is the baseline; refining where gradients are steep (residual-based adaptive refinement, RAR) usually improves convergence a lot.
- High-order derivatives are expensive: the AD graph for second and higher derivatives is memory heavy; first-order system formulation or stochastic dimension methods cut the cost.
- The residual loss is only a soft constraint; at the end of training the residual is nonzero, so always report its magnitude as a physical-consistency metric.
Common pitfalls / boundaries
- Falling training loss does not mean a correct solution: "low loss but completely wrong solution" is a real failure mode. Validate against an independent solution.
- For strongly nonlinear equations (e.g. high-Reynolds Navier-Stokes) the residual landscape is extremely rugged and needs curriculum learning or easy-to-hard scheduling.