Transolver is a Linear Transformer: Revisiting Physics-Attention through the Lens of Linear Attention
Original Research Paper
Reframes Transolver's Physics-Attention as a special case of linear attention and proposes LinearNO (Linear Attention Neural Operator); achieves SOTA on six PDE benchmarks with 40.0% fewer parameters and 36.2% less compute, plus superior performance on AirfRANS and Shape-Net Car industrial datasets.
arXiv preprint (copyright held by authors) Neural OperatorTransformerLinear Attention