PDE Solver Code Perturbation Evaluation Dataset
My rating
This HuggingFace dataset contains 256 PDE solver implementations for evaluating LLM performance on physics-related code tasks. The collection includes 64 base programs split evenly between physically valid implementations and those with injected bugs that still execute. Each base program is expanded into 4 lexical perturbation conditions: original comments preserved, comments removed, comments swapped from different implementations, and descriptive identifiers replaced with meaningless placeholders. The perturbations target only the lexical surface while maintaining executable behavior, enabling systematic evaluation of how well language models handle code understanding under various syntactic variations.
Tags: