DoM: Math Problems to Computation Graphs
A PyTorch transformer that generates structured math programs, with correct and incorrect decodes to inspect.
Result & current limits
On 2,985 MathQA test problems, the committed test log records 78.99% exact graph match with constrained decoding (77.65% without), 85.62% 2-gram BLEU, and 2.56 average graph edit distance. Structural constraints do not guarantee the mathematical answer is correct.
Problem
Math word problems require translating language into operations. A model can produce either a malformed program or a valid program that solves the wrong problem; those failures need separate evaluation.
Solution
Trained a decoder-only sequence-to-sequence transformer on MathQA and restricted decoding choices to valid graph structure. Compared generated computation graphs with reference programs.
What I built
Co-developed the tokenizer, data loader, model, training loop, and constrained decoding in PyTorch. Analyzed successful decodes alongside over-generation and missing-operation errors.
Technologies
More demos, engineering decisions, and evaluation notes
Inspect the remaining errors
The constrained decoder can produce a valid graph with extra operations or omit steps the problem requires. Showing an incorrect decode beside a success separates structural validity from semantic accuracy.
Paper
Where the scores come from
The scores above are from the committed test log. The course report’s table (77.45%, 84.78% and 2.72) matches the validation log instead, so this page does not use it.