← Selected work

DoM: Math Problems to Computation Graphs

A PyTorch transformer that generates structured math programs, with correct and incorrect decodes to inspect.

FIG. 10 — Selected correct decode: an 18-operation payroll program. Aggregate scores come from the test log.

Result & current limits

On 2,985 MathQA test problems, the committed test log records 78.99% exact graph match with constrained decoding (77.65% without), 85.62% 2-gram BLEU, and 2.56 average graph edit distance. Structural constraints do not guarantee the mathematical answer is correct.

Problem

Math word problems require translating language into operations. A model can produce either a malformed program or a valid program that solves the wrong problem; those failures need separate evaluation.

Solution

Trained a decoder-only sequence-to-sequence transformer on MathQA and restricted decoding choices to valid graph structure. Compared generated computation graphs with reference programs.

What I built

Co-developed the tokenizer, data loader, model, training loop, and constrained decoding in PyTorch. Analyzed successful decodes alongside over-generation and missing-operation errors.

Technologies

More demos, engineering decisions, and evaluation notes

Inspect the remaining errors

The constrained decoder can produce a valid graph with extra operations or omit steps the problem requires. Showing an incorrect decode beside a success separates structural validity from semantic accuracy.

FIG. 10A — Selected failure: a valid structure with missing operations.
Decoder-only transformer used to generate computation-graph tokens.
FIG. 10B — Tokenizer, decoder, and structured output pipeline.

Paper

  1. Page 1 of DoM: Math Problems to Computation Graphs
  2. Page 2 of DoM: Math Problems to Computation Graphs
  3. Page 3 of DoM: Math Problems to Computation Graphs
  4. Page 4 of DoM: Math Problems to Computation Graphs
  5. Page 5 of DoM: Math Problems to Computation Graphs

Scroll sideways to read · open the PDF

Where the scores come from

The scores above are from the committed test log. The course report’s table (77.45%, 84.78% and 2.72) matches the validation log instead, so this page does not use it.

Links