Data platforms & applied AI / CASE STUDY
A decision-support data product
Keeping searchable records, calculated answers, and generated explanations consistent as the underlying facts change.
An anonymous account of an individual product or tool I’ve worked on. Product identity and identifying operational details are withheld; the stack and engineering decisions describe implemented work.
- My role
- Product direction and implementation across the data, application, and AI layers, including regression checks and iteration.
The problem
A correct calculation can still be undermined by an explanation quoting an old or invented number. The product needs to turn changing information into useful answers without letting a fluent narrative outrun its evidence.
Stack & system flow
- Next.js
- TypeScript
- React
- PostgreSQL
- Vitest
- Playwright
A typed web application brings structured records, explicit calculations, and generated explanations into the same interface. Database-backed inputs anchor the answer; the model supplies language.
Simplified flow
- Incoming records
- Structured data
- Calculated results
- Generated explanation
- Claim checks
- Reader
Engineering decisions
Give calculations and prose different jobs
Calculated values come from explicit inputs and rules. The answer-processing layer checks numerical claims against the returned data and can substitute deterministic text when a generated answer introduces unsupported figures.
Treat stored explanations as perishable
Narratives retain the context they were generated from. Before display, the product compares that context with current facts. Changed or missing evidence can invalidate a narrative until it is regenerated. That can leave a temporary gap, but avoids silently presenting an explanation that contradicts the data.
Ask for missing inputs
Incomplete information should prompt clarification rather than become a plausible zero. This makes some interactions less immediate, while keeping the distinction between an unknown result and a calculated result.
What was implemented
The implemented product combines ingestion, searchable records, deterministic tools, and explanatory AI. Freshness checks and answer validation sit in the path to the reader rather than existing only as offline evaluation ideas.
Representative regression coverage
- Numerical-answer regression cases include invented figures, missing inputs, empty results, and valid decimals.
- Freshness checks exercise rounded figures, numbers written as words, changed facts, and missing evidence.
- Generation/review checks cover incomplete structured output and record failed runs.
These examples describe repository tests reviewed for this case study, rather than a new production audit.
Tradeoffs & limits
The reviewed implementation and tests establish specific contracts. They do not establish production-wide answer accuracy, complete source coverage, or measured user impact.