Turning Probabilistic AI Nearly Deterministic: How Jev and Gemini Decision-1 Cut 20-Year Knowledge Distillation Costs by 95%
How I used Jev and Google's zero-output-token gemini-decision-1 in AyoWork to backfill 20 years of shop-to-ledger archives at 95% lower cost.

On this page5 sectionsExpandCollapse
Pointing a frontier reasoning model raw at twenty years of company history is a great way to burn cash and still end up with an uncalibrated mess.
Every mid-sized manufacturing and engineering business sits on decades of dark operational knowledge. Across Predator Cycling, MACC Composites, and AyoWork, my own archive stretches back sixteen to twenty years—all the way to 2006. That history lives across custom carbon frame build threads, composite layup work orders, supplier AP invoices, shipping discrepancies, and thousands of historical bank lines.
Five weeks ago, I started building automated harvesting pipelines to categorize and distill that entire historical repository into structured, searchable institutional memory inside AyoBrain. Throwing a large generative chat model directly at millions of raw rows failed two tests immediately: the token bill made mining twenty years back unjustifiable, and raw prose outputs drifted whenever a downstream workflow needed a crisp, repeatable fork.
This past weekend, I completed the migration onto Google’s newly released gemini-decision-1 inside AyoWork’s native GCP Rust stack—completing an architectural evolution that began with my early System-1 harvesting engine Jev and cut historical distillation costs by roughly 95%. Here is how calibrated decision models, dormant confidence scores brought back to life, and frontier models working strictly downstream turn probabilistic AI into an almost deterministic operating tool.
Harvesting 20 Years of History with Jev and the 95% Cost Collapse
Most “AI ingestion” pipelines start backward: they hand an uncurated email body, invoice PDF, or ledger description directly to a flagship generative model and pay for hundreds of output tokens just to hear "this looks like a shipping notification, skip it."
Five weeks ago, I flipped that order. Instead of sending raw history straight into expensive distillation prompts, I put lightweight deterministic Rust pre-filters in front of Jev—my early System-1 decision harvester built to classify, score, and route operational records at high speed before any generative synthesis runs. The economics follow directly from gating what reaches the heavy tier:
Where is the historical corpus size, is the fraction settled outright by zero-cost deterministic rules (~40% on repetitive bank and system notices), is the pennies-per-thousand cost of System-1 classification, and is the clean high-signal slice that actually deserves frontier distillation ().
Dropping total ingestion and distillation cost by ~95% completely changed the scope of what was viable. Backfilling two years of operational records used to feel expensive; at 95% lower cost, mining sixteen to twenty years deep back to 2006 suddenly fit inside a normal engineering budget. Custom frame geometry nuances, historical vendor lead-time anomalies, and complex tax/accounting classifications stopped sitting in cold archives and started indexing cleanly into AyoBrain’s 3,072-dimensional pgvector store.
Even more importantly, structuring granular decision rubrics for that historical harvest worked so cleanly (95.8% live intent benchmark accuracy) that I promoted the exact same pattern to how AyoWork ingests live day-to-day data. Instead of one vague prompt trying to read, classify, decide urgency, and summarize all at once, every incoming event hit focused System-1 decision checkpoints first—raising both accuracy and throughput immediately.
Why Migrating to gemini-decision-1 on GCP Clicked Immediately
When Google launched gemini-decision-1, migrating my live AyoWork decision layer off external harvesting glue this past weekend was a natural architectural consolidation. AyoWork already runs natively inside Google Cloud (Cloud Run, Cloud SQL, BigQuery, and Vertex AI), so pulling System-1 decision evaluation directly into the Gemini family removed network hops, unified credential governance, and unlocked three capabilities that made the routing layer dramatically cleaner:
- Zero Output Tokens in a Single Multi-Question Pass: Calling
POST /v1beta/decisionsevaluates up to1,000independent questions against shared multimodal or text context in a single sub-250ms pass—emitting0output tokens. Instead of paying a chat model to stream JSON prose field by field, the API returns calibrated probabilities natively across three question primitives:noul(binary yes/no noise gates),choice(exact queue or chart-of-accounts selection up to 255 options), andscore(ordered operational urgency or materiality rubrics). - Reviving AyoWork’s Dormant Confidence Factor: Back in AyoWork’s earliest Rust ledger and triage schemas, I wired a
confidencemetric into suggestion tables—and then barely trusted it, because self-reported confidence scores hallucinated inside LLM chat JSON ("confidence": 0.99on pure guesses) are famously uncalibrated. Calibrated probability distributions fromgemini-decision-1finally made that dormant field real telemetry. - Zero-Float Integer Basis-Point Routing (
0to10,000 bps): Consistent with AyoWork’s zero-float ledger rules (MoneyCentsandDecimal), I never evaluate routing thresholds withf64comparisons that silently suffer IEEE rounding drift. Every probability and confidence metric returned fromv1beta/decisionsis parsed via exact integer decimal scaling intou16basis points (0..=10,000 bps) inside my typed Rust validation layer after strict PII/secret redaction:
/// High-level Rust pattern for Tier-1 calibrated routing (sanitized example).
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum DecisionRoute {
DeterministicDispatch, // confidence >= 9,000 bps (high certainty)
HumanInTheLoopReview, // 6,500..=8,999 bps (ambiguous split or edge case)
ArchiveNoiseDropped, // binary noise gate triggered >= 8,500 bps
}
pub fn route_calibrated_signal(
is_noise_bps: u16,
queue_confidence_bps: u16,
requires_synthesis: bool,
) -> (DecisionRoute, bool) {
if is_noise_bps >= 8_500 {
return (DecisionRoute::ArchiveNoiseDropped, false);
}
let route = if queue_confidence_bps >= 9_000 {
DecisionRoute::DeterministicDispatch
} else {
DecisionRoute::HumanInTheLoopReview
};
// Only structured, non-noise operational records wake downstream distillation
(route, requires_synthesis)
}
Just like the typed create_draft_journal_entry boundary I covered in Agents Don’t Need More Autonomy. They Need a Gateway and a Memory., gemini-decision-1 never writes database rows or picks arbitrary categories. It supplies calibrated integer probabilities at granular workflow forks, and deterministic Rust guards decide where the record goes next.
Human-in-the-Loop Gates and Practically Free Frontier Synthesis
The biggest surprise of embedding System-1 decision gates at a granular workflow level is what it unlocks downstream—both for human operators and for frontier reasoning models like Gemini 4 Hydrogen and Gemini 4 Carbon.
When uncalibrated AI handles ingestion, you either distrust everything (and drown your staff in manual review queues) or auto-approve too much (and clean up subtle ledger errors months later). By calibration-gating every workflow transition in basis points, humans only see the narrow mid-confidence slice (6,500–8,999 bps) where ambiguity legitimately exists: an unusual mixed-purpose supplier invoice, a partial composite repair split across two work orders, or an edge-case technical customer thread. Everything clearly routine resolves deterministically at high confidence; everything genuinely unclear lands cleanly in a one-click operator workbench.
Even better, because Tier 0 rules and Tier 1 gemini-decision-1 gates strip out noise and pre-tag structured entities, urgency, and domain boundaries before distillation ever runs, heavyweight reasoning becomes astonishingly cheap:
| Architectural Layer | Naive Frontier-First Pipeline | 4-Tier Calibrated AyoWork Stack | Production Impact |
|---|---|---|---|
| Repetitive / Known Pattern Filter | Paid full chat LLM tokens on every line item | Tier 0 Deterministic Rust (Exact Cents, header rules) | ~40% volume settled at $0 token spend |
| Classification & Granular Routing | Generative JSON chat completions (high output token cost + drift) | Tier 1 gemini-decision-1 (0 output tokens, noul/choice/score) | ~95% backfill cost drop, sub-250ms calibrated u16 bps routing |
| Ambiguity & Audit Safety | Uncalibrated "0.99" self-scores or silent misposts | Calibrated Confidence Bands (6,500–8,999 bps HITL queue) | Deterministic workflow forks + operator review only on true uncertainty |
| Knowledge Distillation & Deep Synthesis | Flagship model wading through raw spam and duplicates | Gemini 4 Hydrogen (Vertex Scheduled Batch, 50% off) + Gemini 4 Carbon (1M ctx) | High-density 3,072-d pgvector indexing & frontier analytics at minimal cost |
Inside AyoWork today, I run scheduled overnight Vertex AI Batch Prediction sweeps on Gemini 4 Hydrogen (capturing an additional 50% scheduled batch credit discount on already-filtered datasets) to distill operational truths into AyoBrain embeddings. Meanwhile, I pin my heaviest reasoning agents and Accounting Controller diagnostics to Gemini 4 Carbon with a full 1M-token context window—analyzing cross-decade trends and multi-year financial patterns on clean, pre-structured signals for pennies per run.
This mirrors the exact pattern I enforce on the shop floor inside Physics as Code and The 5D Manufacturing Methodology: move from unverified guesswork to deterministic data logging (03 DATA LOG), enforce calibrated physical and computational constraints before actuation (04 DRIVE), and reserve top-tier reasoning compute exclusively for problems worthy of a flagship model.
Key Engineering Takeaways
- Never use a frontier generative model as a high-volume sieve: Pairing deterministic Rust pre-filters with Jev and
gemini-decision-1(0output tokens viaPOST /v1beta/decisions) cut my 16–20 year historical archive distillation cost by ~95%, making deep historical mining across2006–2026operational records routine. - Operationalize calibrated confidence as integer basis points: Calibrated probabilities parsed into zero-float
u16basis points (0..=10,000 bps) turn probabilistic model outputs into nearly deterministic workflow switches and cleanly route true edge cases to Human-in-the-Loop queues. - Pre-filtering makes frontier reasoning (
Gemini 4 Carbon) nearly free: When Tier 0 rules and Tier 1 calibrated gates strip away noise and structure metadata upstream, Gemini 4 Hydrogen batch distillation and Gemini 4 Carbon1M-context analytics run over pure signal at minimal cost.
Engineering Resources
- Google Gemini API Documentation (Decisions & Gemini Models)
- Agents Don’t Need More Autonomy. They Need a Gateway and a Memory.
- AyoWork: Orchestrating Unlinked Industrial Datasets (NSF SBIR)
- Physics as Code: Why Foundation Models Fail Without First-Principles Constraints
- AyoWork Agentic Industrial OS
- The 5D Manufacturing Methodology
- Connect with Aram Goganian (Principal Systems Architect & Engineering Leader) on LinkedIn