StackRust · gemini-decision-1 · Gemini 4 Hydrogen & Carbon · Jev · AyoWork

Turning Probabilistic AI Nearly Deterministic: How Jev and Gemini Decision-1 Cut 20-Year Knowledge Distillation Costs by 95%

How I used Jev and Google's zero-output-token gemini-decision-1 in AyoWork to backfill 20 years of shop-to-ledger archives at 95% lower cost.

Optical signal channels passing through machined dark aluminum prism gates separating raw data filaments into four deterministic routing lanes
On this page5 sections

Pointing a frontier reasoning model raw at twenty years of company history is a great way to burn cash and still end up with an uncalibrated mess.

Every mid-sized manufacturing and engineering business sits on decades of dark operational knowledge. Across Predator Cycling, MACC Composites, and AyoWork, my own archive stretches back sixteen to twenty years—all the way to 2006. That history lives across custom carbon frame build threads, composite layup work orders, supplier AP invoices, shipping discrepancies, and thousands of historical bank lines.

Five weeks ago, I started building automated harvesting pipelines to categorize and distill that entire historical repository into structured, searchable institutional memory inside AyoBrain. Throwing a large generative chat model directly at millions of raw rows failed two tests immediately: the token bill made mining twenty years back unjustifiable, and raw prose outputs drifted whenever a downstream workflow needed a crisp, repeatable fork.

This past weekend, I completed the migration onto Google’s newly released gemini-decision-1 inside AyoWork’s native GCP Rust stack—completing an architectural evolution that began with my early System-1 harvesting engine Jev and cut historical distillation costs by roughly 95%. Here is how calibrated decision models, dormant confidence scores brought back to life, and frontier models working strictly downstream turn probabilistic AI into an almost deterministic operating tool.


Harvesting 20 Years of History with Jev and the 95% Cost Collapse

Most “AI ingestion” pipelines start backward: they hand an uncurated email body, invoice PDF, or ledger description directly to a flagship generative model and pay for hundreds of output tokens just to hear "this looks like a shipping notification, skip it."

Five weeks ago, I flipped that order. Instead of sending raw history straight into expensive distillation prompts, I put lightweight deterministic Rust pre-filters in front of Jev—my early System-1 decision harvester built to classify, score, and route operational records at high speed before any generative synthesis runs. The economics follow directly from gating what reaches the heavy tier:

Ctiered=N⋅[(1−rrule) csys1+pdistill⋅cfrontier]≈0.05 CnaiveC_{\text{tiered}} = N \cdot \Big[(1 - r_{\text{rule}}) \, c_{\text{sys1}} + p_{\text{distill}} \cdot c_{\text{frontier}}\Big] \approx 0.05 \, C_{\text{naive}}

Where NN is the historical corpus size, rruler_{\text{rule}} is the fraction settled outright by zero-cost deterministic rules (~40% on repetitive bank and system notices), csys1c_{\text{sys1}} is the pennies-per-thousand cost of System-1 classification, and pdistillp_{\text{distill}} is the clean high-signal slice that actually deserves frontier distillation (cfrontierc_{\text{frontier}}).

Dropping total ingestion and distillation cost by ~95% completely changed the scope of what was viable. Backfilling two years of operational records used to feel expensive; at 95% lower cost, mining sixteen to twenty years deep back to 2006 suddenly fit inside a normal engineering budget. Custom frame geometry nuances, historical vendor lead-time anomalies, and complex tax/accounting classifications stopped sitting in cold archives and started indexing cleanly into AyoBrain’s 3,072-dimensional pgvector store.

Even more importantly, structuring granular decision rubrics for that historical harvest worked so cleanly (95.8% live intent benchmark accuracy) that I promoted the exact same pattern to how AyoWork ingests live day-to-day data. Instead of one vague prompt trying to read, classify, decide urgency, and summarize all at once, every incoming event hit focused System-1 decision checkpoints first—raising both accuracy and throughput immediately.

AyoWork 4-tier calibrated decision and distillation pipeline showing Tier 0 deterministic Rust rules, Tier 1 gemini-decision-1 zero-output-token scoring in integer basis points, Human-in-the-Loop uncertainty routing, and Tier 2/3 Gemini 4 Hydrogen and Carbon synthesis
Figure 1: AyoWork's 4-tier routing ladder. Tier 0 Rust invariants and Tier 1 gemini-decision-1 (zero output tokens, integer 0..10,000 bps gates) isolate edge cases for humans and keep Gemini 4 Hydrogen and Carbon focused strictly on high-signal synthesis.

Why Migrating to gemini-decision-1 on GCP Clicked Immediately

When Google launched gemini-decision-1, migrating my live AyoWork decision layer off external harvesting glue this past weekend was a natural architectural consolidation. AyoWork already runs natively inside Google Cloud (Cloud Run, Cloud SQL, BigQuery, and Vertex AI), so pulling System-1 decision evaluation directly into the Gemini family removed network hops, unified credential governance, and unlocked three capabilities that made the routing layer dramatically cleaner:

  1. Zero Output Tokens in a Single Multi-Question Pass: Calling POST /v1beta/decisions evaluates up to 1,000 independent questions against shared multimodal or text context in a single sub-250ms pass—emitting 0 output tokens. Instead of paying a chat model to stream JSON prose field by field, the API returns calibrated probabilities natively across three question primitives: noul (binary yes/no noise gates), choice (exact queue or chart-of-accounts selection up to 255 options), and score (ordered operational urgency or materiality rubrics).
  2. Reviving AyoWork’s Dormant Confidence Factor: Back in AyoWork’s earliest Rust ledger and triage schemas, I wired a confidence metric into suggestion tables—and then barely trusted it, because self-reported confidence scores hallucinated inside LLM chat JSON ("confidence": 0.99 on pure guesses) are famously uncalibrated. Calibrated probability distributions from gemini-decision-1 finally made that dormant field real telemetry.
  3. Zero-Float Integer Basis-Point Routing (0 to 10,000 bps): Consistent with AyoWork’s zero-float ledger rules (MoneyCents and Decimal), I never evaluate routing thresholds with f64 comparisons that silently suffer IEEE rounding drift. Every probability and confidence metric returned from v1beta/decisions is parsed via exact integer decimal scaling into u16 basis points (0..=10,000 bps) inside my typed Rust validation layer after strict PII/secret redaction:
/// High-level Rust pattern for Tier-1 calibrated routing (sanitized example).
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum DecisionRoute {
    DeterministicDispatch, // confidence >= 9,000 bps (high certainty)
    HumanInTheLoopReview,  // 6,500..=8,999 bps (ambiguous split or edge case)
    ArchiveNoiseDropped,   // binary noise gate triggered >= 8,500 bps
}

pub fn route_calibrated_signal(
    is_noise_bps: u16,
    queue_confidence_bps: u16,
    requires_synthesis: bool,
) -> (DecisionRoute, bool) {
    if is_noise_bps >= 8_500 {
        return (DecisionRoute::ArchiveNoiseDropped, false);
    }

    let route = if queue_confidence_bps >= 9_000 {
        DecisionRoute::DeterministicDispatch
    } else {
        DecisionRoute::HumanInTheLoopReview
    };

    // Only structured, non-noise operational records wake downstream distillation
    (route, requires_synthesis)
}

Just like the typed create_draft_journal_entry boundary I covered in Agents Don’t Need More Autonomy. They Need a Gateway and a Memory., gemini-decision-1 never writes database rows or picks arbitrary categories. It supplies calibrated integer probabilities at granular workflow forks, and deterministic Rust guards decide where the record goes next.


Human-in-the-Loop Gates and Practically Free Frontier Synthesis

The biggest surprise of embedding System-1 decision gates at a granular workflow level is what it unlocks downstream—both for human operators and for frontier reasoning models like Gemini 4 Hydrogen and Gemini 4 Carbon.

When uncalibrated AI handles ingestion, you either distrust everything (and drown your staff in manual review queues) or auto-approve too much (and clean up subtle ledger errors months later). By calibration-gating every workflow transition in basis points, humans only see the narrow mid-confidence slice (6,500–8,999 bps) where ambiguity legitimately exists: an unusual mixed-purpose supplier invoice, a partial composite repair split across two work orders, or an edge-case technical customer thread. Everything clearly routine resolves deterministically at high confidence; everything genuinely unclear lands cleanly in a one-click operator workbench.

Even better, because Tier 0 rules and Tier 1 gemini-decision-1 gates strip out noise and pre-tag structured entities, urgency, and domain boundaries before distillation ever runs, heavyweight reasoning becomes astonishingly cheap:

Architectural LayerNaive Frontier-First Pipeline4-Tier Calibrated AyoWork StackProduction Impact
Repetitive / Known Pattern FilterPaid full chat LLM tokens on every line itemTier 0 Deterministic Rust (Exact Cents, header rules)~40% volume settled at $0 token spend
Classification & Granular RoutingGenerative JSON chat completions (high output token cost + drift)Tier 1 gemini-decision-1 (0 output tokens, noul/choice/score)~95% backfill cost drop, sub-250ms calibrated u16 bps routing
Ambiguity & Audit SafetyUncalibrated "0.99" self-scores or silent mispostsCalibrated Confidence Bands (6,500–8,999 bps HITL queue)Deterministic workflow forks + operator review only on true uncertainty
Knowledge Distillation & Deep SynthesisFlagship model wading through raw spam and duplicatesGemini 4 Hydrogen (Vertex Scheduled Batch, 50% off) + Gemini 4 Carbon (1M ctx)High-density 3,072-d pgvector indexing & frontier analytics at minimal cost

Inside AyoWork today, I run scheduled overnight Vertex AI Batch Prediction sweeps on Gemini 4 Hydrogen (capturing an additional 50% scheduled batch credit discount on already-filtered datasets) to distill operational truths into AyoBrain embeddings. Meanwhile, I pin my heaviest reasoning agents and Accounting Controller diagnostics to Gemini 4 Carbon with a full 1M-token context window—analyzing cross-decade trends and multi-year financial patterns on clean, pre-structured signals for pennies per run.

This mirrors the exact pattern I enforce on the shop floor inside Physics as Code and The 5D Manufacturing Methodology: move from unverified guesswork to deterministic data logging (03 DATA LOG), enforce calibrated physical and computational constraints before actuation (04 DRIVE), and reserve top-tier reasoning compute exclusively for problems worthy of a flagship model.


Key Engineering Takeaways

  • Never use a frontier generative model as a high-volume sieve: Pairing deterministic Rust pre-filters with Jev and gemini-decision-1 (0 output tokens via POST /v1beta/decisions) cut my 16–20 year historical archive distillation cost by ~95%, making deep historical mining across 2006–2026 operational records routine.
  • Operationalize calibrated confidence as integer basis points: Calibrated probabilities parsed into zero-float u16 basis points (0..=10,000 bps) turn probabilistic model outputs into nearly deterministic workflow switches and cleanly route true edge cases to Human-in-the-Loop queues.
  • Pre-filtering makes frontier reasoning (Gemini 4 Carbon) nearly free: When Tier 0 rules and Tier 1 calibrated gates strip away noise and structure metadata upstream, Gemini 4 Hydrogen batch distillation and Gemini 4 Carbon 1M-context analytics run over pure signal at minimal cost.

Engineering Resources