Skip to content
CloudMagicTalk to us

Financial Services

Illustrative example

Rebuilding document AI on Gemini, at 60% lower cost

A specialty insurance underwriter

An underwriter's OpenAI and Bedrock stack, consolidated onto Gemini inside their own cloud

60%
lower cost per document than the previous LLM stack
Industry
Specialty insurance
Scale
2M documents backfilled, hundreds of pages per claim
Platform
Gemini Enterprise Agent Platform
Migration
6 weeks, evaluation-gated, dual-run

The challenge

  • Claim files ran to hundreds of pages, so the OpenAI pipeline chunked every document into a retrieval index before a model ever saw it
  • Three model vendors meant three bills, three rate limits, and three security reviews
  • Cost per claim rose with volume, and finance could not predict the unit economics
  • Documents left the cloud perimeter for external APIs, which audit flagged every quarter

What it had to do

  • Read whole claim files without a chunking pipeline
  • Match or beat extraction accuracy on a measured golden set before any cutover
  • Keep every document inside the existing Google Cloud perimeter
  • Make cost per document predictable at twice the current volume

What we built

We rebuilt the workload on the Gemini Enterprise Agent Platform behind the client's existing abstraction layer, so application code never changed. Gemini's long context reads a full claim file in one pass, police reports, medical records, adjuster notes, and scanned photographs together, which retired the chunking and retrieval pipeline outright. A model cascade routes each document by confidence: Gemini Flash handles the bulk path, and the long-context Pro model takes the complex files, with thresholds tuned against the golden set. Context caching holds the recurring policy corpora, so repeated pages stop being billed as fresh input, and the two-million-document backfill ran through batch predictions at half the online rate. Cutover was gated on a 1,200-document evaluation harness scored by extraction F1, dual-run against the old stack for two weeks.

Long context, no chunker

Whole claim files in one model call. The retrieval index, its sync jobs, and 2,800 lines of chunking code were deleted.

Model cascade with confidence routing

Flash for the bulk path, Pro for the hard files, human review below threshold. Cost follows difficulty instead of averaging it.

Context caching and batch pricing

Recurring policy corpora are cached instead of re-billed, and the backfill ran as batch jobs at half the online rate.

Evaluation-gated cutover

A 1,200-document golden set with extraction F1 as the gate. The new stack shipped only after it beat the old one.

Reference architecture

Sources

  • Claim intake
  • Document store
  • Policy corpora

Routing

  • Model cascade router
  • Confidence thresholds

Models

  • Gemini Flash
  • Gemini Pro, long context
  • Context cache

Assurance

  • Evaluation harness
  • Human review queue
  • BigQuery analytics

Results

  • Cost per document fell 60% against the previous multi-vendor stack
  • Extraction F1 rose from 91 to 94 on the golden set
  • p95 latency dropped from 14 seconds to 4 seconds
  • Every document now stays inside the client's cloud perimeter, on one invoice

For claims teams

A full claim file summarized with citations in seconds, not minutes.

For engineering

One API surface and no chunk tuning. The pipeline they maintain got smaller, not bigger.

For finance

A predictable unit cost per claim, on the same bill as the rest of the cloud estate.

  • Gemini Enterprise Agent Platform
  • Gemini Pro
  • Gemini Flash
  • Context caching
  • Batch predictions
  • BigQuery
  • Cloud Storage
  • VPC Service Controls
  • Cloud KMS

Facing something similar?

Tell us where you are now. A senior engineer replies, usually within one business day.

Talk to a customer engineer