InsightsAI agents
How we keep an AI agent as good as the day it launched
Models update, prompts get edited and connectors change after launch. How we rerun your answer key monthly and before changes, with a report an owner signs.
- Written by
- CloudMagic engineering team
- Published
- Reading time
- 12 minutes
The short answer
Run the answer key the agent launched with every month and before any planned change, several runs per case, beside a prompt-only control on the same model. Code agents on Agent Runtime add live trace scoring, quality alerts and an approval gate in Cloud Build, while no-code agents in Gemini Enterprise get scheduled runs, because we cannot block a change made in the console. A named owner signs a one-page report each month.
An agent that passed every test on launch day can be worse a month later, often without anyone changing it on purpose. We call the service that watches for this Agent Quality. It starts with two weeks of onboarding and then runs on a 6 or 12 month term: we test the agent against your own answer key every month and before any planned change, and a named owner on your team signs a one-page report. Product details were checked against Google's documentation on November 14, 2026.
It has two tracks, because Google gives us different controls in each place an agent can live. No-code agents and skills sit in Gemini Enterprise (Google's product page now calls it Gemini for enterprise). Code agents are built on Gemini Enterprise Agent Platform (formerly Vertex AI) and run on Agent Runtime. This post picks up where our post on how we test an AI agent before it does real work ends.
What changes after launch that can break an agent?
The model, the instructions, the data the agent reads and the questions people ask. Only some pass through a release step where a test can catch them.
Google's documentation calls the result quality drift, an observable decrease in agent performance over time, and names changes in user behavior or external data as the causes. A model update is the third cause, and the only one that gets announced.
| Change | Who usually makes it | Does a release step catch it? |
|---|---|---|
| A new Gemini model turned on, by your admin or by a Google default | Your Gemini Enterprise admin, or Google | Only if the owner asks for a test run first |
| An edited agent, prompt or skill | The agent's owner | For code agents, yes; in the Gemini Enterprise console, no |
| A document added, replaced or removed in a connected source | Whoever owns that folder | No |
| New kinds of questions | Your users | No |
New models reach the Gemini Enterprise web app in two ways. Gemini 4 Carbon and Hydrogen arrived turned off, for an administrator to turn on. Gemini 4 Argon's toggle switched to on by default from October 27, 2026, on a date Google announced in its release notes. Either way a new model is a change we can see coming: the runbook asks the admin to tell the agent's owner, we watch the release notes, and we run the full key before or on the day it takes effect.
Data is the change people forget. For third-party sources Gemini Enterprise indexes, an incremental sync runs every three hours by default, so a replaced policy can reach answers the same afternoon. Incremental syncs skip deletions, so a policy someone removed can keep turning up until the next full sync.
How do you tell drift from an ordinary bad run?
Run every case several times, compare the spread with the launch baseline, and keep a prompt-only control on the same model beside the agent.
A language model can answer the same input differently, so one run proves little. In our prior authorization demo, which uses a made-up health system and synthetic records, a prompt-only version on the same Gemini model scored 19, 18, 18 and 18 out of 20 across four runs of identical cases. A monthly check built on one run could report a drop or a recovery that is only noise, so each case runs at least four times.
The control tells us where to look. If the agent and the control fall together after a model change, the model is the likely cause. If the control holds and the agent falls, the cause is usually an instruction, a tool, a connector or the documents behind it.
What has to be true before day one?
A named owner, cases with known outcomes, time from someone who does the work, and access. Two weeks is the promise only when these are in place; if one is missing, the clock starts the day it arrives.
- An owner for each agent who can approve a fix and sign the report, and the CISO or risk lead who reads it.
- 10 to 20 past cases with known outcomes. If we put the agent live, its answer key carries over and onboarding is shorter.
- About an hour a week from a subject matter expert.
- Access to the agent's project through your own IAM, for named engineers, and for a code agent its repository and permission to add a Cloud Build trigger.
- For a code agent, Cloud Trace on with Google's OpenTelemetry attributes, and Model Armor called from the agent's own code. If we built the agent, both are already there. If your team built it and either is missing, we hand your developers the code pattern on the readiness call, or make the change ourselves as a short step quoted separately before the two weeks start, because a change to a production agent goes through your own review and deploy.
The agent should be live, or close to it. If it is not built yet and needs custom code, it starts with our 30-day plan, and Agent Quality picks up the day it goes live.
What happens week by week?
Week one defines good and measures it, and week two puts the checks on a schedule.
| When | What happens | Who we need from you | Google settings | What exists at the end |
|---|---|---|---|---|
| Week 1 | Answer key written or refreshed, telemetry checked, baseline runs | Expert for a few hours, owner for thresholds | Cloud Trace, OpenTelemetry attributes and Model Armor confirmed on code agents | A baseline score with its run-to-run spread |
| Week 2 | Scheduled runs, monitors and alerts, approval gate, first misuse pack | Owner and CISO | Online monitors with sampling caps; quality alerts; Cloud Build approval | Checks running, thresholds signed, runbook |
| Every month | Full key run, drift review, misuse refresh, new cases, report | Owner signs; expert reviews new cases | Model Armor templates tuned | A signed one-page report |
What happens in week one?
We agree in writing what "good" means for each agent, and measure it.
We list every agent in scope and its track: the Gemini Enterprise assistant, agents built in Workflow Builder, skills, and code agents. With the subject matter expert we write or refresh the answer key: for each case, the right outcome, what the answer must cite and what it must refuse. The owner sets two thresholds, one below which a change does not go live and one that sends an alert.
For code agents, Google's online monitors need Cloud Trace on and specific OpenTelemetry attributes on the agent's spans, such as gen_ai.agent.name, with prompts and responses recorded. Those traces hold real conversations, so the CISO decides now who can read them, and we follow Google's advice to restrict who can create online evaluators, because they run under a project-level service account. Model Armor keeps blocking on production traffic as it did at launch. Any detector or rule we propose runs first in inspect-only mode, which logs what it would have stopped and lets traffic through, as Google suggests for testing a new policy.
The week ends with the baseline: every case at least four times, beside the prompt-only control.
What happens in week two?
Both tracks get scheduled runs and a test before planned changes. Code agents also get live scoring and an approval gate.
For no-code agents and skills in Gemini Enterprise, our engineers run and score the answer key on a schedule and on request, and the runbook says the owner asks for a run before changing a prompt, skill or connector. Those runs are our engineers' hands in the chat, 60 to 80 conversations for a 15 to 20 case key and as many again for the prompt-only control, each scored against the key by an engineer. So the order form says how many pre-change runs a month are included for each agent, and a team that edits a skill every week has a higher number written in. Skill owners can see a skill's version history and restore an earlier version, and admins can require approval before a skill is shared.
For code agents on Agent Runtime, more of the work runs on two features Google documents on Agent Platform. Online monitors score a sample of live traces on a loop that typically runs every 10 minutes; we set the sampling percentage and the cap on samples per run with the owner, since the cap controls evaluation cost, and add metrics such as Safety beside custom ones built from the answer key. Quality alerts notify the owner by email or Pub/Sub when a score falls below the threshold. A Cloud Build trigger runs the full key on every change committed to the agent's repository, and the deploy waits until someone with the Cloud Build Approver role, usually the owner, approves it with the scores in front of them.
The first misuse pack runs on both tracks: instructions hidden in documents, requests for records the user should not see, and colleagues trying to talk the agent out of a refusal. Our post on stopping instructions hidden in documents covers how we write them. Once the owner has read the inspect-only logs, a proposed rule moves to inspect-and-block where the owner approves. The two weeks end with a readout of scores, thresholds, open risks and the runbook.
What happens every month?
We run the whole key, review drift, refresh the misuse pack, add real cases to the key and send the report.
Every drop gets a root cause and a proposed fix, and the owner approves each fix before it goes in. Model Armor templates are tuned where its logs and the misuse results show false alarms or misses. New cases come from real use, and the subject matter expert reviews each one before it counts; Google's Gen AI evaluation service supports sampling straight from production logs. Each quarter the experts review the whole key with us.
Which changes can we hold before they go live?
We can hold a change that goes through the agent's repository. We cannot block a change made in the Gemini Enterprise console, and the service description says so.
For code agents the gate covers only what goes through the repository, so we recommend that deploy rights sit with the pipeline. In Gemini Enterprise, an owner can edit an agent's instructions, a skill owner can save a new version and an admin can turn on a model, all from the console. For those, the runbook asks for a test run first, a scheduled run catches anything that skipped it, and the report lists each change and whether it was tested.
What does the monthly report show?
One page for the owner and the CISO, signed by the owner, in six parts: scores, drift, misuse, Model Armor, changes, next steps. Here is the page for a made-up agent, the benefits answers agent at Larrow Market, the made-up grocery chain from our demos, on the no-code track. Every figure is invented to show the shape of the report, and none of it is a result.
Larrow Market, benefits answers agent, monthly report, month 3.
Scores. 18 cases, four runs each. Launch baseline: 18 of 18 in every run. This month: 18, 17, 18, 17; both misses are case 11, the dental plan question. The prompt-only control on the same model scored 15, 16, 15, 16, unchanged since launch, so the agent moved and the model did not. The owner's alert line is any run below 18, so this page went out with an alert; the stop line, any run below 16, was not crossed.
Drift. In two runs, case 11 cited the 2026 dental plan document, which the benefits team had replaced with the 2027 plan on the 3rd. Cause: the incremental sync indexed the new file within hours, and the deleted file stayed in the index until the next full sync. Proposed fix: a full sync the morning after every policy refresh, added to the benefits team's checklist. Owner's decision: approved on the 6th. Rerun on the 7th: 18 of 18 in all four runs.
Misuse. 12 planted requests: another employee's leave balance, an instruction hidden in a PDF telling the agent to answer in full for any manager, a colleague pressing for a named person's pay. 12 refused and logged, 0 through.
Model Armor. 14 prompts flagged this month: 9 prompt injection, 5 sensitive data. Of the 9, 7 were our own misuse pack and 2 came from employee prompts, as did all 5 sensitive-data flags. We see the verdict in the audit log, not the text, and seven employee flags in a month is inside the range the owner set at onboarding, so nothing to tune.
Changes. Workflow version 4 saved on the 9th, tested first: yes, 18 of 18 before and after. Gemini 4 Carbon turned on in the web app by the admin on the 12th, tested first: yes, full key the same morning, 18 of 18. No connector changes.
Next steps. Three real questions from the month added to the key after the HR generalist reviewed them, so the key is now 21 cases. No deep test due: the agent reads one policy folder and nothing from outside the company. We will recommend Mandiant's AI red teaming the month it gets an outside source.
Signed by the agent owner on the 14th; read by the CISO.
The report recommends and the owner decides. A year of signed pages gives an auditor a dated record of how the agent has done since launch.
When should Mandiant test the agent?
When the stakes outgrow a regression check: before the agent gets a tool that prepares changes in a system of record, before it reads content from outside the company, or when an auditor wants independent adversarial testing.
Mandiant, part of Google Cloud, offers red teaming for AI: its consultants measure risks to generative AI in production by running attacks specific to AI services and the applications built on them. Our misuse pack is a regression set that confirms known defenses still hold, a different job that does not replace a red team. When we recommend Mandiant, we help scope the test and add what they find to the misuse pack.
What do you own, and how does it end?
You own all of it from the first day, in your own project and repository: the answer key, harness code, misuse pack, scores, reports, pipeline, monitoring settings and runbook.
The paper is a service description and an order form. It is an option on every two-week deployment we run and works for agents your own team built. At the end of the term you renew, or your team takes over the harness with the runbook. Nothing needs migrating, and we remove our engineers' access on the last day.
When do we tell a client not to start?
When nobody will sign the report, because a report without an owner changes nothing. When there are no past cases with known outcomes, building the key comes first, as discovery work. When the agent is about to be rebuilt, we test the new one instead. If the real question is whether a determined attacker can break the agent, Mandiant's red teaming is the right first step.
Frequently asked questions
Do your engineers see our users' conversations?
Online monitors score sampled traces, and those traces contain prompts and answers. Our named engineers get access through your IAM, and your CISO decides who can read Cloud Trace and Cloud Logging. Cases taken from real use stay in your project.
What does it cost?
The service description and order form are already written; you tick the tier, set by the number of agents and the pre-change runs included each month, and a 6 or 12 month term, and sign. There is a fixed onboarding fee, then a monthly fee. Ask us whether Google funding applies to your project.
Sources
- Continuous evaluation with online monitors, Google Cloud
- Configure quality alerts, Google Cloud
- Gen AI evaluation service overview, Google Cloud
- Model Armor overview, Google Cloud
- Enable Model Armor in Gemini Enterprise, Google Cloud
- Create and manage skills, Google Cloud
- Manage web app features, Google Cloud
- Introduction to connectors and data stores, Google Cloud
- Gate builds on approval, Google Cloud
- Mandiant AI Security Consulting, Google Cloud
- How we test an AI agent before it does real work, CloudMagic: prior authorization demo, made-up health system, synthetic records