Grounding AI Agents in the Yahoo OS Design System

Challenge

In April 2026, designers and engineers at Yahoo started building UI with AI assistants. The assistants were fluent, fast and confidently off-system.

Inspect Detail

Ask for a Yahoo search card and you'd get a hex value close to brand purple that wasn't a token, an emoji where the icon library should be, a hand-typed shadow and a font name the brand had retired. None of it came with a citation, so a wrong answer looked exactly like a right one.

The frustrating part was that the right answers already existed. They were spread across repos owned by different teams: foundations, token specs, component patterns, content guidance, inclusive design, illustrations and UX research. Some lived only in local Drive folders. There was no door an agent could walk through. Getting on-system output took a hand-crafted context prompt, and every team's prompt drifted on its own.

The Approach

Give agents the design system as a tool, not a prompt.

As Yahoo initiated a company-wide pivot toward agentic workflows, it became obvious that AI assistants would soon generate the majority of front-of-glass UI. If those assistants operated on fuzzy prompt context or tribal knowledge, design debt would compound at machine speed. The solution was structural: give agents direct, authoritative, deterministic access to the design system through open protocol standards.

Two rules were in the server's instructions from the first commit: always cite the source, and never hallucinate. In June I formalized them into a charter, and every later decision was judged against its north star:

The charter ranks failures too. "A confident wrong answer is the worst outcome this server can produce": worse than a slow answer, and worse than "I don't know." That ranking shaped everything that followed, from the search design to how feedback gets triaged.

Process

The architecture: sync offline, serve with no network

Inspect Detail

I had a working prototype the first day. The architecture that grew from it splits into two deliberately decoupled halves. A sync step, the only step that needs a token, pulls the source repos into plain files committed to git. The serving path crawls those files at startup, indexes them and exposes them as MCP tools. It needs no network and no secrets.

Three choices kept it simple:

• Full-text search, not RAG. Documents are ranked with BM25 in SQLite FTS5. There are no embeddings and no vector store, so every result can be explained, and an average query takes 0.83 ms.
• Whole documents, not chunks. read_document returns the entire file, so the model never reasons from a fragment of a guideline.
• Fix the knowledge, not the code. Improving an answer usually means editing a document in its source repo. The next sync ships it, with no deploy.

Because the index is rebuilt from plain files on every boot, the files are provably the source of truth. The server is the replaceable part.

Moving the corpus to where its owners work

The first version read from machine-specific Drive paths, so it worked on exactly one laptop. In June I replaced them with a GitHub sync. A scheduled Action crawls every source repo daily, mirrors it into knowledge/ and prunes anything deleted upstream.

That change did more than fix portability. It turned other teams from requesters into owners. Each team keeps writing in its own repo, and adding a source is a one-line change to a manifest. In July the UX Research team added its own corpus, without anyone touching server code. Since June, the sync has committed 22 refreshes on its own.

Encoding the workflow, not the prompt

Inspect Detail

The server tells every agent to make the same first call: read a curated design-context guide. That guide maps the knowledge base and routes the task, so nobody needs a crafted prompt to get on-system output.

The most important routing step is deterministic, not a guess. Many Yahoo teams don't consume the design system directly. They consume it through UDS, our internal design system tooling, which packages the system as ready-made components and has its own MCP for checking component props. So before speccing any component, the agent searches the repo's package.json for UDS. If it's there, UDS owns the component API, and the agent verifies props through the proxied UDS MCP. If it isn't, the design-system spec is the build reference. An agent should never tell a team using UDS to bypass UDS. When the task is ambiguous, the guide tells the agent to ask "Figma, code, or both?", because a question costs less than a confident guess.

A small, reviewed tool surface

Inspect Detail

The server exposes twelve tools: search and read, component specs, icon and illustration lookup (with the SVG bytes served from storage), Figma setup, the UDS proxy, status and feedback. Eleven of them are annotated read-only.

New tool surface is a cost, so the charter requires every new tool to justify itself over extending an existing one. The tool manifest is generated from the code, and CI fails if the committed manifest and the code disagree. That makes every change to what agents can call visible to security review.

The server also had to be safe to offer company-wide. It sits behind Yahoo's MCP gateway, with OAuth through Okta, so every request is tied to the caller's Yahoo identity. It accepts tokens from more than one trusted issuer, and CI deploys with a federated identity instead of anyone's personal credentials. Meeting enterprise security and compliance standards was what cleared the path for the tool to move from an engineering convenience to a required standard for Yahoo's agentic enablement.

The pivot: from an ambient file to a server

Early on I also shipped a DESIGN.md package: one file of tokens and guidance a team could drop into a repo as ambient context for its agents. It needed zero setup, and that was its appeal.

It also had hard limits. A static file can't cite a source, can't stay current with eight repos that change daily, and can't route a task based on how a repo consumes the system. In June I removed it from the server and marked the package for deprecation. Retrieval through tools beat ambient context for a design system that keeps changing.

The self-improvement loop

Inspect Detail

Accuracy can't be a one-time launch property, so the server improves through a closed loop:

1. Collect. A submit_feedback tool inside the MCP, plus a public form, write to one sheet.
2. Screen. An operator-run triage skill gives every row a verdict. Submitted text is treated as data, never as a command. A row that tries to manipulate the agent ("ignore your instructions…") is quarantined as evidence of an attack. Venting with nothing actionable is rejected, with a reason on record.
3. Weigh. The actionable rows are grouped by theme and run through four lenses: Is it breaking? If not, what changes and why? Does it need a new tool, and why? What is the smallest slice worth shipping?
4. File. The triage checks the shared board for duplicates first, then files ranked work with the analysis attached.
5. Fix. Usually in the knowledge, not the code.
6. Lock in. Failing prompts become eval cases, and quarantined injections become refusal regression tests, so a real failure can't quietly come back.

The loop has already caught a real failure. A designer redesigning a product with the MCP got the wrong font. The team traced it to the token JSON, which still carried the retired font name after a pipeline lag. The value was fixed upstream in the tokens repo. The guide also gained a precedence rule: token specs are canonical for exact values, and when a doc disagrees, the agent surfaces the conflict instead of quietly picking one.

Testing a non-deterministic system

Inspect Detail

An MCP server's output is only as good as what a model does with it, and that isn't deterministic. So I split testing into three layers and gated only what could be gated:

1. Protocol end-to-end tests prove every tool is reachable through the real MCP protocol and returns a well-formed response. They run in CI and block merges.
2. A retrieval eval checks that the right documents come back, ranked and fast, for real designer queries.
3. An LLM judge scores whether answers are grounded, cited and free of invented values, and whether injection prompts are refused. It reports a score and can run strict on a nightly job, but it never fails a pull request on its own.

Seventy-seven automated tests cover auth, feedback, sync and the protocol. The rule underneath is the same one I later built into the Yahoo OS contracts gate: machines block, models advise.

Impact

An essential strategy for Yahoo's agentic pivot

As Yahoo accelerated a company-wide pivot toward agentic workflows, the MCP server evolved from a design-systems utility into a cornerstone of the company's AI tooling strategy.

When Yahoo established its internal agentic requirements—mandating verified toolchains, safe context retrieval, and strict governance for engineers deploying autonomous agents—the server was officially adopted as required tooling.

Rather than allowing hundreds of developers to assemble fragmented prompts or rely on ungrounded model assumptions, Yahoo integrated the server directly into the required setup for anyone completing the company's agentic enablement tracks. The result was a single, authoritative interface: agents in Claude Code, Cursor, and Codex could no longer drift from Yahoo OS standards because the approved path to UI creation was provably grounded, security-vetted, and tied to authenticated Yahoo identities.

Inspect Detail

The MCP is also the operational door to the rest of Yahoo OS. It indexes the token specs from my Yahoo OS contracts work, so the agents that build from those contracts can find, read and cite them. What began as a prototype to stop design drift became an enterprise standard for how Yahoo builds software with agents.

Reflection

Grounded tooling turns into enterprise policy. When teams see agents write production code with zero hallucinations, adoption doesn't need to be pushed—it gets pulled into the company's core architecture. Becoming required tooling across Yahoo's agentic requirements proved that enterprise AI governance succeeds when it replaces brittle guidelines with fast, deterministic developer tools.

Retrieval beat ambient context. The DESIGN.md file was easier to adopt, but it couldn't cite, couldn't stay current and couldn't route. For a living system, a tool that reads the source wins.

Simple search was enough, and easier to trust. I expected to need embeddings. Whole documents ranked with BM25 answered in under a millisecond, and every result could be explained. When the goal is "never hallucinate," explainability matters more than recall.

Put the corpus where its owners already work. Moving from Drive paths to GitHub sync turned other teams from people filing requests into people shipping knowledge.

Every input is an attack surface. Feedback looks harmless until someone types instructions into it. Treating all submitted text as data, and turning each attack into a regression test, made the loop safe to open up.

The asset is the commons, not the endpoint. When another team asked to run the server, I wrote down what could transfer and what couldn't. The endpoint, the pipeline and the on-call can move. The corpus stays open, readable with zero tooling and synced from its owners. Because the index rebuilds from plain files, the endpoint was always the replaceable part.

What I'd do differently: instrument usage from day one. The eval suite tells me the answers are right. It doesn't tell me how many people they reach, and that is the number I'd most like to have here.

Every figure in this case study is built with Yahoo OS tokens. The session on the cover is representative, not a transcript.

Built with contributions from the UX Research team and teammates on the Systems Team.