Skip to content
DineshKumar Sarangapani
All writing

Building an Enterprise MCP Tool Broker: Turning OpenAPI into Agent Tools

How to connect AI agents to hundreds of existing internal APIs dynamically using the Model Context Protocol.

AIMCPArchitectureAgentic-AI

AI agents are only as useful as the tools they can call. An agent without tools can only chat. An agent with tools can query databases, create tickets, inspect files, and trigger deployments.

In most enterprises, those capabilities already exist as REST APIs. Over the last decade, organizations built hundreds of internal microservices documented with OpenAPI.

When we started connecting agents to our internal systems, we faced a choice. Should we manually write dedicated tool wrappers for every API? Or could we build a broker that translates existing OpenAPI specifications into agent tools automatically?

We chose to build a centralized tool broker using the Model Context Protocol (MCP). Here is how the idea works, how it performed in production, and what to watch out for.

The Idea: A Dynamic Tool Broker

Writing dedicated tool servers for hundreds of microservices does not scale. Each engineering team would need to learn a new protocol, maintain extra code, and keep tool schemas manually synchronized with API updates.

The core idea was to build a single enterprise tool broker.

The broker sits between AI agents and internal APIs. To the agent, the broker looks like an MCP server offering a unified catalog of tools. Behind the scenes, the broker ingests existing OpenAPI specifications from registered services and dynamically translates each API operation into a typed, callable tool.

When an agent invokes a tool, the broker validates the caller’s identity, translates the arguments into an HTTP request, calls the backend service, and returns the formatted result back to the model.

How It Worked Well

  1. Zero Work for Service Teams: Existing microservices did not need code changes. As long as a team maintained an accurate OpenAPI specification, their service became agent-callable instantly.
  2. Standardized Tool Discovery: Instead of teaching every agent about individual services, agents connect to one broker endpoint. The broker dynamically filters which tools an agent can see based on the caller’s permissions and role.
  3. Decoupled Architecture: Upstream microservices remain pure REST APIs. If the agent protocol changes or new client interfaces emerge, the translation logic lives in the broker rather than scattered across dozens of repositories.
  4. Governed Access Controls: The broker enforces centralized authorization before executing any backend request. We can enforce least-privilege access, audit tool invocation logs, and disable high-risk actions without modifying the underlying microservices.

What to Watch Out For

  1. Vague Endpoint Summaries Confuse Models: Large language models pick tools based on natural language descriptions. If an API has a generic description like Process data, the model will either hallucinate its purpose or never call it. Clear, descriptive summaries in your API specs are critical for agent accuracy.
  2. Schema Reference Inlining ($ref): Many enterprise API specifications use internal schema references ($ref) to share data types across endpoints. Many LLM engines and agent frameworks cannot resolve external reference pointers automatically. The broker must resolve and inline referenced schemas before handing tool definitions to the model.
  3. Circular Schemas: Real-world enterprise APIs frequently contain circular schema definitions (for example, a Folder that contains a list of Folder items). If your schema resolver does not set recursion depth limits, it will enter an infinite loop or produce massive schemas that exhaust the model’s context window.
  4. Context Window Blowups from Large Payloads: A GET endpoint that returns 10 megabytes of JSON is fine for a frontend data table, but it will overflow an LLM’s context window. The broker must detect oversized responses, summarize them, or paginate results before returning them to the agent.
  5. Write Actions Need Confirmation: Read-only tools (searching data, fetching records) are safe to execute autonomously. Write tools (deleting resources, triggering updates) should require explicit human confirmation or stricter permission gates before execution.