My current product has hundreds of APIs. Every time I need to refer to the API specification, I have to navigate to the Swagger link, scroll endlessly, or use browser search to find what I need—and then manually filter things out. It’s frustrating and painfully slow. Even worse, every developer who needs to integrate with the API has to go through the same experience. This frustration led me to explore how AI could improve the process. This post is a deep dive into that journey and how it evolved into something simple, robust, and effective. Preparation Rich API Documentation As a first step, I reviewed entire api documentation to make sure it has clear summary and description details in the swagger docs. This is a critical step for better discoverability in later stages. I also made sure the summary and description are unique for each api. paths": {

    "/users/accounts/{account-id}": {

      "put": {

        "tags": [

          "Account API"

        ],

        "summary": "Update Test suite by test-suite-id",

        "externalDocs": {

          "description": "Robust get account details description",

          "url": "https://mydocs.com/description"

        }, paths": {

    "/users/accounts/{account-id}": {

      "put": {

        "tags": [

          "Account API"

        ],

        "summary": "Update Test suite by test-suite-id",

        "externalDocs": {

          "description": "Robust get account details description",

          "url": "https://mydocs.com/description"

        }, Categorization of the APIs With hundreds of APIs available, it can be challenging to identify which ones are related. Categorizing the APIs makes management more efficient and later simplifies selecting the right API based on natural language input. This categorization was implemented using the tagging concept defined in the OpenAPI specification. "paths": {

    "/users/accounts/{account-id}": {

      "put": {

        "tags": [

          "Account API"

        ],

        "summary": "Update Test suite by test-suite-id",

        "externalDocs": {

          "description": "Robust get account details description",

          "url": "https://mydocs.com/description"

        }, "paths": {

    "/users/accounts/{account-id}": {

      "put": {

        "tags": [

          "Account API"

        ],

        "summary": "Update Test suite by test-suite-id",

        "externalDocs": {

          "description": "Robust get account details description",

          "url": "https://mydocs.com/description"

        }, Building Natural Language Search User Input Users would enter the natural language question related to the API. Example: How to retrieve account details? Classify Category At this stage, the question and all the available categories are sent to the LLM. The LLM is tasked to return the high-level category the question falls into. The output of this step is one of the categories. Classify Specific API Based on the LLM’s identified category, the system sends another request to the model with the same question — but this time, it includes all API details within that specific category detected in the previous step. This is where the earlier preparation pays off: the more descriptive and well-structured the API documentation, the better the results. Clear descriptions help the LLM accurately determine which API the user is requesting information about. The output of this step is a single, specific API. Enrich API Response Details The OpenAPI specification of the API is then provided to the LLM to generate a detailed, context-rich description of the API alongside the original question. For example, if the user asks, “How can I retrieve account details using an account ID?”, the response will include the relevant specification details of the Account API. “How can I retrieve account details using an account ID?” Extention With the system’s enhanced ability to accurately detect the appropriate API, users can now go a step further — generating code snippets to interact with various APIs directly. For example: “Share Python code to call the Get Account Details API for a given ID.”
“Provide a cURL command to fetch account details by ID.”
“Generate a Go client to retrieve account details for a specific ID.” “Share Python code to call the Get Account Details API for a given ID.” Get Account Details “Provide a cURL command to fetch account details by ID.” “Generate a Go client to retrieve account details for a specific ID.” Lessons Learned and Insights Rich documentation is imperative for better accuracy when working with AI systems. Precise, clear, and to-the-point documentation is essential for robustness. Bonus: We also used LLM to generate a summary and description for each API, which helped immensely.
Categorize First

Why: With hundreds of APIs, categorization reduces cognitive load and improves retrieval.
How: Group related APIs into a small set of clear categories. AI systems perform better when the label space is limited.
Scale tip: If the catalog is very large, add sub-categories for finer routing.


Build Iteratively

Start small: Take a subset of the spec and train/validate a router that can reliably select the correct API.
Expand gradually: Add more APIs over time, measure accuracy, and prioritize areas with misclassifications.
Focus: Optimize precision/recall rather than breadth at the outset.


Close the Loop with Users

Collect feedback: Capture cases where the system picked the wrong API.
Act on signals: Refine the misidentified APIs’ descriptions, summaries, and tags; clarify overlapping scopes.
Repeat: Re-evaluate after each change to confirm that accuracy improves and regressions are avoided. Rich documentation is imperative for better accuracy when working with AI systems. Precise, clear, and to-the-point documentation is essential for robustness. Bonus: We also used LLM to generate a summary and description for each API, which helped immensely. Categorize First

Why: With hundreds of APIs, categorization reduces cognitive load and improves retrieval.
How: Group related APIs into a small set of clear categories. AI systems perform better when the label space is limited.
Scale tip: If the catalog is very large, add sub-categories for finer routing. Why: With hundreds of APIs, categorization reduces cognitive load and improves retrieval.
How: Group related APIs into a small set of clear categories. AI systems perform better when the label space is limited.
Scale tip: If the catalog is very large, add sub-categories for finer routing. Why: With hundreds of APIs, categorization reduces cognitive load and improves retrieval. Why: How: Group related APIs into a small set of clear categories. AI systems perform better when the label space is limited. How: Scale tip: If the catalog is very large, add sub-categories for finer routing. Scale tip: sub-categories Build Iteratively

Start small: Take a subset of the spec and train/validate a router that can reliably select the correct API.
Expand gradually: Add more APIs over time, measure accuracy, and prioritize areas with misclassifications.
Focus: Optimize precision/recall rather than breadth at the outset. Start small: Take a subset of the spec and train/validate a router that can reliably select the correct API.
Expand gradually: Add more APIs over time, measure accuracy, and prioritize areas with misclassifications.
Focus: Optimize precision/recall rather than breadth at the outset. Start small: Take a subset of the spec and train/validate a router that can reliably select the correct API. Start small: Expand gradually: Add more APIs over time, measure accuracy, and prioritize areas with misclassifications. Expand gradually: Focus: Optimize precision/recall rather than breadth at the outset. Focus: Close the Loop with Users

Collect feedback: Capture cases where the system picked the wrong API.
Act on signals: Refine the misidentified APIs’ descriptions, summaries, and tags; clarify overlapping scopes.
Repeat: Re-evaluate after each change to confirm that accuracy improves and regressions are avoided. Collect feedback: Capture cases where the system picked the wrong API.
Act on signals: Refine the misidentified APIs’ descriptions, summaries, and tags; clarify overlapping scopes.
Repeat: Re-evaluate after each change to confirm that accuracy improves and regressions are avoided. Collect feedback: Capture cases where the system picked the wrong API. Collect feedback: Act on signals: Refine the misidentified APIs’ descriptions, summaries, and tags; clarify overlapping scopes. Act on signals: descriptions, summaries, and tags Repeat: Re-evaluate after each change to confirm that accuracy improves and regressions are avoided. Repeat: Conclusion As the number of available APIs continues to grow, exploring and managing them requires a new approach. With the rise of AI agents powered by large language models (LLMs), developers now have a more intuitive and efficient way to discover and interact with APIs—saving countless hours previously spent searching for the right endpoints. The potential doesn’t stop there. This concept can evolve into a standalone product capable of seamlessly ingesting OpenAPI specifications at runtime and exposing them through a natural language interface—offering users an out-of-the-box solution for API exploration. Hopefully, this article has illustrated how to leverage LLMs effectively and how well-structured API documentation can create a smoother, more intelligent discovery experience.

Walkthroughs, tutorials, guides, and tips. This story will teach you how to do something new or how to do something better.

I Built an AI Agent That Lets You Explore APIs in Plain English

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

From Automation to Autonomy: How AI is Transforming Site Reliability Engineering

Stop Rebuilds! Reload Your Code Live in a Docker Container With Bind Mounts

System Design in a Nutshell

7 Ways You Can Use Artificial Intelligence For Explosive Business Growth

AI for Marketing Automation: Strategies and Tools for Marketers

From Automation to Autonomy: How AI is Transforming Site Reliability Engineering

Stop Rebuilds! Reload Your Code Live in a Docker Container With Bind Mounts

System Design in a Nutshell

7 Ways You Can Use Artificial Intelligence For Explosive Business Growth

AI for Marketing Automation: Strategies and Tools for Marketers

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps