Search systems have historically been optimized for retrieval: given a query, return the most relevant documents. That model breaks down the moment user intent shifts from finding information to solving problems. retrieval finding information solving problems Consider a query like: “How will tomorrow’s weather in Seattle affect flight prices to JFK?” “How will tomorrow’s weather in Seattle affect flight prices to JFK?” This isn’t a search problem. It’s a reasoning problem — one that requires decomposition, orchestration across multiple systems, and synthesis into a coherent answer. reasoning problem This is where agentic search comes in. agentic search In this article, I’ll walk through how we designed and productionized an agentic search framework in Go — not as a demo, but as a real system operating under production constraints like latency, cost, concurrency, and failure modes. agentic search framework in Go From Search to Agentic Search Keyword and vector search systems excel at matching queries to documents. What they don’t handle well is: Multi-step reasoningTool coordinationQuery decompositionAnswer synthesis Multi-step reasoning Tool coordination Query decomposition Answer synthesis Agentic search treats the LLM not as a text generator, but as a planner — a component that decides what actions to taketo answer a question. planner what actions to take At a high level, an agentic system must be able to: Understand user intentDecide which tools to callExecute those tools safelyIterate when necessarySynthesize a final response Understand user intent Decide which tools to call Execute those tools safely Iterate when necessary Synthesize a final response The hard part isn’t wiring an LLM to tools. The hard part is doing this predictably and economically in production. predictably and economically in production High-Level Architecture We structured the system around three core concerns: Planning – deciding what to doExecution – running tools efficientlySynthesis – producing the final answer Planning – deciding what to do Planning Execution – running tools efficiently Execution Synthesis – producing the final answer Synthesis Here’s the end-to-end flow: Each stage is deliberately isolated. Reasoning does not leak into execution, and execution does not influence planning decisions directly. Flow Orchestrator: The Control Plane The Flow Orchestrator manages the full lifecycle of a request. Its responsibilities include: Flow Orchestrator Coordinating planner invocationsExecuting tools concurrentlyHandling retries, timeouts, and cancellationsStreaming partial responses Coordinating planner invocations Executing tools concurrently Handling retries, timeouts, and cancellations Streaming partial responses Instead of a linear pipeline, the orchestrator supports parallel execution using Go’s goroutines. This becomes essential once multiple independent tools are involved. parallel execution Query Planner: Mandatory First Pass, Conditional Iteration The Query Planner is always invoked at least once. Query Planner at least once First Planner Call (Always) On the first invocation, the planner: Analyzes the user queryProduces an initial set of tool callsEstablishes a consistent reasoning baseline Analyzes the user query Produces an initial set of tool calls Establishes a consistent reasoning baseline Even trivial queries go through this step to maintain uniform behavior and observability. Lightweight Classifier Gate Before invoking the planner a second time, we run a lightweight classifier model to determine whether the query is: second time lightweight classifier model Single-stepMulti-step Single-step Single-step Multi-step Multi-step This classifier is intentionally cheap and fast. Second Planner Call (Only for Multi-Step Queries) If the query is classified as multi-step: The planner is invoked againIt receives:The original user queryTool responses from the first executionIt determines:Whether more tools are requiredWhich tools to call nextHow to sequence them The planner is invoked again It receives: The original user query Tool responses from the first execution It determines: Whether more tools are required Which tools to call next How to sequence them This prevents uncontrolled planner loops — one of the most common failure modes in agentic systems. Tool Registry: Where Reasoning Meets Reality Every tool implements a strict Go interface: // ToolInterface is the tool interface for developers to implement which uses
// generics with strongly typed 
type ToolInterface[Input any, Output any] interface {
  // Execute initiates the execution of a tool.
  //
  // Parameters:
  // - input: Strong typed tool request input.
  // - output: Strong typed tool request output.
  // - toolContext: Additional output data that is not used by the agent model.
  // - err: structured error from tool. in some cases error is passed to LLM. eg: no_response from tool
  Execute(ctx context.Context, requestContext *RequestContext, input Input) (output Output, toolContext ToolResponseContext, err error)

  // GetDefinition gets the tool definition sent to Large Language Model.
  GetDefinition() ToolDefinition
} // ToolInterface is the tool interface for developers to implement which uses
// generics with strongly typed 
type ToolInterface[Input any, Output any] interface {
  // Execute initiates the execution of a tool.
  //
  // Parameters:
  // - input: Strong typed tool request input.
  // - output: Strong typed tool request output.
  // - toolContext: Additional output data that is not used by the agent model.
  // - err: structured error from tool. in some cases error is passed to LLM. eg: no_response from tool
  Execute(ctx context.Context, requestContext *RequestContext, input Input) (output Output, toolContext ToolResponseContext, err error)

  // GetDefinition gets the tool definition sent to Large Language Model.
  GetDefinition() ToolDefinition
} This design gives us: Natural-language outputs for planner feedbackStructured metadata for downstream useCompile-time safetySafe parallel execution Natural-language outputs for planner feedback Structured metadata for downstream use Compile-time safety Safe parallel execution The Tool Registry acts as a trust boundary. Planner outputs are treated as intent — not instructions. trust boundary Parallel Tool Execution Planner-generated tool calls are executed concurrently whenever possible. concurrently Go’s concurrency model makes this practical: Lightweight goroutinesContext-based cancellationEfficient I/O-bound execution Lightweight goroutines Context-based cancellation Efficient I/O-bound execution This is one of the reasons Go scales better than Python when agentic systems move beyond prototypes. Response Generation and Streaming Once tools complete, responses flow into the Response Generator. Response Generator Knowledge-based queries are summarized and synthesized using an LLMDirect-answer queries (weather, sports, stocks) bypass synthesis and return raw tool output Knowledge-based queries are summarized and synthesized using an LLM Knowledge-based queries Direct-answer queries (weather, sports, stocks) bypass synthesis and return raw tool output Direct-answer queries Responses are streamed via Server-Sent Events (SSE) so users see partial results early, improving perceived latency. Server-Sent Events (SSE) Caching Strategy: Making Agentic Search Economical One production reality became clear almost immediately: LLM calls have real cost — in both latency and dollars. LLM calls have real cost — in both latency and dollars. Once we began serving beta traffic, caching became mandatory. Our guiding principle was simple: Avoid LLM calls whenever possible. Avoid LLM calls whenever possible. Layer 1: Semantic Cache (Full Response) We first check a semantic cache keyed on the user query. semantic cache Cache hit → return response immediatelyEntire agentic flow is bypassed Cache hit → return response immediately Entire agentic flow is bypassed This delivers the biggest latency and cost win. Layer 2: Planner Response Cache If the semantic cache misses, we check whether the planner output (tool plan) is cached. planner output Skips the planner LLM callExecutes tools directly Skips the planner LLM call Executes tools directly Planner calls are among the most expensive and variable operations — caching them stabilizes both latency and cost. Layer 3: Summarizer Cache Finally, we cache summarizer outputs. Tool results often repeatFinal synthesis can be reusedReduces LLM load during traffic spikes Tool results often repeat Final synthesis can be reused Reduces LLM load during traffic spikes Each cache layer short-circuits a different part of the pipeline. Lessons from Production A few hard-earned lessons: LLM calls are expensive — caching isn’t optional at scaleSemantic caching pays off immediatelyPlanner loops must be gatedMost queries are simpler than they lookTools fail — retries and fallbacks matterObservability is non-negotiableAgents aren’t autonomous — orchestration beats autonomy LLM calls are expensive — caching isn’t optional at scale LLM calls are expensive Semantic caching pays off immediately Semantic caching pays off immediately Planner loops must be gated Planner loops must be gated Most queries are simpler than they look Most queries are simpler than they look Tools fail — retries and fallbacks matter Tools fail — retries and fallbacks matter Observability is non-negotiable Observability is non-negotiable Agents aren’t autonomous — orchestration beats autonomy Agents aren’t autonomous — orchestration beats autonomy

This story contains AI-generated text. The author has used AI either for research, to generate outlines, or write the text itself. 

Beyond Keywords: Engineering a Production-Ready Agentic Search Framework in Go

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Daniel Saks Predicts Agentic AI Will Empower Individuals and Boost Productivity

Active Inference AI: Here's Why It's The Future of Enterprise Operations and Industry Innovation

The Trick That Agentic Frameworks Pulled On Us

Exploring the Spatial Web Protocol at the 4th Annual Applied Active Inference Symposium

What Is Agentic AI?

Building an AI Trading Agent Using Anthropic’s MCP - Part II

Daniel Saks Predicts Agentic AI Will Empower Individuals and Boost Productivity

Active Inference AI: Here's Why It's The Future of Enterprise Operations and Industry Innovation

The Trick That Agentic Frameworks Pulled On Us

Exploring the Spatial Web Protocol at the 4th Annual Applied Active Inference Symposium

What Is Agentic AI?

Building an AI Trading Agent Using Anthropic’s MCP - Part II

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps