Using Natural Language Search to Query a Large Codebase

Natural language search across a large codebase runs into trouble long before anyone writes a line of model code. The obstacle is a vocabulary mismatch: the words developers use to name things and the words a searcher uses to describe them rarely overlap, and that gap is what every retrieval system on this subject is ultimately built to close. Consider a search for "authentication logic" run as a plain grep over a few hundred repositories. It will return every file where the string "auth" happens to appear: configuration files, test fixtures, a variable named authorTag. It will not return the OAuth handler in one service, the token validator in another, or the middleware that ties them together, because none of those files necessarily contains the word "authentication" at all. The system behaves exactly as designed. It just was never designed to understand what the code does.

Nazari et al.'s 2026 paper on Merlin states the limitation precisely: grep cannot answer questions that require semantic or inter-procedural reasoning. It has no concept of a call chain, and it has no way of knowing that a function named fetchClientDetails() and a request to "retrieve user profile" describe the same operation. String matching treats code as text. Code is not text so much as a record of decisions, and the record format varies by team, by era, and by the individual engineer who happened to write the function. The larger the codebase and the more people who have named things across more years, the further developer intent drifts from the literal names left behind in source. Scale does not merely expose the vocabulary gap, it widens it.

The instinct to solve this by routing the whole problem to a large language model does not hold up either. Large codebases blow past context limits quickly, and pasting hundreds of repositories into a prompt is a resource constraint that no amount of clever prompting resolves. A model can reason only over what it has in front of it, and a few million lines of source spread across a few hundred repos simply will not fit. That constraint reframes the entire problem correctly. This is a retrieval problem before it is a generation problem: the right code has to be found before any model, human or artificial, can reason about it. Everything that follows, semantic vector search and structural search alike, exists to solve that retrieval step first.

Semantic vector search

Semantic vector search closes the vocabulary gap by giving up on matching words altogether and matching meaning instead. Both the natural language query and the candidate code get mapped into a shared vector space, where concepts that mean the same thing end up geometrically close regardless of what words were used to express them. A question phrased as "retrieve user profile" and a function named fetchClientDetails() can land near each other in that space even though they share no common text, because the embedding captures what each one is doing rather than how each one is spelled.

One encoder processes the query, a second processes the code, and the two outputs are scored against each other by cosine similarity to rank candidates, in the standard dual encoder architecture, sometimes called a bi-encoder. The backbone models doing this work are transformer-based and built specifically to understand programming syntax rather than treating code as ordinary prose, with CodeBERT, UniXcoder, and GraphCodeBERT standing as the common choices. These models were trained to recognize the structure of a function rather than just the characters inside it.

AST-aware models push the idea one step further by fusing the token sequence of the code with a serialized version of its abstract syntax tree, so the model sees syntactic structure alongside semantic content rather than one or the other. Research from Gu et al. found that this kind of tree serialization and token fusion produced a substantial jump in retrieval quality as measured by mean reciprocal rank. The architecture matters, but it is not the only lever. Data quality carries comparable weight: attaching docstrings, comments, and commit messages to a code snippet narrows the gap between what a developer intended and what the raw function looks like on its own, and Bahrami et al.'s 2021 AugmentedCode framework showed meaningful MRR gains on standard benchmarks from exactly this kind of augmentation, without any change to the underlying model. Most of this research gets measured against CodeSearchNet, a benchmark spanning six programming languages and millions of functions, which gives a sense of the scale these systems are expected to operate at before anyone trusts them in production.

A local, privacy-preserving version of this pattern already exists in practice: tools built on the sentence-transformers library can generate embeddings of functions entirely on local hardware, cache them in a local file, and perform nearest-neighbor search at query time with nothing transmitted externally. That matters for teams unwilling to send proprietary source code to a third-party service just to search it.

Semantic search earns its keep on fuzzy, cross-vocabulary questions, the ones where the person asking does not know the exact term the codebase uses. It has a hard boundary. Asking it to "find every function that calls X and passes the result to a logging function" will cause it to struggle, because that is not a question about meaning, it is a question about shape, and shape is a different kind of query.

Structural search: retrieving code by shape

Structural search engines exist precisely to answer the shape-of-code questions that semantic search cannot. Tools built for this purpose can trace call chains, follow data flow, and reason about type relationships across a codebase, capabilities that no amount of vector similarity will reconstruct on its own. These engines run on domain-specific query languages that most working developers have never learned to write fluently, and that unfamiliarity is the actual barrier standing between a developer and a correct answer.

Pairing a large language model with the structural engine as a translator removes that barrier without touching the engine itself. The pattern is straightforward: a developer types a question in plain English, the model generates the corresponding query in the structural engine's domain-specific language, and the engine executes it and returns a precise, structurally grounded answer. A July 2025 paper (arXiv:2507.02107) demonstrated this exact pairing using Semgrep and GQL, converting natural language into executable structural queries rather than asking the model to answer from raw code directly.

Merlin, introduced in Nazari et al.'s 2026 paper, extends this pattern considerably. It wires a large language model into CodeQL through a retrieval-augmented, iterative query-generation loop built to handle inter-procedural reasoning, the kind of multi-step tracing across functions and files that neither grep nor plain semantic search can follow. Access to Merlin increased task accuracy by an average of 3.8 times compared to working without it, the paper's findings show.

Semantic search and structural search are not competing for the same question. Semantic search handles the fuzzy, cross-vocabulary case, where the searcher does not know the right term to use. Structural search, reached through an LLM translator, handles the precise case, where the question is already well-formed but the query language standing between the question and the answer is not one the developer speaks. Asking "where does this service handle authentication?" is a semantic question. Asking "which functions call tokenValidator() and pass the result to a logging function?" is a structural one. Knowing which mechanism a given question needs is itself part of using either one well.

The retrieval problem now blocking AI coding agents

The vocabulary gap that makes grep unreliable for human searchers does not disappear when the searcher is an AI agent instead of a person. An agent working without full codebase context runs into the identical mismatch between how code is named and how a task is described, but the cost of getting that retrieval step wrong compounds at every subsequent step of the agent's reasoning, rather than stopping at one bad search result the way it would for a human.

An agent confined to local files or a narrow context window has no choice but to reconstruct relationships that the codebase already encodes elsewhere, rediscovering structure that a properly indexed system would simply hand over. That reconstruction is not only slower. An agent piecing together call chains and dependencies from raw source is more likely to reason against a deprecated pattern it happens to encounter first, or to miss a cross-repo dependency that never surfaces in the files it was given to read. The failure mode is not a stall or a timeout, it is a confidently wrong answer built on an incomplete picture.

That changes what the retrieval layer is for. Whether the mechanism is semantic vector search or LLM-translated structural search, it stops functioning as a developer convenience and starts functioning as a precondition for the agent producing a correct answer at all. An agent that cannot retrieve the right code accurately cannot reason about it accurately either, no matter how capable the underlying model is.

The MCP context layer as infrastructure for agents

The architectural answer taking shape across the industry is to stop asking agents to retrieve and reconstruct on the fly, and instead precompute the structured intelligence a codebase contains, call graphs, dependency maps, data flow, and expose that precomputed intelligence to agents through the Model Context Protocol, so the agent receives an answer rather than raw material it still has to interpret.

The division of labor this produces is clean: deterministic static analysis discovers the relationships that already exist in the source code, an AI model reasons over what static analysis found, and MCP delivers the result to the agent that asked for it, a framing HoundDog.ai captures as "static analysis discovers; AI reasons; MCP delivers". An agent querying a code intelligence MCP server gets back the authentication service's specification, its dependencies, the business rules governing token expiry, and the full cross-service call chain running from HTTP request through middleware to database, rather than the raw files it would otherwise have to parse and connect itself.

An ecosystem of MCP servers built specifically for this purpose is already forming. Code Pathfinder constructs a comprehensive call graph through AST-based analysis, letting an agent query callers, trace dependencies, and run dataflow analysis directly. GitNexus uses Tree-sitter to build a knowledge graph combined with Graph RAG, serving structural context to agents through the same protocol. KiroGraph offers a fully local semantic graph with hybrid search, and codebase-memory-mcp, a high-performance server written in C, indexes a codebase into a persistent local knowledge graph that any agent harness can query. Each tool takes a different technical route, but all of them are solving the same problem: turning a codebase into something an agent can query structurally instead of something it has to re-read from scratch every session.

Practitioners have started calling the resulting layer the "context layer": architecture decision records, design patterns, and guardrails written once and surfaced to every agent through MCP, instead of hoping each separate model run rediscovers the same conventions independently. For this to work at the pace agents operate, the underlying analysis has to be fast enough to regenerate continuously rather than run as a periodic batch job. HoundDog.ai's deterministic scanner, written in Rust, can analyze ten million lines of code in under a minute on standard CPU infrastructure, which is the kind of speed that makes continuous context regeneration practical instead of theoretical.

The pattern has also reached the level of formal intellectual property. A Microsoft patent granted in May 2025 describes a system for searching code examples to augment model prompts: building a codebase segment table containing embeddings of code and associated metadata, searching that table at inference time for examples similar to the user's query, and using the results to construct an augmented prompt. It is, in effect, a patented instance of the same retrieval-then-reason architecture running through every example in this section, a sign that the pattern has moved well past experimental tooling and into infrastructure that large technology companies consider worth protecting.

Where this retrieval infrastructure matters beyond senior engineers

The same retrieval layer built to give AI agents accurate codebase context has a second beneficiary that has nothing to do with agents at all: the people, inside and outside engineering, who have always depended on asking someone else where things are.

New hire onboarding is itself a retrieval problem, just a slower one. Without structured access to what a codebase means, a new developer spends months decoding undocumented systems before they can contribute a feature with any confidence. CodeOrient, a 2026 undergraduate project out of NTU, applies NLP, code graph visualization, and vector-based retrieval directly to this problem, helping new developers understand unfamiliar codebases and cutting the time to a first commit. A Shopify case study found that adopting Glean reduced onboarding time for new engineers by 41%.

The benefit extends past engineers to people who have never been expected to read code. A product manager, a security auditor, or a legal reviewer asking "where does this service process payment data?" can get a navigable, cited answer without needing a developer to walk them through the relevant files, provided the retrieval layer supports natural language queries across the full codebase. Teams that have adopted self-serve AI-powered search describe this as a concrete target: eliminating most of the repetitive questions that otherwise land on senior engineers, replacing twenty-plus minutes of folder navigation with answers delivered in under thirty seconds.

Realizing that benefit organization-wide, rather than just for the engineers comfortable running their own tooling, depends on where the system lives and what it connects to. A platform that indexes every repository, connects to Jira, Linear, and Confluence through MCP-based connectors, and deploys inside an organization's own infrastructure rather than requiring code to leave the network is the architecture that makes this accessible to the full organization without the compliance tradeoffs a SaaS tool introduces.