Using Natural Language Search to Query a Large Codebase

Grep finds where a word sits in a file. It has no idea what that code actually does. That gap is the whole reason natural language search across a codebase is even a hard problem, and it's why two very different technical approaches, semantic vector search and LLM-translated structural search, have emerged to close it.

Try this on a codebase with a few million lines spread across a few hundred repos. Search for "authentication logic." Grep will hand back every file where the string "auth" appears in config files, test fixtures, and a variable named authorTag that has nothing to do with logging anyone in. What it won't hand back is the actual authentication system: the OAuth handler in one service, the token validator in another, the middleware that ties them together. None of those files necessarily contain the word "authentication." The developer's intent and the engineer's naming choices from three years ago live in totally different vocabularies, and at scale that gap only gets wider.

This isn't a tooling gripe, either. Nazari et al.'s 2026 paper on Merlin makes the point directly: standard tools like grep can't answer questions that require semantic or inter-procedural reasoning, full stop. Grep matches strings. It doesn't trace a call chain, and it doesn't know that fetchClientDetails() and "retrieve user profile" mean the same thing.

So hand the problem to an LLM instead? Not so fast. Large codebases blow past context limits fast, and you can't just paste a few hundred repos into a prompt and ask a model to "take a look." That's a resource problem, and no amount of clever prompting fixes it.

Which points to the actual shape of the problem. This isn't a generation problem, it's a retrieval problem. Before any model reasons about code, something has to find the right code first. Everything below is about how that retrieval actually happens.

How semantic vector search retrieves meaning rather than text

Semantic vector search works on a simple idea: turn both the query and the code into vectors, then measure the distance between them. A natural language question like "retrieve user profile" and a function named fetchClientDetails() end up close together in vector space, even though they don't share a single word. That shared space is the whole trick. It's what makes the search vocabulary-independent.

The dominant architecture here is the dual encoder, sometimes called a bi-encoder. One encoder handles the query, another handles the code, and the two get scored against each other for similarity. The backbone models tend to be transformer-based, code-aware variants built specifically to understand programming syntax rather than treating code like ordinary English prose.

Some systems go further and fuse token sequences with a serialized abstract syntax tree, capturing the code's actual structure alongside its meaning. Research from Gu et al. In 2021, research from Gu et al. found this kind of tree serialization and token fusion boosted retrieval quality by up to 17.8% in MRR (mean reciprocal rank, a standard measure of how close the right answer lands to the top of the results), a substantial improvement rather than a marginal one. That's not a marginal bump.

Data quality matters just as much as architecture. Attaching natural language descriptions, docstrings, comments, commit messages, to code snippets narrows the intent gap considerably. Bahrami et al.'s 2021 AugmentedCode framework showed MRR gains of up to 0.09 on standard benchmarks just from this kind of augmentation. And the benchmark most of this research gets measured against, CodeSearchNet, spans six programming languages and more than six million functions, which gives a sense of the scale these systems are built to handle.

Semantic search shines when the query is fuzzy, when it crosses languages, or when the person searching genuinely doesn't know what the thing they want is called. What it can't do is answer a precise structural question. "Find every function that calls X with argument Y" It's a shape problem, not a meaning problem. That's a different mechanism entirely.

How LLM-translated structural search retrieves code by shape rather than meaning

Structural search engines like Semgrep and GQL let someone query code by its actual shape: data flow, call chains, type relationships. The architecture underneath the words on the page produces the power of these tools, not the words themselves. These tools are genuinely powerful. They're also written in domain-specific languages that most developers have never opened a manual for, let alone learned to write fluently.

The fix is obvious once you see it: let an LLM do the translation. A developer types a plain English question, the model generates the DSL query, and the structural engine runs it against the indexed code. A July 2025 paper (arXiv:2507.02107) demonstrated exactly this pairing with Semgrep and GQL.

Merlin, from Nazari et al.'s 2026 paper, takes this further by wiring an LLM into CodeQL through a RAG-based, iterative query-generation loop. The paper spends real effort solving the problem that a query can be perfectly valid syntax and still come back empty, or worse, quietly wrong. So Merlin runs a self-test step using what the paper calls "assistive queries," designed to catch semantic flaws before the query ever reaches the user. The user study result is the headline number here: Merlin access multiplied task accuracy several times over and cut total task time by 31%. Sit with that for a second. Those are substantial gains in both accuracy and time.

Structural search is built for exactly the kind of task semantic search can't touch: finding every call site of a function that's about to get deprecated, tracing how user input flows to a dangerous sink, spotting polymorphic usages scattered across a dozen files. What it can't do is help someone who has no idea what shape they're even looking for. That's semantic search's job.

Putting the two side by side makes the division of labor obvious. One handles "I don't know what this is called." The other handles "I know exactly what I need, I just can't write the query for it." A search system that only does one of these is only solving half the problem.

The agentic search pattern that combines both approaches into a working answer

When both mechanisms are placed in the hands of an agent, the pattern looks like this: a plain English question goes in, an answer with citations to actual file locations comes out, and somewhere in between an agent is running searches, checking results, and deciding what to do next.

The agent breaks the question into concrete steps, regex here, a symbol lookup there, maybe a structural query, follows the references it turns up, reads the files that matter, and only then writes an answer grounded in specific code. What makes this "deep" rather than a single lucky guess is the loop: after every search, the agent checks whether it knows enough yet, and if not, picks the next move.

Claude Code's agentic search works this way, walking directories, reading files, running grep, following references as it goes. That approach sidesteps some of the staleness problems that come with a pre-built embedding index (an index built ahead of time can drift out of sync with the code; live traversal can't go stale in the same way). But there is an important catch: agentic search works best when the agent already has a rough idea where to look. A vague question across a massive codebase can burn through its available context before it ever gets to the useful part.

A pre-built code intelligence index matters at scale for this reason. An agent that starts with structure already in hand doesn't waste its budget wandering. One that starts from a blank directory listing does.

Why stuffing files into a prompt is not a substitute for real retrieval

There's a shortcut a lot of teams reach for first: paste files into the context window until the answer is probably somewhere in there. Zero infrastructure, easy to understand, and it falls apart the moment the codebase gets big. Intuitive doesn't mean it works.

Repowise.dev breaks the failure down into separate pieces. First, waste: most of what gets stuffed into the prompt is boilerplate, imports, comments, none of it relevant to the actual question, and the model still pays the latency and token cost of reading past it. Second, reasoning degradation: as the context window fills up, the model's ability to recall details buried in the middle drops off. The well-documented "Lost in the Middle" problem is a structural weakness in how these models attend to long input, not a minor quirk. A bigger context window doesn't fix either issue; it just pushes the breaking point further out. It just pushes the breaking point further out.

Made concrete: stuff 50 files into a single prompt, and a critical interface definition sitting in file 23 often just doesn't make it into the model's answer. LLMs systematically favor whatever sits at the start or the end of a long context, and the middle becomes a kind of dead zone.

Repowise.dev ran the comparison directly. Against a bare-agent control, a structured-tools approach cut output tokens by 31.6% and reached an answer in 3.8 tool calls versus 7.2 for the bare agent. Nearly half the tool calls, for a better-grounded answer.

Anthropic's own engineering team has reported that code execution paired with MCP can cut context overhead by up to 98.7% in some configurations. That's the ceiling, the number that shows what proper retrieval actually buys you, not a headline claim to lead with but the payoff. Retrieval is the thing that makes reasoning over a large codebase possible at all, not a nice-to-have layered on top of prompting. It's the thing that makes reasoning over a large codebase possible at all.

How MCP connects AI coding agents to a code intelligence index

MCP, the Model Context Protocol, was open-sourced by Anthropic on November 25, 2024. In December 2025 it moved to the Agentic AI Foundation, a Linux Foundation directed fund co-founded by Anthropic, Block, and OpenAI, which made it vendor-neutral rather than tied to one company's roadmap.

Adoption moved fast after that. VS Code declared MCP generally available in version 1.102 back in June 2025, and by early 2026 the list of tools built around it included Claude Code, Cursor, VS Code Copilot, Codex, Windsurf, Zed, Continue.dev, Cline, and Goose. That's a wide enough spread that MCP is functioning less like one vendor's feature and more like a shared plumbing standard.

The interesting application for code specifically, as that same publication described, is generating evidence straight from source using deterministic static analysis, keeping that evidence current as the code changes, and exposing it to an agent through MCP as a callable tool rather than a pile of raw files to read. The practical difference matters: instead of getting handed a folder of text, the agent gets symbol definitions, cross-references, and call graphs already worked out. That's faster and it's a lot more precise, particularly once the codebase turns from a tidy single repo into an enterprise sprawl.

None of this comes free of risk. OWASP's MCP Top 10 calls out token mismanagement, privilege escalation, tool poisoning, command injection, and missing telemetry as real concerns. Mature MCP setups lean toward least-privilege access, loading tools only when needed, and keeping auditable logs of every call, because "the agent can just call any tool" is a security nightmare waiting for its incident report. The 2026 MCP spec refinements include Streamable HTTP and OAuth 2.1 as production-safe enterprise transport options.

What to expect from natural language search across repositories in practice

For engineers already working in a codebase, the gains are visible mostly in debugging and refactoring. Ask "show me where the failed payment retry logic is implemented," and a well-built system surfaces the retry handler and the exception handling in one pass, instead of the old routine of grepping across five different services and hoping the naming conventions line up. Refactoring safely works the same way: finding every usage of a function, including the indirect ones nobody remembers exist, understanding what a change touches before it goes out, not after.

For someone new to a codebase, this changes the onboarding experience almost entirely. "Where are user-facing routes defined for the payment service?" becomes a question the tool answers, not a question that requires cornering a senior engineer at their desk. That's not a small thing. New hire ramp time is one of the quiet costs nobody puts on a slide deck.

Results are only as good as the index behind them, though. An agent that only sees half the repositories can't answer a question that spans all of them, and multi-repo indexing is what turns cross-repo questions from "theoretically possible" into "actually works." What a decent answer looks like: specific files, specific symbols, citations tying the claim back to real code. Not a list of file paths with no explanation, and not confident prose with nothing behind it.

A wildly ambiguous query with no structural shape to it will get semantic search some candidates, but the agent may need a follow-up turn or two to actually converge on the right one. That's just how the interaction works when the question itself is vague. That's just how the interaction works when the question itself is vague.

How self-hosted code intelligence platforms deliver this capability without sending code off-premise

For a lot of engineering orgs, none of the above matters if it means shipping proprietary code to somebody else's servers. Most cloud-hosted AI coding tools do exactly that, and for healthcare, defense, and financial services teams, that's a hard constraint, not a preference to be negotiated around.

The good news is that self-hosted and air-gapped AI coding tools have matured into a real market. Teams no longer have to trade capability for control, iternal.ai notes, which used to be the whole tradeoff.

A well-built self-hosted platform generally covers a few things at once. It builds and queries the code intelligence index, symbols, cross-references, call graphs, entirely inside the customer's own infrastructure, nothing leaving the building. It exposes that index through an MCP-based context layer that any AI agent the team already uses can plug into, so there's no lock-in to a single model provider. It connects to the tools engineers already live in, Jira, Linear, Confluence, so an agent can tie code context to a ticket without stepping outside the secure environment. And ideally it ships as something closer to a single Docker container than a six-month infrastructure migration, because "self-hosted" shouldn't mean "self-inflicted."

Cost matters here too. Weavai.app's pricing data shows enterprise-grade code search with an AI context layer now runs at less than half the price of the leading legacy tool, which lists at $59 per user per month. That's a real gap for engineering leaders weighing build versus buy.

None of this works, though, if the index is partial, stale, or sitting somewhere it shouldn't. Completeness means every repository is actually in the index, not just the ones somebody remembered to add. Currency means the index updates as code changes, not once a quarter. And trust means the sensitive code never leaves the place it was always supposed to stay. Get all three right, and natural language search becomes infrastructure rather than a party trick.