Text-to-SQL: How AI turns a plain-English question into a database query

Text-to-SQL is the task of turning a question written in everyday language into a query a database can run. SQL (Structured Query Language) is the standard way to ask databases for data. A text-to-SQL system lets someone who has never written SQL ask for numbers and get them.

Modern versions use large language models. Google’s new Data Cloud connectors for Gemini, covered in our news story, are a recent commercial example.

One question, one query, one answer

Imagine a retail manager types: “Which five stores sold the most coats last December?” A database does not understand that sentence. It understands something like this:

SELECT store, SUM(units) FROM sales WHERE category = 'coats' AND month = '2025-12' GROUP BY store ORDER BY 2 DESC LIMIT 5;

A text-to-SQL system writes that query for the manager, runs it, and returns the table. The manager never sees the code unless they want to check it. The hard part is not the SQL grammar. It is knowing that “sold” means the units column, that “coats” is a category value, and that the data lives in a table called sales.

The model has to read the map before it writes the route

A database has a schema: a list of tables, columns and how they connect. A language model cannot guess it. The system has to show the model the relevant parts, and then the model writes the query.

Most systems follow a loose pipeline:

1. Find the right data. Large companies have thousands of tables, so the system first picks candidates. Researchers call this schema linking. Surveys say it matters partly because listing every table in a prompt is impractical, and because irrelevant tables can distract the model.
2. Add meaning. Column names like cat_cd say little. Descriptions, glossaries and examples help the model map everyday words to database terms.
3. Write the SQL. The model produces a query, often guided by a few example question and query pairs.
4. Check and retry. Many systems run the query, and if it errors or returns nothing, the model tries again.

Think of a new analyst on day one. Without a data dictionary they would waste hours guessing. With one, they work quickly. A language model is the same, only faster and more confident when wrong.

From hand-built parsers to language models

Turning language into database queries is an old research problem. The modern benchmark era began with Spider, released in 2018 by Tao Yu, Dragomir Radev and colleagues at Yale. It has 10,181 questions across 200 databases. Its key idea was that test databases differ from training ones, so a model must handle schemas it has never seen. In the paper, the best model then reached only 12.4% exact-match accuracy in that setting.

Large language models changed the picture. A 2024 survey, accepted by ACM Computing Surveys, organizes the LLM era around two approaches: prompt engineering, where a general model is guided with instructions and examples, and fine-tuning, where a model is trained further on question and SQL pairs.

Why real company data is harder than the benchmark

Spider’s databases are small and tidy. In 2023, Jinyang Li, Binyuan Hui and colleagues released BIRD, a benchmark built to look more like real work. It has 12,751 question and SQL pairs across 95 databases totaling 33.4 GB, spanning 37 professional domains.

BIRD’s authors argued that three things trip models up:

• Dirty values: the data inside the tables is messy, not just the table names.
• Outside knowledge: questions can depend on business definitions that appear nowhere in the database.
• Speed: a correct query that takes hours on a huge table is not useful.

At release, the best model the authors tested, ChatGPT, reached 40.08% execution accuracy. Human performance was 92.96%. Execution accuracy means the predicted query returns the same result as the correct one. Later systems have narrowed the gap on public leaderboards, but those scores are self-reported on a fixed test set. They do not guarantee similar results on your own company’s tables.

Where Gemini’s connectors fit

Google’s connectors put this pipeline inside a business chat tool. According to Google, Gemini searches its Knowledge Catalog, a metadata service, for candidate data. It uses business glossaries and semantic models to map terms like “cold weather products” to categories such as sweaters, coats and jackets. It then writes SQL and runs it under the asker’s own credentials, so existing row and column permissions still apply.

That design addresses two of the weak points above. The catalog supplies the context a model otherwise lacks, and the permission model limits what a wrong or manipulated query can reach. Google also separates the use case: connectors for exploratory, ad hoc questions, and certified Conversational Analytics agents for repeatable reporting. Other vendors and open-source projects offer similar text-to-SQL tools, each with its own approach.

The limits and what to watch

The main risk is a wrong answer that looks right. A query can run without errors and still count the wrong thing, for example by joining two tables incorrectly or using the wrong definition of “active customer.” The output is a clean number, so the mistake is easy to miss. Showing the generated SQL, or limiting the system to certified definitions, lets a person check.

Other limits are practical:

• Metadata quality: surveys list complex and domain-specific schemas as an open problem. A catalog with sparse descriptions gives the model little to work with.
• Ambiguous questions: “last quarter” may mean calendar or fiscal quarters. Good systems ask a follow-up instead of guessing.
• Cost and speed: a careless query on a huge table can be slow and expensive, which is why BIRD scores efficiency and why Google says admins can set a billing project to control query costs.
• Security: running queries under the user’s own permissions, rather than a shared account, is a common safeguard.

Things to watch: benchmarks closer to real enterprise data, systems that ask clarifying questions, and tighter links between catalogs and models. For now, text-to-SQL works best as a fast first draft of an analysis, with a human checking anything that drives a decision.