By Sakshi Shah · 13 November 2026 · 11 min read
Asking Business Data Questions in Plain English: What It Takes to Get Reliable Answers
Natural language business analytics can answer questions in seconds — or give confidently wrong numbers. How to set it up so the answers can be trusted.
Introduction
"How did sales in the western region compare with last year, by product line?" In most businesses, a question like that goes into a queue. Someone in finance or analytics pulls data from two or three systems, reconciles it in a spreadsheet, and sends an answer days later — by which time the person asking has often moved on to the next question.
AI now makes it possible to type that question in plain English and get an answer, with a chart, in seconds. It is one of the most requested AI features among management teams, and one of the easiest to demonstrate. It is also one of the easiest to get dangerously wrong. A chatbot that confidently reports revenue 12% higher than the finance team's figure does more damage than no chatbot at all, because the next decision may be made on it.
This guide explains how natural language business analytics works, why its answers go wrong, and the foundations — defined metrics, permissions, visible working and testing — that make the answers trustworthy enough to act on.
How Plain-English Analytics Works
There are two common approaches, and many systems combine them.
Question to query. The user asks a question. A language model translates it into a database query, the query runs against your data, and the result is returned as a number, table or chart, often with a short written explanation. The model never "knows" your numbers; it writes the query that fetches them.
Calculated metrics to narrative. Instead of answering arbitrary questions, the system calculates a fixed set of metrics on a schedule — sales, margin, cash, orders, backlog — using ordinary code, then asks a language model to write a plain-language summary of what changed and why it might matter. This is how our Executive Analytics Assistant works: it pulls metrics from many connected sources into a scheduled report, with trend analysis and anomaly flags, replacing the slides a team used to assemble by hand each week.
The second approach is less flexible but much easier to make reliable, because every number comes from tested code. The first is more powerful, and it is the one that needs the most care.
Why the Answers Go Wrong
When a plain-English analytics tool gives a wrong number, the model has usually written a query that is technically valid and answers a slightly different question from the one the user meant. The root causes are almost always in the data and its definitions, not in the model.
| Cause | Example | Result |
|---|---|---|
| Ambiguous definitions | "Revenue" could mean ordered, invoiced, or collected; gross or after returns and discounts | A plausible number that does not match finance |
| Several sources of the same data | Sales recorded in both the CRM and the ERP, with differences | Different answers depending on which table the query used |
| Joins that double count | Joining orders to order lines and then summing order totals | Totals inflated without any obvious error |
| Time periods | "This year" meaning the calendar year to one person and the April–March financial year to another | Comparisons over the wrong period |
| Silent filters | Cancelled orders, test accounts or internal transfers left in | Numbers that look right but include records they should not |
| Stale data | The data was last refreshed two days ago | Correct calculation, out-of-date answer |
Every one of these is a question a human analyst would also have to resolve. The difference is that an analyst would ask; a model, unless designed otherwise, will pick an interpretation and answer with confidence.
The Foundation: Define Your Metrics Once
The most reliable fix is a semantic layer: a defined, agreed set of business metrics and dimensions that sits between the AI and the raw data. "Net revenue" is defined once — which table, which filters, which date, how returns are treated — and agreed with finance. "Region", "product line" and "customer segment" are defined once. The model's job becomes choosing the right defined metrics and filters for a question, rather than writing complex queries against raw tables from scratch.
Building the semantic layer is mostly a business exercise, not a technical one. It means sitting down with finance, sales and operations to agree what each key metric means, and writing those definitions down. Many organisations discover, during this exercise, that two departments have been reporting different figures under the same name for years. Resolving that pays off in its own right, with or without AI.
Underneath the semantic layer sits the data itself, which needs to arrive reliably and on time from the systems that produce it — the work of data pipeline engineering. If the CRM and the ERP disagree, decide which is the source of truth for each metric before asking any AI to report on it.
Show the Working
An answer people can trust is an answer they can check. Every response from a plain-English analytics tool should show:
- which metric definition was used, in plain language ("net revenue: invoiced sales after returns and discounts, excluding tax");
- which filters were applied, including any the user did not explicitly ask for;
- the exact time period, with dates;
- when the data was last refreshed;
- a way to see the underlying rows or the generated query, for those who want to check.
When the question is ambiguous, the tool should ask rather than guess: "Do you mean the calendar year or the financial year?" A clarifying question costs a few seconds; a wrong assumption can cost a decision.
Let code calculate; let the model explain. Numbers should come from queries and tested calculations. The language model's job is to interpret the question, pick the right metrics and explain the result — never to do arithmetic itself or fill in a number it cannot find.
Permissions and Safety
Business data is not equally visible to everyone. A regional manager may be entitled to see their own region's figures but not the whole company's margins; salary data should not be reachable from a sales question. The analytics tool must enforce the same access rules as your other reporting — ideally row-level and column-level permissions applied in the data layer itself, based on who is asking, rather than relying on the model to decline politely.
The tool should also have read-only access to the data, with limits on how long a query may run and how much data it may return, so that a badly formed question cannot slow down the systems the business runs on.
Testing Before You Trust It
Before anyone relies on the answers, build a test set: the twenty to fifty questions management actually asks most often, each with the correct answer agreed by finance or the relevant team. Run the tool against it, investigate every mismatch, and fix the definition, the data or the instructions until it passes. Run the same test set after every change — new data source, new metric, new model version. The broader discipline of evaluation sets is covered in How to Build a Production-Ready AI System, Not Just an AI Prototype.
Collect the real questions
The questions management asks each week, and the reports people spend the most time assembling.
Agree the definitions
Define the metrics and dimensions those questions need, with finance and operations signing off.
Build and test
Connect the data, build the semantic layer, and test against the agreed answers until it passes.
Pilot with a few users
Give access to a small group, log every question and answer, and review the misses each week.
The questions users ask during the pilot are a useful by-product: they show which metrics are missing, which definitions are unclear, and which reports people actually want. It is also worth knowing where plain-English analytics stops being enough. Answering questions is useful; noticing problems nobody asked about, and acting on them, is a different job — the distinction drawn in AI OpsPilot vs Traditional Business Intelligence: What Changes?
Conclusion
Plain-English analytics can turn a question that used to take days into an answer in seconds, but only if the answers are right. Most wrong answers come from ambiguous definitions, duplicate data, double counting, time periods and silent filters — problems that exist with or without AI. Fix them with a semantic layer of agreed metrics, show the working with every answer, ask instead of guessing, enforce permissions in the data layer, and test against questions with known answers before anyone relies on it. Let code calculate and let the model explain.
Spending a day a week assembling management reports?
We will look at the questions your team actually asks and show what can be answered directly — reliably — from your existing data. See the Executive Analytics Assistant or start the conversation.
Frequently Asked Questions
Can AI analytics replace our BI dashboards?
Usually not entirely. Dashboards remain useful for metrics people check every day. Plain-English analytics is best for the many one-off and follow-up questions that dashboards do not cover, and for scheduled written summaries.
How do we stop the AI giving wrong numbers?
Define key metrics once in a semantic layer agreed with finance, have the AI choose from those definitions rather than writing queries against raw tables, show the definition and filters with every answer, ask clarifying questions when a request is ambiguous, and test against questions with known answers.
Does the model see all our business data?
It does not need to. The model can work from descriptions of the available metrics and the user's question, with queries run on your own systems under the user's permissions. Only the results needed to explain the answer are passed back.
What data foundations do we need first?
Reliable, regularly refreshed data from the key systems in one place, agreed definitions for the main metrics, and a clear source of truth where systems disagree. Without those, any reporting — AI or not — will be inconsistent.
How long does it take to set up?
A pilot covering a focused set of questions over one or two data sources can be running in weeks once the data is accessible. Agreeing metric definitions across departments often takes longer than the technical work.