Every BI vendor on earth now has a chatbot. Most of them are the same thing: a text box bolted onto the same dashboards, wired to a model that turns your question into SQL and runs it. The demo is mesmerizing. The production experience is a coin flip. And the gap between those two things is the most misunderstood part of this entire wave.
We have spent a lot of time figuring out which parts of "conversational BI" are real and which are theater. Here is the honest split.
What's mostly hype
"Ask any question in plain English and get the right answer." This is the demo that sells the category and the promise that breaks in production. Text-to-SQL against a real warehouse is not a solved problem, because the hard part was never the SQL syntax. It is that the model does not know your business. It does not know that "revenue" has six definitions depending on who is asking, that two tables join on a key that means something subtly different in each, that the status column has a value everyone knows to exclude. On a clean demo schema it looks like magic. On your actual warehouse it confidently returns a number that is wrong in a way no one catches until it ends up in a board deck.
"It replaces your analysts." It does not. It changes what they spend their time on, which is a real and good shift, but the idea that a chatbot eliminates the need for people who understand the data is the kind of thing said by people who have never had to explain why two dashboards disagree.
"Dashboards are dead." Overstated. For monitoring a known set of metrics you look at every day, a dashboard is still the right tool. You do not want to type a question every morning to find out whether yesterday's numbers were normal. The dashboard is not dying. Its monopoly on how you interact with data is.
So if the chatbot, the SQL generation, and "death of the dashboard" are mostly noise, what actually changed? Quite a lot. It is just not the part the demos emphasize.
What genuinely changed
The economics of an ad-hoc question went to zero
This is the real unlock, and it has almost nothing to do with the chat interface itself.
In the old model, getting an answer to a question nobody anticipated meant filing a ticket with the analytics team and waiting. Three days, a week, longer. Because the cost of asking was so high, the overwhelming majority of questions simply never got asked. People made decisions on the dashboard they had, because the dashboard they needed was not worth the wait. There is an enormous long tail of questions ("why did the Northeast dip last week," "which cohort is driving the churn spike," "is this customer's usage trending the way the renewal assumes") that were economically irrational to ask and so never were.
Collapse the cost of asking to near zero and that long tail finally gets answered. That is the shift. Not "you can chat with your data" but "the questions that were never worth the wait are now worth asking." The interface is incidental; the change in cost is the point.
The semantic layer went from nice-to-have to the whole product
Here is the architectural fact that separates a product from a demo: the systems that work do not let the model write freeform SQL against raw tables. They constrain it to a semantic layer: a governed definition of your metrics, dimensions, and the relationships between them.
The reason chat-on-top-of-Looker disappoints is that there is nothing grounding the model. It is guessing at your business logic. The reason a well-built conversational system works is that the model is not guessing. It is composing an answer against metrics that have already been defined and verified. "Revenue" means one specific thing because someone defined it once. The join is correct because the relationship is declared. The model's job shrinks from "infer this company's entire data model from a question" to "map this question onto a set of governed building blocks," and that is a job it can actually do reliably.
For fifteen years the semantic layer was the unglamorous plumbing nobody wanted to fund. LLMs turned it into the single most important asset in the stack, because it is the difference between a system that is trustworthy and one that is merely impressive.
The question changed from "what happened" to "what will happen" and "what should I do"
Classic BI is retrospective by construction. It is very good at telling you what revenue was last quarter. It has almost nothing to say about what it will be, or what happens if you raise prices five percent, because answering those questions requires a forecasting model and a way to express a hypothetical, neither of which a dashboard provides.
This is the shift we care about most. Once you can ask a question in plain language and have the system orchestrate the right machinery behind it, the natural next questions are not retrospective. "What will cash look like in eight weeks." "What if our two biggest accounts slip a month." "Which deals actually move the quarter." These are forecasting and scenario questions, and they were effectively impossible to self-serve before, because building each one was a project. Making them conversational is not a cosmetic upgrade to BI. It is a different category of tool.
Explanation became a first-class output
Old BI shows you the number. The genuinely new capability is why the number moved: decomposing a change into its drivers, flagging an anomaly and explaining it, turning a chart into a paragraph a human can act on. This is the thing LLMs are actually, natively good at: the synthesis-and-narrative layer that sits on top of computed results. Not the arithmetic, but the explanation of the arithmetic.
The architecture lesson underneath all of this
If there is one principle that separates the products that work from the ones that hallucinate, it is this: the LLM is the interface and the orchestrator, not the calculator.
The systems that fail are the ones that hand raw numbers to a language model and ask it to reason over them: to do the math, compute the trend, run the comparison. Language models are not arithmetic engines, and the moment you ask one to be one, it will produce a number that looks plausible and is wrong. The systems that work use the model for what it is good at (understanding the question, mapping it to governed definitions, routing it to the right deterministic engine, and narrating the result) and they let proven, boring machinery do the actual computation. SQL does the retrieval. A real forecasting model does the forecast. The statistics are deterministic and auditable. The LLM is the connective tissue between the person and that machinery, and it never touches the math itself.
Get that boundary right and you have a system you can trust. Blur it (let the model "reason" over the raw data) and you have a very smooth demo and a liability in production.
So what is the category, really
It is not "BI with a chatbot." Bolting a text box onto retrospective dashboards is the least interesting thing you can do with this technology, and it is most of what is being shipped.
The actual shift is toward decision intelligence: a tool where the operative question is no longer "what happened" but "what should I do," and answering that requires retrieval, forecasting, scenario modeling, and explanation working together, with a language model as the layer that lets a person reach all of it by asking, and deterministic engines doing the work behind the curtain. The chat box is the visible one percent. The semantic grounding, the forecasting machinery, and the discipline to keep the model away from the arithmetic are the ninety-nine percent that decide whether any of it is worth trusting.
That is the part the demos skip. It is also the only part that matters.