TL;DR
Analytics agents are only as trustworthy as the context they run on. Why a product-analytics semantic layer prevents hallucinated SQL and methodology errors.
Point an LLM at a thousand-table warehouse and ask about activation. It will find something — a table that looks like events, a column that looks like a user id, a status field whose values it guesses. The query runs, a number comes back, and nobody can say where it came from. This is the trust problem every analytics agent has to solve, and the solution is not a better model. It is better context.
That context layer is the semantic layer: the agent’s grounded vocabulary of what exists in the data and what it means. This post covers what a semantic layer has to contain for an analytics agent to be trustworthy — and why the BI-shaped version most tools use isn’t enough for product analytics.
Three ways an ungrounded agent fails
- Invented vocabulary. Without a catalog of real events and properties, the agent guesses names — filtering on
country = 'Germany'when the column holdsDE, or querying asignupevent that is actually calledaccount_created. The query succeeds and silently returns the wrong population. - Wrong methodology. Funnels need conversion windows and step ordering; retention needs cohort bucketing; segmentations need user deduplication. An agent improvising SQL gets these subtly wrong in ways that read as plausible numbers.
- Unverifiable answers. If every answer is produced by a fresh, freehand generation, an analyst has to audit each query individually. That doesn’t scale past a demo, and it is why many agent rollouts stall after the first wrong number reaches a decision.
What a product-analytics semantic layer stores?
Most agentic analytics tools ground the AI in a BI semantic layer: metrics, dimensions, entities, joins, hand-authored in YAML (dbt MetricFlow, Cube, LookML). That shape was designed for dashboards, and it has two problems as agent context: it takes weeks of data engineering to write and maintain, and it cannot express the concepts product analytics runs on.
A semantic layer specialised for product analytics stores a different set of things:
- Events and event properties — the behavioural vocabulary: what users do, with which attributes
- Entities — users, sessions, accounts, teams — and the dimension properties attached to them
- Sampled property values — real values pulled from the warehouse, so filters get suggested from what actually exists rather than invented by the model
- Join relationships — how events connect to users, accounts, and the dimension tables around them
- Accumulated context — saved insights, dashboards, and named cohorts that become vocabulary for future questions
Sampled values deserve emphasis, because they close the most embarrassing failure mode. When the semantic layer knows the real values of plan_tier or country, the agent filters on values that exist. Nothing is left to the model’s imagination.
Built automatically, maintained where people can see it
A semantic layer that takes weeks of YAML authoring doesn’t just delay setup — it goes stale, because updating it competes with everything else on the data team’s backlog. An agent grounded in a stale layer confidently answers from last quarter’s schema.
Mitzu’s answer is to make construction itself agentic. The Configuration Agent scans your data warehouse, identifies event and dimension tables, recognises common schemas (Segment, Snowplow, Firebase, GA4, custom event tables), maps user and group identifiers, and configures join relationships. The analyst reviews and adjusts — nobody hand-writes YAML. The layer lives in the app where analysts, PMs, and marketers can all read it, and the agent helps keep it current as new data lands.
The semantic layer alone isn’t enough: methodology and determinism
Grounding fixes vocabulary. It does not fix methodology. Even with perfect context, an LLM authoring funnel SQL can still botch the conversion window — the semantic layer told it what the events are, not how a funnel must be computed. This is the central argument of analytics agents vs text-to-SQL and BI chatbots: methodology has to live in code, not in a prompt.
In Mitzu’s architecture the two layers work together. The semantic layer gives the Analytics Agent its vocabulary; the agent assembles an analysis specification — funnel steps, conversion window, breakdown; or cohort definition, return event, time granularity — and a deterministic query engine turns that specification into SQL. The agent never writes SQL. The same specification produces the same SQL and the same answer, every time.
| Layer | What it prevents | How |
|---|---|---|
| Semantic layer | Invented events, properties, and filter values | Catalog of real events, entities, and sampled values from the warehouse |
| Analysis specification | Ambiguity about what was computed | The agent’s output is a structured, inspectable spec — not freehand SQL |
| Deterministic query engine | Methodology errors in funnels, retention, cohorts | Product analytics methodology encoded in the engine, identical SQL for identical specs |
| SQL transparency | Unverifiable answers | Reviewable SQL behind every result, generated by the engine |
SQL transparency: trust needs an audit trail
Every answer in this architecture comes with the SQL the engine generated, open for inspection. Note what this changes about the review burden: the analyst is verifying the output of a deterministic engine — the same code path every funnel goes through — rather than auditing a fresh LLM generation per question. Verify the methodology once, and it holds for every answer the engine produces.
Precision matters when vendors say “hallucination-free.” In Mitzu, the SQL is deterministic — the engine cannot hallucinate a query. The agent’s natural-language summary is still model-generated prose, which is exactly why the SQL and the underlying charts stay one click away.
This combination — grounded vocabulary, deterministic SQL, reviewable output — is what lets a data team put an agent in front of PMs and marketers without becoming the fact-checking department. It is also what a warehouse-native architecture protects: the layer describes data that stays in your warehouse, a point covered in vendor-silo analytics agents vs warehouse-native.
To see the semantic layer built against a real schema, the AI analytics agents for your warehouse page shows how the Configuration Agent and Analytics Agent work together from setup to answer.
FAQ
What does a semantic layer do for an analytics agent?
It gives the agent a grounded vocabulary: which events, properties, and entities exist, how they join, and what real property values look like. Grounded in that catalog, the agent builds analyses from things that exist instead of guessing table and column semantics.
How is a product-analytics semantic layer different from a BI one?
BI semantic layers (dbt MetricFlow, Cube, LookML) define metrics, dimensions, and joins — a shape designed for dashboards, authored by hand in YAML. A product-analytics semantic layer stores events, entities, dimension properties, and sampled property values, and pairs with an engine that knows funnel, retention, and cohort methodology.
Does a semantic layer prevent AI hallucinations?
It prevents a specific class of them: invented events, properties, and filter values. Methodology errors need a second mechanism — a deterministic query engine so the SQL is generated by code rather than by the model. In Mitzu, the SQL output is deterministic; the agent’s prose summary is still worth reading against the charts.
Who maintains the semantic layer?
In Mitzu, the Configuration Agent builds it by scanning the warehouse, and analysts review and adjust in the app — no YAML files to version and maintain. Saved insights, dashboards, and named cohorts extend it automatically as the workspace gets used.
