TL;DR
ChatGPT is good at data analysis on files you upload and at drafting or explaining SQL. On its own it does not know your metric definitions or permissions. Pointing ChatGPT or Claude straight at the warehouse lets the model write its own SQL for every question, so two people can get two revenue numbers. Connecting the assistant to an analytics MCP server keeps the chat interface but answers from the governed metrics and permissions your data team set.
ChatGPT data analysis works well when the data fits in a file you can upload, or when you need SQL drafted or explained. It gets harder when the question is about your company's own metrics: what was net revenue last week, which channel drove the most activated users, why did retention drop. Those answers depend on live warehouse data, agreed metric definitions, permissions and logic someone can review. This post covers where ChatGPT helps, where it breaks, the three ways teams connect it to company data, and where a purpose-built AI analytics agent fits.
What ChatGPT is actually good at for data work?
- Analyzing a CSV or spreadsheet you upload, with Python it writes and runs.
- Drafting SQL quickly from plain-English intent.
- Explaining complex query logic in simpler terms.
- Suggesting data model patterns and dbt-style transformations.
- Generating starter scripts for data cleanup or analysis.
The right framing is not "ChatGPT is bad at data". It is a capable general assistant, and for drafting, exploration and thinking support it is excellent. The limits show up when its answer becomes a number someone reports.
The five things ChatGPT cannot do for analytics on its own
1) It only sees the data you give it
Out of the box, you export, upload or paste data. For production-scale questions that is often impractical and sometimes not permitted. See OpenAI security and privacy for how hosted chat products handle customer data.
2) It does not know your metric definitions
Even with schema snippets, it lacks durable grounding in your definitions. "Activated user" in your model may be a strict lifecycle condition, not a generic event count, and "revenue" may exclude refunds and tax.
3) Without a connection, it cannot run queries on your warehouse
You copy SQL out, paste it into your query tool, run it, interpret the output and iterate by hand. That loop is useful for analysts but it is not self-service for the rest of the company. Connectors change this, with trade-offs covered below.
4) It can be confidently wrong
A wrong join looks as polished as a correct one. So does a funnel with no conversion window, or a count that includes the same user twice. Without checks against your real model, confidence is not quality.
5) It does not keep a governed data model over time
Teams re-explain terms, table roles and edge cases in every conversation. Each person's prompt becomes a private definition of the metric, which is how a company ends up with several versions of the same number.
Three ways to do ChatGPT data analysis on company data
Teams that want to ask ChatGPT or Claude about company data usually try one of three setups. They feel the same in the chat window. They differ in what the model answers from and who writes the SQL.
- Upload a CSV. Someone exports a table or a report and the assistant analyzes the file. Quick for a one-off, but the data is a copy, it is stale from the moment it is exported, and the definitions are whatever went into the export.
- Point the LLM at the warehouse. A warehouse connector or MCP server lets the assistant read schemas and run SQL. The data is live, but the model writes the SQL for every question from table and column names, so the same question can return different numbers, and the connection's warehouse role decides what it can see.
- Connect an analytics MCP server. The assistant calls an analytics agent that answers from the metric definitions and permissions the data team set. The model asks the question; the governed layer produces the answer and the SQL behind it.
| Upload a CSV | LLM on the warehouse | Analytics MCP server | |
|---|---|---|---|
| What the model sees | A file someone exported | Raw tables and schemas | Answers from an analytics agent |
| Who writes the SQL | Nobody; the model analyzes the file | The language model, per question | Depends on the agent. In Mitzu, a deterministic engine |
| Metric definitions | Whatever went into the export | Inferred from column names, or a semantic model you maintain | The governed metrics the data team defined |
| Permissions | Whoever has the file | The warehouse role behind the connection | Each person's role in the analytics platform |
| Same question, same answer | Only for the same file | Not guaranteed | With deterministic SQL |
| Good for | One-off exploration of a small dataset | Engineers exploring tables and drafting SQL | Business questions on numbers the company reports |
Mitzu's MCP server is an example of the third setup. It gives ChatGPT, Claude, Cursor, GitHub Copilot and any other MCP client one tool: Mitzu's analytics agent. The assistant asks a question and gets back an answer built on your governed metrics, with the SQL behind it. Each person sees only what their role allows, nobody needs a warehouse account, and the assistant receives the answer, not your warehouse. The MCP server launch post has the details.
What a purpose-built AI analytics agent does differently?
- Connected to live warehouse data, not pasted extracts.
- Grounded in metric definitions and a semantic layer the data team owns.
- Runs the analysis and returns results directly, including follow-ups such as why a number moved.
- Shows the SQL behind each answer for verification.
- Applies each person's permissions, wherever the question is asked.
This is the core of LLM vs analytics agent. A general LLM is a capable assistant for drafting and explanation. An analytics agent is built to deliver trusted answers inside your real data environment. With MCP the two are not either-or: the assistant can be the interface and the agent the source of the numbers.
A side-by-side scenario
Question: "Which marketing channel drove the most activated users last month?"
With ChatGPT on its own, a growth manager gets a plausible SQL template and then asks, "What are the right table names?" They run and rerun queries, then still ask a data analyst whether the "activated" definition is correct.
With an AI analytics agent, the manager asks in the analytics app, in Slack, or in ChatGPT through the MCP server. The agent uses the governed definition of an activated user, runs the analysis on the current warehouse data, and returns ranked channels with the SQL. If the result looks surprising, the analyst checks the SQL instead of rebuilding the query. Decision time drops from days to minutes.
When to use ChatGPT for data work (and when not to)?
Use ChatGPT on its own for drafting SQL, explaining queries, scaffolding transformations and exploring files you upload. Use a governed analytics agent for production metrics, custom business definitions, company-wide self-service, and anything that needs an auditable answer. If your team already lives in ChatGPT or Claude, connect the agent through MCP so they get both.
What to look for in an AI analytics agent?
- Runs on your warehouse without copying data out.
- Visible SQL for every answer.
- Metric definitions the data team controls.
- Permissions that follow each person into Slack and AI assistants.
- Clear behavior when it cannot answer, instead of fabricated confidence.
Mitzu is an analytics platform on your data warehouse for BI, product analytics and experiment analysis, where the data team governs the metrics and everyone uses them through dashboards, an AI agent, Slack and their own AI assistants. Its analytics agent builds analysis specifications that a deterministic engine turns into SQL, so the same question gets the same answer. If you have been stretching ChatGPT for analytics beyond what it was built for, start free on your warehouse or book a demo.
FAQ
Can you use ChatGPT for data analysis?
Yes, especially for analyzing files you upload, drafting SQL, debugging logic and explaining analysis methods. It is less suited, on its own, to being the source of company metrics, because it does not know your definitions or permissions.
What is the difference between ChatGPT and an AI analytics agent?
ChatGPT is a general assistant. An AI analytics agent is connected to live warehouse data, runs analyses on governed metric definitions, and returns verifiable answers with the SQL behind them. Through MCP, ChatGPT can call an analytics agent and pass its answer on.
Can ChatGPT query my data warehouse?
Not by default. With a warehouse connector or MCP server it can run SQL, but then the model writes that SQL itself from table and column names, under whatever role the connection uses. An analytics MCP server is the alternative: the assistant asks the question and the answer comes from governed definitions, with each person's permissions.
Is it safe to upload company data to ChatGPT?
That depends on your company's AI policy and your ChatGPT plan's data controls. Exports also go stale and escape the permissions set in the warehouse. With an analytics MCP server, the data stays in the warehouse and the assistant receives only the answer.
