On this page11 sections
- What is private AI?
- Private AI, private LLM and private GPT: what do the words mean?
- How private can AI be? Four steps of control
- What do the big AI vendors actually promise in November 2026?
- What is inside a private AI system?
- Do you need private AI? Seven signals
- What are the biggest myths about private AI?
- Why is private AI getting attention now?
- What do the rules say in the EU, the US and Canada?
- How do you start with private AI?
- Questions people ask
"Private AI" is now on every vendor's website, and it means something different on each one. Some mean software on a server in your office. Some mean a business plan with a no-training promise. Some mean a consumer feature. Other companies simply share the name, and Google's own "Private AI Compute" is a feature for phones and consumer apps, not a business private-LLM product.
This guide gives you one working definition, a decoder for the jargon, the four real ways to get private AI, and what the big vendors actually promise today. It is written to be skimmed: use the headings.
What is private AI?
Private AI means you keep control of three things: where your data lives, who can access it, and what it can be used for. In practice that includes:
- Your prompts and documents are not used to train models other people use.
- Data is stored and processed in the places you choose, for example a specific country.
- Only the right people can see the right documents, and you can show who asked what.
- You can change the model or the vendor without losing your own knowledge base.
HPE describes private AI as using an enterprise's own internal data as the foundation of a language model while keeping assets and intellectual property secure. IBM's 2026 breach report uses the related idea of AI sovereignty: control over where AI systems run, how data is processed and who has access. Cisco adds the national angle: AI developed, deployed and governed inside a country's legal and operational boundaries.
Sovereign AI is private AI plus jurisdiction: which country's laws govern the data and the compute. For most small businesses, "private" is the question to answer first and "sovereign" comes second.
Private AI, private LLM and private GPT: what do the words mean?
| Term | What people usually mean | What to check |
|---|---|---|
| Private AI | The umbrella: AI with your data under your control | Who holds the data, and under what contract |
| Private LLM | A language model you run or control: open-weight on your hardware or cloud, or dedicated hosting | Which model, which licence, who patches it |
| Private GPT | An informal name for a ChatGPT-style assistant over your own documents | Which model sits underneath; GPT is OpenAI's term |
| On-premise AI | AI on hardware you own, see our on-premise guide | Hardware, power, people to run it |
| Sovereign AI | Private AI plus control of jurisdiction and location | Where the compute physically is, and whose law applies |
| Shadow AI | Staff using AI tools the company has not approved | What they are pasting into them |
If a vendor says "private", ask one question: private from whom? From other customers, from the vendor's staff, from model training, from a foreign government's reach, or from your own employees. The answers differ.
How private can AI be? Four steps of control
Control rises as you move right, and so does the work of running it.
| Option | Where your data lives | What protects it | Trade-off |
|---|---|---|---|
| 1. Public chatbot | Vendor's servers | The vendor's consumer terms | Cheapest, least control. Fine for public information only |
| 2. Business or enterprise plan | Vendor's servers | A contract: no training by default, admin controls, sometimes residency | Easy to adopt. You still trust the vendor's systems |
| 3. Private cloud | Your own cloud account or a dedicated tenant | Your cloud security, network rules and keys | More control and more setup. Needs cloud skills |
| 4. On-premises | Hardware you own | Your physical and network security | Most control. You run and update everything |
Most businesses end up with a mix: a business-tier plan for general work, and a private layer for the data that is too sensitive to leave.
What do the big AI vendors actually promise in November 2026?
"Not used for training" is common now. Residency and retention are where plans differ. This table summarises each vendor's own documentation as of November 2026. It changes often, so check the linked pages before you rely on it.
| Vendor and plan | Used for training by default? | Residency and retention highlights |
|---|---|---|
| OpenAI ChatGPT Business, Enterprise, API | No (OpenAI, updated January 2026) | Enterprise: storage in 10 regions including Canada, Europe, UK, US. Inference residency only in Europe, US, UAE. Business: storage region at sign-up, storage only. API: abuse logs kept up to 30 days by default. Canada API residency is storage only, not processing |
| Anthropic Claude commercial products and API | No (Anthropic, August 2026) | First-party API offers US-only or global inference, no EU or Canada option. For EU in-region Claude, use a cloud provider such as Bedrock or Vertex. Zero data retention is available on request for the API |
| Microsoft 365 Copilot, Azure OpenAI | No | EU users covered by the EU Data Boundary. Canada: Advanced Data Residency covers stored data. Microsoft says in-country Copilot processing is expected in Canada in 2027 |
| Google Gemini on Vertex | No, without your permission | EU and US multi-region endpoints keep processing in that region. Canadian processing exists only for selected models |
| AWS Bedrock | No | Geographic cross-region inference keeps processing inside a geography, but across several regions. Single-region use is possible by direct invocation |
Three things to take from it. First, storage and processing are different: data can be stored in Canada while the model runs elsewhere. Second, retention is a separate switch: ask about abuse-monitoring logs and zero data retention. Third, business plans are not private cloud: your data is still on the vendor's systems. For a deeper comparison of ownership and lock-in, see private AI vs ChatGPT Enterprise.
What is inside a private AI system?
The model is one part. Most of what makes a system private is the plumbing around it.
- Access gate: sign-in and rules for who can ask what.
- Retrieval: the step that finds the right passages in your documents, but only ones that person is allowed to see. This is often called RAG, retrieval-augmented generation.
- Your data: documents, CRM, email and files, left where they are and connected, rather than copied into someone else's tool.
- Model: an open-weight model you host, or a vendor model under a dedicated agreement. You should be able to swap it.
- Audit log: a record of who asked what and what was returned.
Notice what is missing: "training a model on your data". Most private AI does not train anything. It connects a model to your documents at question time. Our post on whether you can train AI on your company data explains why that is usually the better design.
Do you need private AI? Seven signals
You probably do if three or more are true:
- Staff handle customer records, patient, legal or financial information.
- Clients, insurers or regulators ask where your data goes.
- Contracts or professional rules restrict sharing data with third parties.
- You want AI to answer from your own documents, not from the public internet.
- You operate across borders and need data to stay in a country or region.
- Your team is already using public tools on work data. (Most are, see below.)
- You want to switch AI vendors later without rebuilding your knowledge.
If none apply, a business-tier plan with a written policy may be enough for now. See what not to share with ChatGPT at work for a policy starting point.
What are the biggest myths about private AI?
| Myth | Reality |
|---|---|
| "Private means offline." | Private cloud is private and online. Control comes from contracts, keys and access rules, not from being disconnected |
| "A business plan is private enough." | It may be, for some data. Check retention, residency and whether processing leaves your region |
| "Private AI means training our own model." | Usually it means connecting a model to your documents. Training is rarely needed |
| "Private AI is only for big companies." | You can start small: one dataset, one team, one use case |
| "Private AI cannot make things up." | It controls where data goes, not whether answers are right. Keep review steps |
Why is private AI getting attention now?
- of breached organizations had a breach involving an AI model or app (IBM Cost of a Data Breach 2026)
- 21%
- of those lacked proper AI access controls (IBM 2026)
- 92%
- of security incidents involved shadow AI, up from 20% (IBM 2026)
- 43%
IBM's 2026 study covered 602 breached organizations between March 2025 and February 2026. Incidents involving shadow AI averaged $5.39 million. That is the average cost of those incidents, not an extra amount on top of a normal breach. Two-thirds (68%) of breached organizations lacked AI governance policies.
Employee behaviour is part of the picture. A KPMG and University of Melbourne study of more than 48,000 people in 47 countries (2025) found that almost half of employees admit misusing AI at work, including uploading sensitive company information to free public tools. A vendor survey by PagerDuty (June 2026, 1,250 office workers in Australia, Japan, the UK and US) reported that 88% had shared work information with public AI tools. Treat that second number as indicative, since it comes from a vendor.
On the buying side, Gartner forecasts worldwide sovereign cloud infrastructure spending of $80 billion in 2026, up 35.6%. That is cloud infrastructure, not AI alone. Vendor-sponsored surveys point the same way: Cloudera and Wakefield found 66% of 1,500 enterprise architects moved AI workloads from public cloud to private or on-premises in the past year, and a Broadcom-commissioned survey reported 56% using private cloud for AI inference. Both are vendor-backed, so read them as direction, not as proof.
What do the rules say in the EU, the US and Canada?
This is general information, not legal advice. Laws on AI are changing quickly.
European Union
- GDPR applies whenever personal data goes into an AI tool. Your AI vendor is usually a processor, and you need a processing agreement, appropriate security and a lawful basis for any transfer outside the EU.
- EU-US transfers: the EU's General Court upheld the EU-US Data Privacy Framework in September 2025. An appeal is pending, so keep a fallback plan.
- EU AI Act: the AI-literacy duty has applied since February 2025. The Commission says there is no mandatory certificate, and national authorities enforce from August 2026. Under the AI Omnibus, in force since 27 July 2026, high-risk obligations moved to 2 December 2027 for stand-alone systems and 2 August 2028 for AI built into regulated products. Transparency duties, such as telling people they are talking to AI, still apply from August 2026.
United States
- There is no single federal privacy law for AI. Sector rules apply: for example, health-data vendors need a HIPAA business associate agreement.
- State laws are the main action. Colorado replaced its 2024 AI Act with SB 26-189, signed May 2026, with main obligations from 1 January 2027. California's rules on automated decision-making technology took effect in January 2026, with a later compliance date for existing uses.
- Federal preemption is contested. A December 2025 executive order set up a federal push against state AI laws, but an executive order does not itself override state law, and Congress had not passed preemption as of mid-November 2026.
Canada
- PIPEDA still governs how businesses handle personal information, including in AI tools. The Privacy Commissioner's generative-AI principles (December 2023) say existing privacy law applies, accountability stays with the organization using the tool, and prompts should not be retained or reused for secondary purposes unless required.
- Quebec's Law 25 adds requirements for automated decisions and for transferring data outside Quebec.
- Federal reform: Bill C-36, introduced on 15 June 2026, would replace parts of PIPEDA. It has had first reading only and is not law. Canada's national AI strategy, launched in June 2026, plans AI transparency measures over five years. There is no federal AI statute in force.
- In October 2025 to 2026 the Commissioner and the Quebec, British Columbia and Alberta regulators concluded their joint investigation into OpenAI, finding that initial training did not comply and that OpenAI made changes.
How do you start with private AI?
- Write a one-page AI use policy. Say what can and cannot go into public tools. Do this before you buy anything.
- Map your sensitive data. List where customer, financial and legal data lives, and who needs access.
- Choose the lowest step that fits. Business plan, private cloud or on-premises. The decision flow in our on-premise guide helps.
- Pilot on one dataset and one team for 30 to 90 days with a measured goal.
- Test the answers. Ask 20 questions you already know the answers to. Check citations and permissions.
- Review quarterly. Vendor terms, models and laws change. Our sources below are dated for that reason.
Support exists but moves. Canada's Digital Adoption Program has closed to new applications. EU AI Factories run access calls that give SMEs and start-ups compute time, not cash. Check the current status of any programme before planning around it.
If you want help choosing a starting point, tell us your goal or see how we work. If you are comparing providers, read how to choose AI consulting services.
Sources
- HPE, What is private AI?
- IBM, Cost of a Data Breach Report 2026, July 2026; IBM, press release, July 2026
- Cisco, What is sovereign AI?, June 2026
- Google, Private AI Compute, November 2025
- OpenAI, Enterprise privacy, January 2026; API data controls; API data residency; ChatGPT data residency
- Anthropic, Is my data used for model training?, August 2026; Data residency
- Microsoft, Microsoft 365 Copilot privacy; in-country data processing announcement, November 2025, updated April 2026
- Google Cloud, Data residency
- AWS, Bedrock data retention; Bedrock cross-region inference, August 2026
- KPMG and University of Melbourne, Trust, attitudes and use of AI, April 2025
- PagerDuty, Shadow AI workplace survey, June 2026 (vendor survey)
- Gartner, Sovereign cloud IaaS spending forecast, February 2026
- Virtualization Review, Cloudera survey on AI workload repatriation, August 2026 (vendor survey); Network World, Broadcom on private cloud for AI inference, June 2026 (vendor survey)
- European Commission, AI literacy Q&A, October 2026; European Parliament, AI Omnibus procedure file
- Colorado General Assembly, SB 26-189; California Privacy Protection Agency, CCPA updates; HHS, HIPAA business associates
- Office of the Privacy Commissioner of Canada, Principles for responsible, trustworthy and privacy-protective generative AI technologies, December 2023; speech, October 2026
- Parliament of Canada, Bill C-36; Prime Minister of Canada, AI for All national strategy, June 2026
- BDC, Canada Digital Adoption Program; EuroHPC, AI Factories access calls
Questions people ask
What is private AI?
Private AI means running or buying AI so your company's data stays under your control: it is not used to train a vendor's public models, and it is kept in the places and under the access rules you choose. It can run on your own hardware, in your own cloud account, or in a dedicated, contractually protected vendor environment.
What is the difference between private AI and public AI?
Public AI tools such as free chatbots run on the vendor's servers under the vendor's terms, and your prompts may be stored or used for improvement unless you opt out. Private AI puts control with you: where the data lives, who can access it, how long it is kept and which model answers. The difference is control and contract, not how smart the model is.
What is a private LLM?
A private LLM is a large language model that you control, either an open-weight model running on your own hardware or cloud account, or a model in a dedicated vendor environment with strict data terms. Your prompts and documents are not shared with other customers or used to train public models. It is the engine inside a private AI system, not the whole system.
What is a private GPT?
Private GPT is an informal name for a ChatGPT-style assistant that answers from your own documents and runs where you control the data. GPT is OpenAI's term, so check which model is actually used. Most private GPT builds pair a language model with retrieval over your files, plus sign-in, permissions and logging.
Is private AI the same as on-premise AI?
No. On-premise AI is one form of private AI: it runs on hardware you own. Private AI also includes a private cloud, where the AI runs in your own cloud account, and dedicated business-tier vendor plans with contractual limits. On-premise gives the most control and needs the most upkeep.
Is ChatGPT Enterprise private AI?
It is a business-grade option, not full private AI. OpenAI states it does not train on Business, Enterprise or API data by default, and Enterprise offers data residency in several regions including Canada and Europe. But data still sits on OpenAI's systems, API logs are kept up to 30 days by default, and Canadian API residency covers storage, not processing.
Do small businesses need private AI?
Not always. If your team only drafts marketing copy from public information, a business-tier plan with a clear policy may be enough. Private AI makes sense when you handle customer records, contracts, health, legal or financial data, when clients or regulators ask where data goes, or when you want AI that learns your own documents without sharing them.
Can private AI still be wrong?
Yes. Private AI controls where your data goes, not whether the answers are correct. Any language model can state something false, so you still need approval steps for risky actions, source citations, and tests on your own examples. Privacy and accuracy are separate problems.




