Skip to content

Private AI12 min read

What is private AI? A plain-English guide for business owners

What is private AI? Private AI, private LLM and private GPT explained for business: the four ways to get it, what is inside, and EU, US and Canadian rules.

Illustration: four document cards orbit an AI chip inside a locked boundary ring, while a cloud outside cannot reach in.
On this page11 sections
  1. What is private AI?
  2. Private AI, private LLM and private GPT: what do the words mean?
  3. How private can AI be? Four steps of control
  4. What do the big AI vendors actually promise in November 2026?
  5. What is inside a private AI system?
  6. Do you need private AI? Seven signals
  7. What are the biggest myths about private AI?
  8. Why is private AI getting attention now?
  9. What do the rules say in the EU, the US and Canada?
  10. How do you start with private AI?
  11. Questions people ask

"Private AI" is now on every vendor's website, and it means something different on each one. Some mean software on a server in your office. Some mean a business plan with a no-training promise. Some mean a consumer feature. Other companies simply share the name, and Google's own "Private AI Compute" is a feature for phones and consumer apps, not a business private-LLM product.

This guide gives you one working definition, a decoder for the jargon, the four real ways to get private AI, and what the big vendors actually promise today. It is written to be skimmed: use the headings.

What is private AI?

Private AI means you keep control of three things: where your data lives, who can access it, and what it can be used for. In practice that includes:

  • Your prompts and documents are not used to train models other people use.
  • Data is stored and processed in the places you choose, for example a specific country.
  • Only the right people can see the right documents, and you can show who asked what.
  • You can change the model or the vendor without losing your own knowledge base.

HPE describes private AI as using an enterprise's own internal data as the foundation of a language model while keeping assets and intellectual property secure. IBM's 2026 breach report uses the related idea of AI sovereignty: control over where AI systems run, how data is processed and who has access. Cisco adds the national angle: AI developed, deployed and governed inside a country's legal and operational boundaries.

Sovereign AI is private AI plus jurisdiction: which country's laws govern the data and the compute. For most small businesses, "private" is the question to answer first and "sovereign" comes second.

Private AI, private LLM and private GPT: what do the words mean?

TermWhat people usually meanWhat to check
Private AIThe umbrella: AI with your data under your controlWho holds the data, and under what contract
Private LLMA language model you run or control: open-weight on your hardware or cloud, or dedicated hostingWhich model, which licence, who patches it
Private GPTAn informal name for a ChatGPT-style assistant over your own documentsWhich model sits underneath; GPT is OpenAI's term
On-premise AIAI on hardware you own, see our on-premise guideHardware, power, people to run it
Sovereign AIPrivate AI plus control of jurisdiction and locationWhere the compute physically is, and whose law applies
Shadow AIStaff using AI tools the company has not approvedWhat they are pasting into them

If a vendor says "private", ask one question: private from whom? From other customers, from the vendor's staff, from model training, from a foreign government's reach, or from your own employees. The answers differ.

How private can AI be? Four steps of control

From public chatbot to private AIFour steps of increasing control over AI and your data. A public chatbot runs on the vendor's servers. A business plan adds the vendor's contractual terms. A private cloud runs the AI inside your own cloud account. On-premises AI runs on hardware you own. Control rises at each step, and so does the amount you have to run yourself.More control, more to run yourselfPublicchatbotvendor's serversBusinessplanvendor's termsPrivatecloudyour cloudOn-premisesyour hardware
Private AI is a spectrum, not a switch. Each step up gives you more control and more to look after.

Control rises as you move right, and so does the work of running it.

OptionWhere your data livesWhat protects itTrade-off
1. Public chatbotVendor's serversThe vendor's consumer termsCheapest, least control. Fine for public information only
2. Business or enterprise planVendor's serversA contract: no training by default, admin controls, sometimes residencyEasy to adopt. You still trust the vendor's systems
3. Private cloudYour own cloud account or a dedicated tenantYour cloud security, network rules and keysMore control and more setup. Needs cloud skills
4. On-premisesHardware you ownYour physical and network securityMost control. You run and update everything

Most businesses end up with a mix: a business-tier plan for general work, and a private layer for the data that is too sensitive to leave.

What do the big AI vendors actually promise in November 2026?

"Not used for training" is common now. Residency and retention are where plans differ. This table summarises each vendor's own documentation as of November 2026. It changes often, so check the linked pages before you rely on it.

Vendor and planUsed for training by default?Residency and retention highlights
OpenAI ChatGPT Business, Enterprise, APINo (OpenAI, updated January 2026)Enterprise: storage in 10 regions including Canada, Europe, UK, US. Inference residency only in Europe, US, UAE. Business: storage region at sign-up, storage only. API: abuse logs kept up to 30 days by default. Canada API residency is storage only, not processing
Anthropic Claude commercial products and APINo (Anthropic, August 2026)First-party API offers US-only or global inference, no EU or Canada option. For EU in-region Claude, use a cloud provider such as Bedrock or Vertex. Zero data retention is available on request for the API
Microsoft 365 Copilot, Azure OpenAINoEU users covered by the EU Data Boundary. Canada: Advanced Data Residency covers stored data. Microsoft says in-country Copilot processing is expected in Canada in 2027
Google Gemini on VertexNo, without your permissionEU and US multi-region endpoints keep processing in that region. Canadian processing exists only for selected models
AWS BedrockNoGeographic cross-region inference keeps processing inside a geography, but across several regions. Single-region use is possible by direct invocation

Three things to take from it. First, storage and processing are different: data can be stored in Canada while the model runs elsewhere. Second, retention is a separate switch: ask about abuse-monitoring logs and zero data retention. Third, business plans are not private cloud: your data is still on the vendor's systems. For a deeper comparison of ownership and lock-in, see private AI vs ChatGPT Enterprise.

What is inside a private AI system?

The model is one part. Most of what makes a system private is the plumbing around it.

Inside a private AIYour team sends a question to an access gate inside your own boundary. The gate checks who is asking and keeps an audit log. A retrieval step reads only the documents that person is allowed to see from your data, and passes them to a model of your choice. The answer comes back to the team with sources. Nothing leaves the boundary.Inside your boundaryYourteamAccessgateRetrievalby permissionModelyour choiceYour datadocs, CRM, mailAudit logwho asked whatAnswer, with sources
The model is one part. The gate, the permissions and the log are what make it private.
  • Access gate: sign-in and rules for who can ask what.
  • Retrieval: the step that finds the right passages in your documents, but only ones that person is allowed to see. This is often called RAG, retrieval-augmented generation.
  • Your data: documents, CRM, email and files, left where they are and connected, rather than copied into someone else's tool.
  • Model: an open-weight model you host, or a vendor model under a dedicated agreement. You should be able to swap it.
  • Audit log: a record of who asked what and what was returned.

Notice what is missing: "training a model on your data". Most private AI does not train anything. It connects a model to your documents at question time. Our post on whether you can train AI on your company data explains why that is usually the better design.

Do you need private AI? Seven signals

You probably do if three or more are true:

  1. Staff handle customer records, patient, legal or financial information.
  2. Clients, insurers or regulators ask where your data goes.
  3. Contracts or professional rules restrict sharing data with third parties.
  4. You want AI to answer from your own documents, not from the public internet.
  5. You operate across borders and need data to stay in a country or region.
  6. Your team is already using public tools on work data. (Most are, see below.)
  7. You want to switch AI vendors later without rebuilding your knowledge.

If none apply, a business-tier plan with a written policy may be enough for now. See what not to share with ChatGPT at work for a policy starting point.

What are the biggest myths about private AI?

MythReality
"Private means offline."Private cloud is private and online. Control comes from contracts, keys and access rules, not from being disconnected
"A business plan is private enough."It may be, for some data. Check retention, residency and whether processing leaves your region
"Private AI means training our own model."Usually it means connecting a model to your documents. Training is rarely needed
"Private AI is only for big companies."You can start small: one dataset, one team, one use case
"Private AI cannot make things up."It controls where data goes, not whether answers are right. Keep review steps

Why is private AI getting attention now?

of breached organizations had a breach involving an AI model or app (IBM Cost of a Data Breach 2026)
21%
of those lacked proper AI access controls (IBM 2026)
92%
of security incidents involved shadow AI, up from 20% (IBM 2026)
43%

IBM's 2026 study covered 602 breached organizations between March 2025 and February 2026. Incidents involving shadow AI averaged $5.39 million. That is the average cost of those incidents, not an extra amount on top of a normal breach. Two-thirds (68%) of breached organizations lacked AI governance policies.

Employee behaviour is part of the picture. A KPMG and University of Melbourne study of more than 48,000 people in 47 countries (2025) found that almost half of employees admit misusing AI at work, including uploading sensitive company information to free public tools. A vendor survey by PagerDuty (June 2026, 1,250 office workers in Australia, Japan, the UK and US) reported that 88% had shared work information with public AI tools. Treat that second number as indicative, since it comes from a vendor.

On the buying side, Gartner forecasts worldwide sovereign cloud infrastructure spending of $80 billion in 2026, up 35.6%. That is cloud infrastructure, not AI alone. Vendor-sponsored surveys point the same way: Cloudera and Wakefield found 66% of 1,500 enterprise architects moved AI workloads from public cloud to private or on-premises in the past year, and a Broadcom-commissioned survey reported 56% using private cloud for AI inference. Both are vendor-backed, so read them as direction, not as proof.

What do the rules say in the EU, the US and Canada?

This is general information, not legal advice. Laws on AI are changing quickly.

European Union

  • GDPR applies whenever personal data goes into an AI tool. Your AI vendor is usually a processor, and you need a processing agreement, appropriate security and a lawful basis for any transfer outside the EU.
  • EU-US transfers: the EU's General Court upheld the EU-US Data Privacy Framework in September 2025. An appeal is pending, so keep a fallback plan.
  • EU AI Act: the AI-literacy duty has applied since February 2025. The Commission says there is no mandatory certificate, and national authorities enforce from August 2026. Under the AI Omnibus, in force since 27 July 2026, high-risk obligations moved to 2 December 2027 for stand-alone systems and 2 August 2028 for AI built into regulated products. Transparency duties, such as telling people they are talking to AI, still apply from August 2026.

United States

  • There is no single federal privacy law for AI. Sector rules apply: for example, health-data vendors need a HIPAA business associate agreement.
  • State laws are the main action. Colorado replaced its 2024 AI Act with SB 26-189, signed May 2026, with main obligations from 1 January 2027. California's rules on automated decision-making technology took effect in January 2026, with a later compliance date for existing uses.
  • Federal preemption is contested. A December 2025 executive order set up a federal push against state AI laws, but an executive order does not itself override state law, and Congress had not passed preemption as of mid-November 2026.

Canada

  • PIPEDA still governs how businesses handle personal information, including in AI tools. The Privacy Commissioner's generative-AI principles (December 2023) say existing privacy law applies, accountability stays with the organization using the tool, and prompts should not be retained or reused for secondary purposes unless required.
  • Quebec's Law 25 adds requirements for automated decisions and for transferring data outside Quebec.
  • Federal reform: Bill C-36, introduced on 15 June 2026, would replace parts of PIPEDA. It has had first reading only and is not law. Canada's national AI strategy, launched in June 2026, plans AI transparency measures over five years. There is no federal AI statute in force.
  • In October 2025 to 2026 the Commissioner and the Quebec, British Columbia and Alberta regulators concluded their joint investigation into OpenAI, finding that initial training did not comply and that OpenAI made changes.

How do you start with private AI?

  1. Write a one-page AI use policy. Say what can and cannot go into public tools. Do this before you buy anything.
  2. Map your sensitive data. List where customer, financial and legal data lives, and who needs access.
  3. Choose the lowest step that fits. Business plan, private cloud or on-premises. The decision flow in our on-premise guide helps.
  4. Pilot on one dataset and one team for 30 to 90 days with a measured goal.
  5. Test the answers. Ask 20 questions you already know the answers to. Check citations and permissions.
  6. Review quarterly. Vendor terms, models and laws change. Our sources below are dated for that reason.

Support exists but moves. Canada's Digital Adoption Program has closed to new applications. EU AI Factories run access calls that give SMEs and start-ups compute time, not cash. Check the current status of any programme before planning around it.

If you want help choosing a starting point, tell us your goal or see how we work. If you are comparing providers, read how to choose AI consulting services.

Sources

Questions people ask

What is private AI?

Private AI means running or buying AI so your company's data stays under your control: it is not used to train a vendor's public models, and it is kept in the places and under the access rules you choose. It can run on your own hardware, in your own cloud account, or in a dedicated, contractually protected vendor environment.

What is the difference between private AI and public AI?

Public AI tools such as free chatbots run on the vendor's servers under the vendor's terms, and your prompts may be stored or used for improvement unless you opt out. Private AI puts control with you: where the data lives, who can access it, how long it is kept and which model answers. The difference is control and contract, not how smart the model is.

What is a private LLM?

A private LLM is a large language model that you control, either an open-weight model running on your own hardware or cloud account, or a model in a dedicated vendor environment with strict data terms. Your prompts and documents are not shared with other customers or used to train public models. It is the engine inside a private AI system, not the whole system.

What is a private GPT?

Private GPT is an informal name for a ChatGPT-style assistant that answers from your own documents and runs where you control the data. GPT is OpenAI's term, so check which model is actually used. Most private GPT builds pair a language model with retrieval over your files, plus sign-in, permissions and logging.

Is private AI the same as on-premise AI?

No. On-premise AI is one form of private AI: it runs on hardware you own. Private AI also includes a private cloud, where the AI runs in your own cloud account, and dedicated business-tier vendor plans with contractual limits. On-premise gives the most control and needs the most upkeep.

Is ChatGPT Enterprise private AI?

It is a business-grade option, not full private AI. OpenAI states it does not train on Business, Enterprise or API data by default, and Enterprise offers data residency in several regions including Canada and Europe. But data still sits on OpenAI's systems, API logs are kept up to 30 days by default, and Canadian API residency covers storage, not processing.

Do small businesses need private AI?

Not always. If your team only drafts marketing copy from public information, a business-tier plan with a clear policy may be enough. Private AI makes sense when you handle customer records, contracts, health, legal or financial data, when clients or regulators ask where data goes, or when you want AI that learns your own documents without sharing them.

Can private AI still be wrong?

Yes. Private AI controls where your data goes, not whether the answers are correct. Any language model can state something false, so you still need approval steps for risky actions, source citations, and tests on your own examples. Privacy and accuracy are separate problems.

Keep reading

All posts

Want this working in your business?

Tell us what you want to get done. We will tell you if AI can help.