AI and Data Privacy at Work: The Rules That Actually Prevent the Incident

TL;DR: Almost every AI privacy incident at a normal company has the same shape: an employee pastes something sensitive into a consumer-tier tool, because no one told them the difference between that and the enterprise version. Prevention is three layers: a concrete never-paste list everyone knows, business-tier tools whose contracts exclude training on your data, and PII habits, minimize, redact, retrieve-don’t-paste. This page covers each layer. General guidance, not legal advice.

The threat model, stated plainly

When you evaluate AI privacy risk, it helps to name what you’re actually worried about, because the mitigations differ:

1. Training exposure. Data entered into a tool is used to train future models, and fragments could theoretically surface in outputs later. This is the risk everyone cites; it’s real but tier-dependent, and it’s the easiest to eliminate contractually.
2. Vendor breach or insider access. Your prompts sit on the vendor’s infrastructure under the vendor’s retention policy. A breach there is a breach of your data. This risk exists at every tier and is managed by vendor selection, see the AI security review.
3. Legal disclosure. Entering personal or confidential data into a third-party service is, legally, giving it to a third party. NDAs, privilege, and privacy law don’t care that the recipient was a model. This is the risk companies most underweight.
4. Retention and discovery. Chat histories are records. They can be retained by the vendor, requested by regulators, or surface in litigation. Consumer accounts tied to personal emails put those records entirely outside your control.

Notice what’s not on the list: the model “remembering” your conversation and telling your competitor. A large language model doesn’t retain your session in any usable per-customer way at inference time; the exposure paths are training pipelines, logs, and retention, which is exactly why terms and tiers, not vibes, are the unit of analysis.

Layer 1: The never-paste list

Rules like “be careful with sensitive data” don’t survive contact with a deadline. A list does. This is the same list that anchors the data section of the acceptable-use policy; it belongs in both places.

Never enter the following into any AI tool that hasn’t been explicitly approved for that data class:

The heuristic that makes it memorable, from the acceptable-use playbook: if you wouldn’t paste it into a public web form, don’t paste it into an unapproved AI tool.

Two refinements that keep the list honest:

• The list gates unapproved tools, not all tools. An enterprise deployment with a signed data-processing agreement and no-training terms can be cleared for several of these categories. That clearance is a per-tool, per-category decision, recorded in your approved-tools list, never a blanket assumption.
• Watch the side doors. The paste into a chat window is the obvious path. The AI notetaker in a client call, the email plugin reading threads, the coding assistant indexing a repo, and the “AI summary” feature inside your CRM all move the same data with less friction and less thought. Scope them in explicitly.

Layer 2: Consumer vs enterprise terms, legally different products

The single highest-leverage fact in AI privacy: the consumer and business tiers of the same product are different products under different contracts. Same model, same interface, different legal reality.

The word “typical” is doing deliberate work in that table. Vendors differ, plans differ, and terms change, a vendor’s data-usage policy this quarter may not be its policy next quarter. The operational rule: read the current data-usage terms for the exact plan you’re buying, capture the no-training and retention commitments in writing, and recheck at renewal. This check is a standing item in the security review checklist.

Why the account owner row matters more than it looks: when an employee uses a personal consumer account for work, the chat history, containing whatever they pasted, lives in an account you can’t audit, can’t wipe on offboarding, and can’t produce or protect in a dispute. Even if that employee never pastes anything sensitive, you can’t demonstrate that. Provisioned business accounts turn an unknowable into a control.

What “training on your data” actually means

Worth demystifying, because both the panic and the complacency come from vagueness. When a vendor may use inputs for training:

• Your conversations can enter datasets used to train or fine-tune future models, subject to the vendor’s filtering and de-identification processes.
• The realistic risk is usually not verbatim regurgitation to a competitor, it’s that you’ve disclosed the data to the vendor for a purpose (model improvement) your customers never consented to and your NDAs don’t permit. The legal exposure arrives at the moment of entry, regardless of whether anything ever resurfaces.
• Opt-outs help but are weaker than contractual exclusions: they’re settings (which reset, or which a given employee never toggled), not commitments.

When a business tier excludes training contractually, this whole branch of risk closes. The vendor still processes your data to serve responses, that’s what the DPA governs, but it doesn’t get to keep learning from it.

Layer 3: PII handling when AI legitimately needs the data

Some use cases genuinely involve personal data, support teams summarizing tickets, HR drafting from case notes, finance reconciling customer records. “Never” isn’t a workable rule there; discipline is.

1. Minimize first. Ask what the task actually needs. Summarizing a complaint doesn’t need the customer’s email and account number; classifying tickets doesn’t need names at all. Strip what the task doesn’t require before it enters the prompt.
2. Redact or pseudonymize where identity is irrelevant. “Customer A, enterprise plan, 14 months tenure” preserves the analytical signal of most tasks. Some enterprise AI platforms and gateways offer automatic PII redaction, a meaningful selection criterion for your approved-tools list.
3. Use an approved tool, on the approved tier, for that data class. The clearance work from Layer 2 exists precisely so this step is a lookup, not a judgment call made at 5:45 pm.
4. Prefer retrieval over pasting. Architecturally, retrieval-augmented generation fetches relevant records at answer time under your access controls, instead of employees copy-pasting records into prompts. Data stays in governed systems; the prompt carries a question, not a database.
5. Log the use case. Recurring PII-in-AI workflows should each pass a quick risk assessment and land in your register. This is also where GDPR-style obligations (records of processing, possibly a DPIA for higher-risk processing) attach, a call for your counsel or DPO, not this page.
6. Handle outputs like inputs. An AI-generated summary of personal data is personal data. It inherits the classification, storage rules, and retention limits of its source.

The three habits to institutionalize

If the layers above are the system, these are the behaviors that keep it alive:

• Provision, don’t permit. Don’t just allow business-tier tools, buy them, provision accounts, and make them the path of least resistance. Data flows toward convenience; make the safe tool the convenient one.
• Recheck terms on a calendar. Vendor data-usage terms change quietly. Twice a year, the owner of the approved-tools list re-reads the data terms of every approved tool. Fifteen minutes per tool; it’s the audit that catches drift.
• Make self-reporting safe. The employee who pastes the wrong thing and tells you same-day has given you options (vendor deletion requests, notification analysis, control fixes). The one who stays quiet has given you a time bomb. Your policy should say, explicitly, that prompt self-reporting is treated as doing the right thing.

Teams handling legal-privileged material have an extra layer of considerations, privilege waiver risk in particular, covered from the practice side in the legal guides.

FAQ

Do AI companies train on the data I enter?

Tier-dependent. Consumer tiers of major tools have historically used conversations for model improvement by default or unless the user opts out; business and enterprise tiers typically exclude training on customer data by contract. Policies vary by vendor and change over time, read the current terms for the exact plan, and get the commitment in writing.

What data should never go into an AI tool?

Into unapproved or consumer-tier tools: personal data about customers or employees, anything under NDA or client confidentiality, credentials, unreleased financials, trade secrets, and industry-regulated data. Approved business-tier tools with DPAs and no-training terms can be cleared for specific categories, recorded per tool in your approved-tools list, never assumed.

Is it a GDPR violation to paste customer data into ChatGPT?

It can be. Entering personal data into a third-party service is a disclosure to a processor, which requires a lawful basis, a data-processing agreement, and appropriate safeguards. A personal free-tier account provides none of these; an enterprise deployment with a DPA can. The specifics depend on your data, jurisdiction, and setup, verify with a qualified professional.

Are enterprise AI tools actually private?

Materially more private: contractual no-training commitments, a DPA, admin controls, SSO, configurable retention, and security attestations. Not magically private: your data still leaves your network and sits with a vendor whose security you’re trusting, which is why the security review exists even for enterprise tiers.

How should we handle PII we legitimately need AI to process?

Minimize (send only what the task needs), redact or pseudonymize where identity is irrelevant, use a tool approved for that data class, prefer retrieval-based architectures over pasting records into prompts, log the workflow in your risk register, and treat AI outputs derived from personal data as personal data.

Get one practical AI implementation brief per week, join the free newsletter.