The AI Tool Security Review: A Checklist That Fits in an Afternoon

TL;DR: Every AI tool request should pass one review before it touches company data, and that review can be a 2-4 hour desk exercise for most tools, not a procurement saga. Five parts: vendor posture, data handling, access control, integration scope, and AI-specific risks. This page is the checklist, plus a pre-screen that kills bad candidates in five minutes and a scaling rule so deep reviews are reserved for tools that earn them.

Why AI tools need their own review

You probably already have some vendor-vetting instinct. AI tools stress it in four places ordinary SaaS doesn’t:

1. The input is unstructured and unbounded. A payroll system receives payroll data; an AI chat tool receives whatever anyone types, which over time approximates everything the company knows. You’re not reviewing a data flow, you’re reviewing a data funnel.
2. Training terms are a new clause. Whether the vendor may learn from your inputs is a question that didn’t exist in classic SaaS review. It’s now the single most consequential line in the terms, covered in depth in AI and data privacy.
3. Integrations grant broad read access. “Connect your email/drive/CRM for better answers” is the standard AI pitch. That’s an OAuth grant giving a third party programmatic access to your most sensitive stores, often scoped far wider than the feature needs.
4. The tool can be manipulated by content. Prompt injection means an AI tool that reads untrusted content (inbound email, web pages, uploaded docs) can be given instructions by an attacker through that content. No traditional SaaS review question catches this.

The review below folds all four into a repeatable process. Its output feeds directly into your approved-tools list: a tool that passes gets a row; the conditions it passed under become that row’s “approved for” scope.

Step 0: The five-minute pre-screen

Before spending hours, check four disqualifiers. Any “no” ends the review:

• Is there a business/enterprise tier at all? Consumer-only products can’t offer the contractual data terms work use requires.
• Does the vendor publish a security page and offer a DPA? A vendor that can’t produce either isn’t ready for business customers, whatever the product demo says.
• Are the data-usage terms findable and readable? If you can’t determine from public documentation whether inputs are used for training, that opacity is itself the answer.
• Is the vendor identifiable and accountable? Real company, real jurisdiction, a way to serve legal process. This kills the long tail of wrapper apps with a landing page and a Stripe account, many of which pipe your data through third-party models under terms nobody has read.

The five-part review

Part 1: Vendor security posture

You’re assessing whether this company can protect data at all, before asking what it does with yours.

The subprocessor row deserves emphasis for AI products: many are interfaces over third-party models. Your data’s effective home is the model provider’s infrastructure and terms, so the review has to follow the chain one hop down.

Part 2: Data handling and training terms

The heart of the review, summarized here, treated fully in AI and data privacy:

• Training: Are customer inputs used to train or improve models? You want a contractual exclusion on the business tier, not a user-level opt-out.
• Retention: How long are prompts and outputs kept? Is retention configurable? Is there a deletion path you can invoke, including at offboarding?
• Residency: Where is data processed and stored? Matters for GDPR transfers and any sector rules you’re under.
• DPA: Available and signable on the tier you’re buying, with the subprocessor list attached.
• Human review: Do vendor employees or contractors read customer conversations (e.g., for abuse monitoring or quality)? Under what controls?

Record the answers, not just a pass/fail, “approved, but retention is 30 days fixed and vendor staff may review flagged content” is exactly the nuance the approved-tools list columns exist to carry.

Part 3: Access and authentication

• SSO support (and whether it’s paywalled into a tier above the one you planned to buy, a common surprise).
• MFA enforceable for all users.
• Role-based access, can admins scope who can use which features, connect which integrations?
• Offboarding, deprovisioning through your identity provider, and what happens to a departed user’s history.
• Audit logs, can an admin see who used the tool, with which integrations? (You need this the day something goes wrong.)

Part 4: Integration scope

For every connection the tool requests to your systems, ask three questions:

1. What exactly can it read? “Connect Google Drive” may mean every file the user can see, not the one folder the use case needs. Prefer tools that support scoped grants (specific folders, labels, channels).
2. What can it write or do? Read-only summarization is one risk class; send-email, edit-records, execute-code is another entirely. Write access moves the tool into your deepest review tier automatically.
3. Does the access persist? Standing OAuth grants keep working nights and weekends, on the vendor’s side of the fence. Prefer per-session or narrowly scoped tokens where offered; inventory standing grants where not.

The integration questions get sharper as tools become agentic, an AI agent that plans and executes multi-step tasks holds the same grants with more autonomy in how it uses them.

Part 5: AI-specific risks

The section most generic vendor questionnaires miss:

• Prompt injection surface. Does the tool read content the user didn’t write, inbound email, web pages, shared documents? Combined with any ability to act (Part 4’s write access), that’s the injection risk profile. Ask the vendor what guardrails exist: instruction filtering, action confirmation, sandboxing of untrusted content.
• Output handling defaults. Where do outputs go, retained in the vendor’s history, shareable by public link, inserted into records automatically? Public-link sharing defaults have burned more companies than model failures have.
• Autonomy limits. For anything agentic: can autonomous actions be capped (require human confirmation for sends, purchases, deletions)? Is there a kill switch an admin controls?
• Hallucination posture. Not a security flaw in the strict sense, but reviewable: does the tool cite sources for factual claims, and does the vendor document accuracy limitations honestly? A tool whose marketing denies hallucination exists is telling you how it will handle every other hard question.

Scaling the review to risk

Not every tool deserves the full treatment. Tier it:

The distinction between this review and the risk assessment is worth keeping crisp: the security review vets the tool; the risk assessment vets the use. A tool that passes review for drafting marketing copy hasn’t thereby passed for processing HR complaints, same tool, different use, different assessment.

Making it stick

1. Template the request. A one-page form, tool, tier, use case, data classes, integrations requested, filed by whoever wants the tool. Half the review is the requester’s answers.
2. Timebox the answer. Commit to a decision within five business days for light/standard tiers. Slow reviews recreate shadow IT, which is the failure mode the whole system exists to prevent, the same logic as the acceptable-use policy’s fast exception path.
3. Write down conditions, not just verdicts. Approvals are scoped: which tier, which data classes, which integrations. The scope goes in the approved-tools list.
4. Re-review on triggers, not just calendars. New integration requested, vendor acquired, terms changed, incident reported, or a feature adds write access, each reopens the review. Otherwise, the standing six-month terms recheck covers drift.

FAQ

What should a security review of an AI tool cover?

Five parts: vendor security posture (attestations, encryption, subprocessors, breach handling), data handling (training use, retention, residency, DPA), access and authentication (SSO, MFA, roles, offboarding, logs), integration scope (read/write access and persistence), and AI-specific risks (prompt injection surface, output defaults, autonomy limits).

How long should an AI tool security review take?

Two to four hours of desk review for a standalone tool on non-sensitive data; add a scoped pilot for tools reading company systems; full review plus staged rollout for anything with write access, autonomy, or regulated data. The five-minute pre-screen (business tier exists, DPA available, findable data terms, identifiable vendor) filters out most bad candidates before real time is spent.

What is prompt injection and why does it matter in a tool review?

An attack where instructions hidden in content the AI reads, an email, a document, a web page, override the user’s intent and hijack the tool’s behavior. Risk concentrates in tools that both read untrusted content and can act on your systems. The review question: what can this tool be talked into doing by content, and what confirmations stand in the way?

Do we need this review for AI features inside software we already use?

A lighter pass, yes. New AI features often route existing data to new AI subprocessors under terms that differ from the base product’s. Check where the AI processing happens, whether inputs train models, and whether admins can scope or disable the feature, then record the feature on your approved-tools list like any other tool.

Who should run the security review at a company without a security team?

Your AI governance owner, usually IT or ops, working the standard checklist, escalating to external expertise when a tool will handle regulated data or hold deep system access. Consistency beats credentials here: the checklist run every time catches more than an expert consulted occasionally.

Rolling this out and want a hand? Webisoft helps companies implement AI safely, get in touch.