The AI Security Checklist: 40 Controls Before You Ship an AI Feature

This checklist is designed to be actionable, not theoretical. Each item is a specific control you can verify, assign, and track. It covers the full lifecycle of an AI feature — from design and data handling through deployment, monitoring, and incident response. Use it as a pre-launch gate for new AI features and as an audit baseline for existing ones.

Section 1: Design & Threat Modeling

• Threat model completed — AI-specific threats (prompt injection, model extraction, data poisoning) have been included in the threat model for this feature
• Trust boundaries defined — clear documentation of what inputs the model receives, from which sources, and which are considered untrusted
• Abuse cases documented — adversarial use cases (malicious prompts, jailbreak attempts, automated abuse) have been considered and mitigations planned
• Data classification done — all data that will be used in prompts, passed to the model, or produced by the model has been classified for sensitivity
• Compliance requirements identified — GDPR, HIPAA, CCPA, or other regulatory requirements applicable to AI data processing have been documented

Section 2: Data Handling & Privacy

• PII not in system prompts — system prompts contain no personal data, credentials, or sensitive business information
• Training data audited — if fine-tuning, the training dataset has been reviewed for sensitive data, PII, and copyright issues
• Data minimization applied — prompts contain only the data necessary for the task, not full records or excessive context
• Data retention policy defined — retention periods for prompts, responses, and logs have been defined and are enforced
• Third-party data sharing reviewed — legal/privacy has reviewed what data is being sent to the LLM provider and approved the data processing agreement
• User consent handled — if user data is processed by AI, appropriate consent and disclosure mechanisms are in place
• Data residency requirements met — if applicable, data is processed only in approved geographic regions

Section 3: Prompt Security

• Prompt injection mitigations in place — user-supplied content is treated as untrusted data, not instructions, and is handled accordingly
• Input validation implemented — inputs are validated for length, character set, and expected format before being passed to the model
• Input length limits enforced — maximum input lengths are defined and enforced to prevent token flooding attacks
• Instruction/data separation — where possible, system instructions and user data are clearly separated and user data cannot override system instructions
• Indirect prompt injection considered — if the model processes external content (documents, web pages, emails), indirect injection via that content has been addressed
• System prompt protected — the system prompt is not exposed to users and cannot be extracted through prompt manipulation

Section 4: Output Handling

• Output treated as untrusted — LLM output is validated and sanitized before being passed to downstream systems or rendered in a browser
• Output encoding applied — model outputs rendered in HTML are properly encoded to prevent XSS
• Downstream action validation — if model output triggers actions (API calls, database writes, code execution), those actions are validated independently of the model's instruction
• Hallucination handling — the application handles factually incorrect model outputs gracefully and does not treat model assertions as ground truth for critical decisions
• Sensitive data in outputs controlled — the model cannot output more sensitive data than the requesting user is authorized to receive

Section 5: Model & Vendor Security

• Model integrity verified — if using a self-hosted or fine-tuned model, checksums have been verified and the model source is trusted
• Vendor security reviewed — the LLM provider's security posture, data handling policies, and incident history have been reviewed
• API key management — LLM API keys are stored in a secrets manager (not hardcoded or in environment variables in source code)
• API key rotation policy — a rotation schedule and process for LLM API keys is defined
• Model version pinned — specific model version is pinned so that provider updates don't change behavior unexpectedly
• Fallback behavior defined — behavior when the LLM API is unavailable is defined and tested

Section 6: Access Control & Agent Permissions

• Least privilege applied to agents — AI agents have only the permissions required for their specific function, not broad access
• Tool/plugin permissions scoped — each tool or plugin available to the model has been reviewed and permissions scoped to minimum necessary
• Human-in-the-loop for high-impact actions — actions with significant consequences (data deletion, financial transactions, external communications) require human approval before execution
• User authorization respected — the model cannot retrieve or act on data that the requesting user is not authorized to access
• Plugin/tool input validation — all inputs to tools and plugins called by the model are validated independently

Section 7: Monitoring & Observability

• Prompt and response logging — all model interactions are logged with appropriate masking of sensitive fields
• Anomaly detection configured — alerts exist for unusual token volumes, unusual output patterns, and repeated similar inputs (probing behavior)
• Cost monitoring in place — alerts configured for unexpected spikes in API usage or cost that could indicate abuse
• AI logs in SIEM — AI interaction logs are integrated into your security monitoring infrastructure
• Audit trail for agentic actions — all actions taken by AI agents are logged with enough context to reconstruct what happened and why

Section 8: Incident Response

• AI-specific IR playbook exists — your incident response plan includes scenarios specific to AI failures (prompt injection attack, model manipulation, data extraction via AI)
• Kill switch implemented — there is a documented, tested way to disable AI features immediately if a security issue is discovered
• Rollback plan for model updates — if a model update introduces security issues, there is a process to roll back to the previous version
• Responsible disclosure process — if researchers discover vulnerabilities in your AI implementation, there is a clear process for them to report it

---

How to Use This Checklist

Not every item on this list will apply to every AI feature — a simple text summarization tool has a different risk profile than an AI agent with access to your CRM and email. Start by identifying which sections are most relevant to your specific use case, then work through each applicable item systematically.

Treat this as a living document. As your AI implementation evolves and as new attack techniques emerge, revisit and update your controls. The AI threat landscape in 2026 is moving faster than almost any other area of security.