Security & Compliance Checklist (Expanded)

Professional services firms handle privileged client data daily. Add AI systems to the mix and your attack surface expands dramatically. This checklist gives you the specific controls, configurations, and processes to lock down AI deployments without slowing your practice to a crawl.

Print this. Work through it with your IT director and compliance officer. Check every box before you put an AI tool into production.

Data Collection & Storage

• [ ] Encrypt all data at rest using AES-256. Configure your cloud storage (AWS S3, Azure Blob, Google Cloud Storage) with server-side encryption enabled by default. No exceptions.
• [ ] Encrypt all data in transit using TLS 1.3. Disable TLS 1.2 and earlier. Configure your load balancers and API gateways to reject non-encrypted connections.
• [ ] Store encryption keys in a dedicated key management service. Use AWS KMS, Azure Key Vault, or Google Cloud KMS. Never hardcode keys in application code or configuration files.
• [ ] Implement role-based access to data repositories. Create separate IAM roles for developers (read-only on production), data scientists (read/write on training datasets), and administrators (full access). Document who has what access in a spreadsheet updated monthly.
• [ ] Enable CloudTrail (AWS), Activity Log (Azure), or Cloud Audit Logs (Google Cloud). Set up alerts for: access from new IP addresses, bulk data downloads over 10GB, permission changes, and failed authentication attempts over 5 in 10 minutes.
• [ ] Run automated backups every 24 hours. Store backups in a separate region. Test restoration quarterly by spinning up a complete environment from backup and running a smoke test.
• [ ] Define retention periods by data type. Client work product: 7 years minimum. Training data: 3 years. System logs: 1 year. Implement automated deletion using lifecycle policies in your storage service.

Data Privacy

• [ ] Map all personal data flows. Create a spreadsheet listing: data type, source system, destination system, processing purpose, legal basis (consent, contract, legitimate interest), and retention period. Update this quarterly.
• [ ] Implement a data subject request workflow. Build a form (Google Forms, Typeform, or custom) where individuals can request access, correction, or deletion. Route requests to your compliance officer. Respond within 30 days (GDPR) or 45 days (CCPA).
• [ ] Run a Data Protection Impact Assessment (DPIA) before deploying any AI system that processes personal data. Use the ICO DPIA template or equivalent. Document: what data you're processing, why, what risks exist, and what mitigations you've implemented.
• [ ] Obtain explicit consent for AI processing where required. Add a checkbox to your engagement letters: "I consent to [Firm Name] using AI tools to analyze the information I provide." Store consent records with timestamps.
• [ ] Anonymize or pseudonymize training data. Replace names with ID numbers. Remove email addresses, phone numbers, and street addresses. Use tools like Microsoft Presidio or AWS Comprehend to detect and redact PII automatically.

Data Quality & Integrity

• [ ] Validate all input data against a schema. Define required fields, data types, and acceptable ranges. Reject records that don't match. Log validation failures for review.
• [ ] Run data quality checks before training. Check for: duplicate records, missing values over 5%, outliers beyond 3 standard deviations, and inconsistent formatting (dates, currency). Fix or exclude bad data.
• [ ] Version all training datasets. Use DVC (Data Version Control) or similar. Tag each version with: date created, source systems, number of records, and any transformations applied.
• [ ] Implement append-only logging for data changes. Use database triggers or event sourcing to record: who changed what, when, and why. Store change logs in immutable storage (AWS S3 with Object Lock).
• [ ] Audit data quality monthly. Run automated checks for: schema drift, unexpected null values, distribution shifts, and duplicate records. Assign someone to review the report and take action.

User Authentication

• [ ] Require MFA for all users. Use authenticator apps (Authy, Google Authenticator) or hardware tokens (YubiKey). Disable SMS-based MFA due to SIM-swapping attacks.
• [ ] Implement SSO with your identity provider. Connect AI tools to Okta, Azure AD, or Google Workspace. Disable local accounts except for emergency break-glass access.
• [ ] Create role-based access groups. Define: Viewer (read-only), Contributor (read/write on assigned projects), Admin (full access). Map users to groups based on job function, not individual requests.
• [ ] Set session timeouts to 8 hours for standard users, 1 hour for admins. Force re-authentication after timeout. Log users out after 15 minutes of inactivity.
• [ ] Review user access quarterly. Export a list of all users and their permissions. Send to department heads for confirmation. Revoke access for anyone who's left the firm or changed roles.
• [ ] Monitor for suspicious authentication patterns. Alert on: logins from new countries, multiple failed attempts, concurrent sessions from different IPs, and access outside business hours (unless pre-approved).

Infrastructure Security

• [ ] Segment your network into zones. Create separate VLANs or VPCs for: production AI systems, development/testing, data storage, and corporate network. Restrict traffic between zones using firewall rules.
• [ ] Run a vulnerability scan weekly. Use Nessus, Qualys, or your cloud provider's scanner. Patch critical vulnerabilities within 7 days, high within 30 days.
• [ ] Enable automatic security updates for operating systems. Configure unattended-upgrades (Linux) or Windows Update to install patches during maintenance windows.
• [ ] Implement a Web Application Firewall (WAF). Use AWS WAF, Cloudflare, or similar. Enable OWASP Top 10 rule sets. Block traffic from known malicious IPs.
• [ ] Require VPN or zero-trust network access for remote connections. Use WireGuard, Tailscale, or your cloud provider's VPN. Disable direct SSH/RDP access from the internet.
• [ ] Harden all servers using CIS Benchmarks. Disable unnecessary services, remove default accounts, configure host-based firewalls, and enable audit logging.

Third-Party Access

• [ ] Maintain a vendor register. List all third parties with access to your systems or data. Include: vendor name, service provided, data accessed, contract end date, and last security review date.
• [ ] Require SOC 2 Type II reports from all vendors processing client data. Review the report annually. Check for qualified opinions or control failures. Ask for remediation plans.
• [ ] Create dedicated service accounts for vendor access. Never share employee credentials. Assign minimum necessary permissions. Set expiration dates on vendor accounts.
• [ ] Implement just-in-time access for vendor support. Require vendors to request access via a ticketing system. Approve access for specific time windows (4 hours, 1 day). Revoke automatically when the window expires.
• [ ] Log all vendor activity. Enable session recording for SSH/RDP access. Log all API

  API

  Click to read the full definition in our AI & Automation Glossary.

  calls. Review vendor activity logs monthly for unusual patterns.
• [ ] Include security requirements in vendor contracts. Require: encryption in transit and at rest, MFA for all access, notification of breaches within 24 hours, and right to audit. Make security failures grounds for termination.

Model Provenance

• [ ] Document the source of every training dataset. Record: where the data came from, who provided it, when it was collected, and what permissions you have to use it.
• [ ] Track all model versions in a model registry. Use MLflow, Weights & Biases, or your cloud provider's model registry. Tag each version with: training date, dataset version, hyperparameters, and performance metrics.
• [ ] Maintain a bill of materials for each model. List all: base models (GPT-4, Claude, Llama), fine-tuning datasets, libraries (transformers, scikit-learn), and dependencies. Update when anything changes.
• [ ] Implement model signing. Generate a cryptographic hash of each model file. Store the hash in your model registry. Verify the hash before deployment to detect tampering.
• [ ] Review model provenance before each deployment. Check: Is the training data still valid? Have any dependencies been flagged for security issues? Has the model been tested on current data?

Model Fairness & Bias

• [ ] Test for bias across protected characteristics. Run your model on test sets segmented by: gender, age, race, and geography. Calculate performance metrics (accuracy, precision, recall) for each segment. Flag disparities over 5%.
• [ ] Use fairness metrics appropriate to your use case. For classification: demographic parity, equalized odds, equal opportunity. For ranking: exposure parity, relevance parity. Document which metrics you're using and why.
• [ ] Implement bias monitoring in production. Log model inputs and outputs with metadata (user demographics if available). Run bias audits monthly. Alert if fairness metrics degrade.
• [ ] Establish a bias remediation process. When bias is detected: pause the model, investigate root causes (training data imbalance, feature correlation), retrain with corrected data, and retest before redeployment.
• [ ] Publish a model card for each production model. Include: intended use, training data characteristics, known limitations, fairness metrics, and contact for questions. Make this available to users.

Model Explainability

• [ ] Generate explanations for high-stakes decisions. Use SHAP, LIME, or built-in explanation features (Azure ML Interpretability, AWS SageMaker Clarify). Show which features contributed most to each prediction.
• [ ] Provide confidence scores with all predictions. Display: "This recommendation has 87% confidence." Set thresholds below which predictions require human review (typically 70-80%).
• [ ] Document model logic in plain language. Write a one-page summary explaining: what the model does, what data it uses, how it makes decisions, and what it cannot do. Share this with users.
• [ ] Implement a model explanation API

  API

  Click to read the full definition in our AI & Automation Glossary.

  . Allow users to query: "Why did the model make this recommendation?" Return the top 5 contributing factors with their weights.
• [ ] Test explanations with actual users. Show explanations to 5-10 users. Ask: "Does this make sense? Would you trust this recommendation?" Iterate based on feedback.

Model Governance

• [ ] Establish a model approval process. Require sign-off from: data science lead (technical quality), compliance officer (regulatory requirements), and business owner (fitness for purpose) before production deployment.
• [ ] Define model risk tiers. Tier 1 (high risk): affects client deliverables, financial decisions, or legal advice. Tier 2 (medium risk): internal efficiency tools. Tier 3 (low risk): experimental or non-critical. Apply stricter controls to higher tiers.
• [ ] Schedule model reviews based on risk tier. Tier 1: quarterly. Tier 2: semi-annually. Tier 3: annually. Review: performance metrics, fairness metrics, user feedback, and incident reports.
• [ ] Maintain a model inventory. Track all models in production. Include: model name, owner, risk tier, deployment date, last review date, and retirement date.
• [ ] Implement model retirement procedures. When retiring a model: notify all users 30 days in advance, migrate users to replacement model, archive model artifacts and documentation, and revoke API

  API

  Click to read the full definition in our AI & Automation Glossary.

  access.

Incident Preparedness

• [ ] Write an AI incident response playbook. Define procedures for: model producing incorrect outputs, data breach involving training data, bias discovered in production, model unavailable, and unauthorized model access. Include contact lists and escalation paths.
• [ ] Assign an AI incident response team. Include: data scientist (technical lead), IT security (containment), compliance officer (regulatory), legal counsel (liability), and communications (client notification).
• [ ] Run tabletop exercises twice per year. Simulate scenarios: "A client reports our AI tool gave discriminatory advice" or "Training data was exposed in a breach." Walk through your playbook. Document gaps and update procedures.
• [ ] Establish incident severity levels. Level 1 (critical): data breach, discriminatory output affecting clients, model completely unavailable. Level 2 (high): degraded performance, bias detected in testing. Level 3 (low): minor errors, isolated incidents.
• [ ] Create incident communication templates. Draft emails for: internal notification, client notification, and regulatory notification. Include placeholders for incident details. Review with legal before an incident occurs.

Incident Detection & Analysis

• [ ] Monitor model performance in real-time. Track: prediction latency, error rates, confidence score distribution, and throughput. Alert when metrics deviate from baseline by more than 20%.
• [ ] Implement anomaly detection on model outputs. Flag: predictions outside expected ranges, sudden shifts in prediction distribution, and repeated identical outputs (possible model failure).
• [ ] Collect user feedback on AI outputs. Add thumbs up/down buttons. Track feedback rates. Investigate when negative feedback exceeds 10% of interactions.
• [ ] Centralize AI system logs. Send logs from all AI components to a SIEM (Splunk, Elastic, Datadog). Retain logs for 1 year minimum.
• [ ] Conduct root cause analysis within 48 hours of incident detection. Document: what happened, when it started, what caused it, what the impact was, and what immediate actions were taken.
• [ ] Classify incidents by type. Categories: data quality issue, model drift, bias/fairness, security breach, availability, or user error. Track incident trends monthly.

Incident Containment & Remediation

• [ ] Implement a model kill switch. Build the ability to instantly disable a model via: API

  API

  Click to read the full definition in our AI & Automation Glossary.

  flag, configuration change, or traffic routing. Test the kill switch quarterly.
• [ ] Maintain rollback capability. Keep the previous 3 model versions deployed but inactive. Document the rollback procedure: update routing, verify functionality, notify users. Practice rollback quarterly.
• [ ] Isolate affected systems immediately. For security incidents: disconnect from network, revoke credentials, and preserve logs. For performance incidents: route traffic to backup model or manual process.
• [ ] Implement circuit breakers. Automatically disable a model if: error rate exceeds 10%, latency exceeds 5 seconds, or confidence scores drop below 60%. Route to fallback process.
• [ ] Fix root causes before redeployment. For data issues: correct and revalidate data. For model issues: retrain and retest. For code issues: patch and review. Never just restart and hope.
• [ ] Require post-fix validation. Test the fix in a staging environment. Run the same inputs that triggered the incident. Verify the issue is resolved. Get approval from incident response team before returning to production.

Incident Recovery & Lessons Learned

• [ ] Document recovery time objectives (RTO) and recovery point objectives (RPO). RTO: maximum acceptable downtime (typically 4 hours for critical models). RPO: maximum acceptable data loss (typically 24 hours). Test your ability to meet these targets.
• [ ] Maintain offline copies of critical model artifacts. Store model files, training data, and deployment scripts in offline storage (external drive, offline S3 bucket). Update monthly.
• [ ] Conduct a post-incident review within 5 business days. Invite all incident response team members. Discuss: what went well, what went poorly, what we learned, and what we'll change.
• [ ] Create action items from every incident. Assign owners and due dates. Track completion. Common actions: update monitoring, improve documentation, add test cases, and revise procedures.
• [ ] Share lessons learned across the organization. Write a one-page summary (sanitized of sensitive details). Present at monthly tech meetings. Update training materials.
• [ ] Update your incident response playbook after every major incident. Add new scenarios, refine procedures, update contact lists, and incorporate lessons learned. Version the playbook and track changes.