Security and Privacy for AI Systems
Security and privacy for AI systems on AWS: monitoring AI workloads, the shared responsibility model and secure data engineering best practices for AIF-C01.
DashboardSecurity and Privacy for AI Systems
Monitoring AI systems
- Performance Metrics
- Monitoring model behavior ensures accuracy, reliability, and user trust:
- Accuracy: ratio of correct predictions
- Precision: ratio of correct positive predictions among all positive prediction
- Recall: ratio of actual positives correctly identified
- F1 Score: average of precision and recall (good balanced measure)
- Latency: time taken by the model to make a prediction
- Infrastructure monitoring (catch bottlenecks and failures)
- Detect bottlenecks and failures by monitoring:
- Compute resources (CPU and GPU usage)
- Network performance
- Storage
- System Logs
- Bias and Fairness, Compliance and Responsible AI
AWS Shared Responsibility Model
- AWS Responsibility – Security of the Cloud
- AWS is responsible for protecting the infrastructure that supports its services, including:
- Physical data centers, networking, hardware, and software
- Security of managed services such as Amazon Bedrock, SageMaker, and S3
- AWS is responsible for protecting the infrastructure that supports its services, including:
- Customer responsibility - Security in the Cloud
- Customers are responsible for securing what they build and deploy on AWS:
- For Bedrock, customer is responsible for data management, access controls, setting up guardrails, etc…
- Encrypting application and training data
- Shared Responsibilities:
- Some controls are shared between AWS and customers
- Patch management
- Configuration management
- Security awareness and training
- Awareness and Training
Source: https://aws.amazon.com/compliance/shared-responsibility-model/

Secure Data Engineering – Best Practices
- Data Quality Assessment
- High-quality data is essential for secure and reliable AI systems:
- Completeness: diverse and comprehensive range of scenarios
- Accuracy: accurate, up-to-date, and representative
- Timeliness: age of the data in a data store
- Consistency: maintain coherence and consistency in the data lifecycle
- Data profiling and monitoring
- Data lineage
- Privacy-Enhancing technologies
- Reduce privacy risks using:
- Data masking and obfuscation
- Encryption, tokenization to protect data during processing and usage
- Reduce privacy risks using:
- Data Access Control
- Comprehensive data governance framework with clear policies
- Role-based access control and fine-grained permissions to restrict access
- Single sign-on, multi-factor authentication, identity and access management solutions
- Monitor and log all data access activities
- Regularly review and update access rights based on least privilege principles
- Data Integrity
- Ensure data remains accurate and trustworthy
- Maintain completeness and consistency
- Implement backup and disaster recovery strategies
- Maintain data lineage and audit trails
- Monitor and test the data integrity controls to ensure effectiveness