The alert fired at 3 PM on a Wednesday. Our Java microservices detected unusual access patterns from a service account that had been dormant for six months. The credentials were valid. The IAM policy allowed the actions. Everything looked legitimate according to AWS CloudTrail. Yet something felt wrong. The access pattern did not match historical behavior for that identity.
We faced an insider threat in progress. The attacker had compromised a service account with proper permissions. Traditional IAM controls could not stop them because the credentials were not stolen. They were misused by someone who already had access. Our security team spent the next 72 hours tracing how this happened. We realized that static IAM policies were not enough. We needed behavioral analysis. We needed machine learning to understand what normal access looked like.
In this article, I will share how we built machine learning-driven access control for our Java applications on AWS. I will explain the architecture we designed. I will detail the ML models we trained. I will provide code examples showing how to implement behavioral IAM analysis. This is not theoretical research. This is a practical account of defending against credential misuse in production.
The Limitation of Static IAM Policies
AWS IAM policies define what actions an identity can perform. They do not define when or how those actions should occur. A service account with S3 read permissions can access any bucket at any time. This creates a security gap. Compromised credentials with valid permissions bypass all controls.
Consider a Java application that accesses DynamoDB. The IAM role grants full table access. An attacker who compromises that role can read or delete any item. Traditional controls cannot distinguish between legitimate access and malicious activity. The API calls look identical. The credentials are valid. The policy allows the action.
We learned this when a developer account was compromised. The attacker accessed customer data using valid credentials. CloudTrail logged every action. Nothing triggered alerts because everything was permitted. We needed a different approach. We needed to analyze behavior, not just permissions.
Building Behavioral Baselines for IAM
The key to detecting credential misuse is understanding normal access patterns. If you know what normal looks like, you can spot anomalies. Machine learning excels at this task. It can analyze millions of API calls and learn patterns that humans would miss.
We started by instrumenting our Java applications to emit detailed access telemetry. Every AWS SDK call generates logs containing the identity, action, resource, time, and source IP. We streamed this data to Amazon Kinesis for real-time processing.
This code emits telemetry for every AWS SDK call. The data flows to Kinesis, where Lambda functions process it in real time. We built baselines from this data over a two-week learning period. The system learned normal access frequencies, typical resource patterns, and usual time-of-day behavior for each identity.
Anomaly Detection with AWS Services
We evaluated several approaches for anomaly detection. Statistical methods worked for simple cases. They failed for complex patterns. We needed something more sophisticated. We chose AWS GuardDuty and custom models trained with SageMaker.
The model analyzed multiple features simultaneously. Access frequency per identity. Resource access patterns. Time-of-day consistency. Geographic location. API call sequences. When the model detected a deviation from baseline, it generated a risk score.
This Lambda function runs synchronously for high-risk operations. It adds minimal latency, typically under 50 milliseconds. The model scores each access request in real time. High scores trigger immediate alerts.
Real-Time Access Control with Java
Detection alone is not enough. You must respond quickly. We built an automated response system using Java Spring Security and AWS Lambda. When the risk score exceeded our threshold, the system took action.
import org.springframework.security.access.AccessDeniedException;
import org.springframework.stereotype.Component;
@Component
public class AdaptiveAccessManager {
private final AmazonLambdaAsync lambdaClient;
private final AmazonDynamoDB dynamoDB;
private final String riskAssessmentFunction;
private final String blockedIdentitiesTable;
public AdaptiveAccessManager() {
this.lambdaClient = AmazonLambdaAsyncClientBuilder.defaultClient();
this.dynamoDB = AmazonDynamoDBClientBuilder.defaultClient();
this.riskAssessmentFunction = System.getenv("RISK_ASSESSMENT_FUNCTION");
this.blockedIdentitiesTable = System.getenv("BLOCKED_IDENTITIES_TABLE");
}
public void checkAccess(Authentication authentication, String resource) {
RiskAssessmentRequest request = new RiskAssessmentRequest(
authentication.getName(),
resource,
getClientIp()
);
RiskAssessmentResponse response = assessRisk(request);
if (response.getRiskScore() > 0.95) {
blockIdentity(authentication.getName(), response.getReason());
throw new AccessDeniedException("High risk access detected");
} else if (response.getRiskScore() > 0.85) {
applyStepUpAuthentication(authentication);
logForInvestigation(response);
}
}
private RiskAssessmentResponse assessRisk(RiskAssessmentRequest request) {
InvokeRequest invokeRequest = new InvokeRequest()
.withFunctionName(riskAssessmentFunction)
.withPayload(toJson(request));
InvokeResult result = lambdaClient.invoke(invokeRequest).join();
return fromJson(result.getPayload(), RiskAssessmentResponse.class);
}
}
This manager integrates with Spring Security through custom access decision voters. Every protected resource access passes through the risk assessment system. High-risk identities receive immediate denial. Medium-risk identities face step-up authentication. The system adapts based on confidence levels.
Feature Engineering for IAM Anomalies
The quality of your features determines the quality of detection. We engineered features specifically for identifying credential misuse in AWS environments.
- Access Pattern Entropy: Normal identities access predictable resources. Attackers explore broadly. We calculated the entropy of resource access patterns. Legitimate users show low entropy. Compromised credentials show high entropy.
- Temporal Consistency: Humans and services have natural rhythms. They operate during expected hours. Attackers do not follow these patterns. We analyzed access timing consistency.
- Action Rarity Scoring: Some API actions are rarely used by normal operations. Delete operations, permission changes, and cross-account access fall into this category. We tracked action frequency per identity.
This feature extraction runs before the request reaches AWS services. It adds minimal overhead while providing valuable signals.
Training the Access Control Model
We trained our model using a combination of normal access logs and simulated attacks. The normal data came from two weeks of production CloudTrail logs. The attack data came from security researchers who performed red team exercises.
We used SageMaker's built-in algorithms initially. Random Cut Forest worked well for unsupervised anomaly detection. It identified outliers without requiring labeled attack data. We later supplemented this with supervised models trained on known attack patterns.
The training pipeline ran weekly. It ingested new telemetry data. It retrained models with fresh patterns. It deployed updated models to the endpoint with zero downtime. This ensured the system adapted to changing behavior.
The model achieved 92 percent accuracy on known attacks. More importantly, it detected 81 percent of credential misuse attempts in our test suite. This was far better than static IAM policies, which detected zero percent.
Integration with AWS Identity Services
Our architecture is integrated with existing AWS identity services. We used Cognito for user authentication. We used IAM roles for service identities. We used STS for temporary credentials.
The flow worked like this. A request arrived at our Java application. The application extracted identity context. It called the risk assessment Lambda. If the risk score was acceptable the request proceeded. If not it was blocked or challenged. The entire process added 50 to 100 milliseconds of latency.
This infrastructure is deployed automatically through CI/CD. We tested changes in staging before production. Rollback took minutes if issues arose.
Lessons Learned
Building this system taught us valuable lessons about ML-driven access control.
- False Positives Matter: Aggressive detection blocks legitimate users. We started with a threshold of 0.75. It blocked too many real users. We tuned it to 0.85 for step-up authentication and 0.95 for blocking. This reduced false positives to under 1 percent.
- Context Is Critical: An access that looks suspicious in isolation might be normal in context. A developer accessing production databases is suspicious unless they are on-call. We incorporated on-call schedules into our features. This improved accuracy significantly.
- Latency Trade-offs: Real-time assessment adds latency. We optimized aggressively. We cached identity baselines in DynamoDB. We used provisioned concurrency for Lambda functions. We chose lightweight models. We achieved sub-100ms assessment without sacrificing accuracy.
- Continuous Adaptation: Attackers adapt. Your system must too. We retrained models weekly. We reviewed false negatives monthly. We updated features quarterly. This kept the system effective against evolving threats.
- Compliance Considerations: Automated access decisions affect audit trails. We logged every risk assessment. We stored model versions with each decision. This enabled forensic analysis during investigations. Regulators appreciated the transparency.
Conclusion
Static IAM policies cannot stop credential misuse. Behavioral analysis powered by machine learning can. We built a system that detects anomalous access in real time using AWS Lambda and SageMaker. It integrates with Java applications through Spring Security. It adds minimal latency while providing significant protection.
The key insight is that you do not need to know what an attack looks like. You need to know what normal looks like. Anything that deviates significantly deserves scrutiny. Machine learning excels at learning normal patterns. It spots deviations that humans would miss.
If you are running Java applications on AWS consider implementing behavioral access control. Start with telemetry collection. Build baselines. Train models. Deploy gradually. Monitor false positives closely. Tune thresholds carefully. The effort is worth it. Credential misuse is inevitable. Your defense should be too.