How I Automated AWS WorkMail Backups for Multiple Users

AWS WorkMail is a powerful email and calendar service that combines security, reliability, and ease of use. Its integration with AWS services and compatibility with existing email clients make it an attractive choice for businesses looking to streamline their email and calendar management while ensuring security and compliance. By leveraging AWS WorkMail, organizations can focus on their core activities without worrying about the complexities of managing email infrastructure. Setting up a reliable backup system for your AWS WorkMail is crucial to ensure that your emails are secure and easily recoverable. In this tutorial, I'll walk you through the process of automating AWS WorkMail backups for multiple users, handling up to 10 concurrent mailbox export jobs. We will use AWS services and Python to create a robust backup solution. Ah! I remember the time I was using the CPANEL mail server and then meddling with my own mail server. Those single-instance mail servers with limited access to IP addresses in the early 2010s were smooth, but as more and more people worldwide started exploiting these servers, Comon, we still get a lot of spam and marketing. Things were tough making sure your email did not land in the spam box. The choice between AWS WorkMail, a self-hosted mail server, and a cPanel mail server depends on several factors, including budget, technical expertise, scalability needs, and security requirements. AWS WorkMail is ideal for businesses seeking a scalable, secure, and managed email solution with minimal maintenance overhead. Self-Hosted Mail Server is suitable for organizations with in-house technical expertise and specific customization needs but involves higher costs and maintenance efforts. cPanel Mail Server offers a balanced approach for small to medium-sized businesses that need a user-friendly interface and are comfortable with some level of maintenance, benefiting from the bundled hosting services. My main concerns were always the following; Security AWS WorkMail: Provides built-in security features, including encryption in transit and at rest, integration with AWS Key Management Service (KMS), and compliance with various regulatory standards. Self-Hosted Mail Server: Security depends entirely on the expertise of the administrator. Requires configuration for encryption, spam filtering, and regular security updates to protect against threats. cPanel Mail Server: Offers security features, but the responsibility for implementing and maintaining security measures falls on the user. cPanel provides tools for SSL/TLS configuration, spam filtering, and antivirus, but proper setup and regular updates are essential. Then came AWS to the rescue, and since using AWS from 2015, things have been a breeze, except when there came a case of backing up my AWS Account to move to a new account. Well, I have been searching over the internet to find a reliable solution because, as far as we speak as of now, AWS WorkMail doesn’t provide a straightforward way of backing up your email to your local computer or to S3, and it is understandable based on the security compliance, but still, I am still expecting that AWS should provide some GUI or a tool for that. Well while browsing, I did come across tools definitely PAID, so I decided to take on the path of developing my own tool of it. After rigorous testing, based on my experience with AWS, 9/10 problems always occur when one service is unable to communicate with the other service due to policy or IAM role issues. This is my contribution to all those people who are having a hard time taking backups of AWS Work Mail Accounts. How does the backup work? We created an IAM Role in your account, giving the role access to export from AWS WorkMail, and then we attached a policy to the same role to allow it to access S3. We then create a KMS Key, which is given access to the IAM Role. The S3 bucket also needs access to the IAM role to function properly. Step 1: Configure AWS CLI First, ensure that the AWS CLI is installed and configured with appropriate credentials. Open your terminal and run: aws configure Follow the prompts to set your AWS Access Key ID, Secret Access Key, default region, and output format. Step 2: Set Up Required AWS Resources We need to create an IAM role, policies, an S3 bucket, and a KMS key. Save the following bash script as setup_workmail_export.sh. #!/bin/bash # Configuration ROLE_NAME="WorkMailExportRole" POLICY_NAME="workmail-export" S3_BUCKET_NAME="s3.bucket.name" AWS_REGION="your-region" ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text) # Create Trust Policy cat trust-policy.json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": "export.workmail.amazonaws.com" }, "Action": "sts:AssumeRole", "Condition": { "StringEquals": { "sts:ExternalId": "$ACCOUNT_ID" } } } ] } EOF # Create IAM Role aws iam create-role --role-name $ROLE_NAME --assume-role-policy-document file://trust-policy.json # Create IAM Policy cat role-policy.json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "s3:*", "Resource": [ "arn:aws:s3:::$S3_BUCKET_NAME", "arn:aws:s3:::$S3_BUCKET_NAME/*" ] }, { "Effect": "Allow", "Action": [ "kms:Decrypt", "kms:GenerateDataKey" ], "Resource": "*" } ] } EOF # Attach the Policy to the Role aws iam put-role-policy --role-name $ROLE_NAME --policy-name $POLICY_NAME --policy-document file://role-policy.json # Create S3 Bucket aws s3api create-bucket --bucket $S3_BUCKET_NAME --region $AWS_REGION --create-bucket-configuration LocationConstraint=$AWS_REGION # Create Key Policy cat key-policy.json { "Version": "2012-10-17", "Id": "workmail-export-key", "Statement": [ { "Sid": "Enable IAM User Permissions", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::$ACCOUNT_ID:root" }, "Action": "kms:*", "Resource": "*" }, { "Sid": "Allow administration of the key", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::$ACCOUNT_ID:role/$ROLE_NAME" }, "Action": [ "kms:Create*", "kms:Describe*", "kms:Enable*", "kms:List*", "kms:Put*", "kms:Update*", "kms:Revoke*", "kms:Disable*", "kms:Get*", "kms:Delete*", "kms:ScheduleKeyDeletion", "kms:CancelKeyDeletion" ], "Resource": "*" } ] } EOF # Create the KMS Key and get the Key ID and ARN using Python for JSON parsing KEY_METADATA=$(aws kms create-key --policy file://key-policy.json) KEY_ID=$(python3 -c "import sys, json; print(json.load(sys.stdin)['KeyMetadata']['KeyId'])" s3_bucket_policy.json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::$ACCOUNT_ID:role/$ROLE_NAME" }, "Action": "s3:*" "Resource": [ "arn:aws:s3:::$S3_BUCKET_NAME", "arn:aws:s3:::$S3_BUCKET_NAME/*" ] } ] } EOF # Apply the Bucket Policy aws s3api put-bucket-policy --bucket $S3_BUCKET_NAME --policy file://s3_bucket_policy.json # Clean up temporary files rm trust-policy.json role-policy.json key-policy.json s3_bucket_policy.json # Display the variables required for the backup script cat <<EOF Setup complete. Here are the variables required for the backup script: # Print out the Variables organization_id = 'your-organization-id' user_id = 'your-user-id' s3_bucket_name = '$S3_BUCKET_NAME' s3_prefix = 'workmail-backup/' region = '$AWS_REGION' kms_key_arn = '$KEY_ARN' role_name = '$ROLE_NAME' account_id = '$ACCOUNT_ID' EOF Make the script executable and run it: chmod +x setup_workmail_export.sh ./setup_workmail_export.sh Step 3: Write the Backup Script Now, let's write the Python script to export mailboxes in batches of 10. Save the following script as workmail_export.py. import boto3 import json import time from datetime import datetime # Configuration organization_id = 'your-organization-id' user_emails = { 'user-id-1': 'user1@example.com', 'user-id-2': 'user2@example.com', 'user-id-3': 'user3@example.com', 'user-id-4': 'user4@example.com', 'user-id-5': 'user5@example.com', 'user-id-6': 'user6@example.com', 'user-id-7': 'user7@example.com', 'user-id-8': 'user8@example.com', 'user-id-9': 'user9@example.com', 'user-id-10': 'user10@example.com', 'user-id-11': 'user11@example.com', 'user-id-12': 'user12@example.com' # Add more user ID to email mappings as needed } s3_bucket_name = 's3.bucket.name' region = 'your-region' kms_key_arn = 'arn:aws:kms:your-region:your-account-id:key/your-key-id' role_name = 'WorkMailExportRole' account_id = 'your-account-id' # Get the current date current_date = datetime.now().strftime('%Y-%m-%d') # Set the S3 prefix with the date included s3_prefix_base = f'workmail-backup/{current_date}/' # Initialize AWS clients workmail = boto3.client('workmail', region_name=region) sts = boto3.client('sts', region_name=region) def start_export_job(entity_id, user_email): client_token = str(time.time()) # Unique client token role_arn = f"arn:aws:iam::{account_id}:role/{role_name}" s3_prefix = f"{s3_prefix_base}{user_email}/" try: response = workmail.start_mailbox_export_job( ClientToken=client_token, OrganizationId=organization_id, EntityId=entity_id, Description='Backup job', RoleArn=role_arn, KmsKeyArn=kms_key_arn, S3BucketName=s3_bucket_name, S3Prefix=s3_prefix ) return response['JobId'] except Exception as e: print(f"Failed to start export job for {entity_id}: {e}") return None def check_job_status(job_id): while True: try: response = workmail.describe_mailbox_export_job( OrganizationId=organization_id, JobId=job_id ) print(f"Full Response: {response}") # Log full response for debugging state = response.get('State', 'UNKNOWN') print(f"Job State: {state}") if state in ['COMPLETED', 'FAILED']: break except Exception as e : print(f"Error checking job status for {job_id}: {e}") time.sleep(30) # Wait for 30 seconds before checking again return state def export_mailboxes_in_batches(user_emails, batch_size=10): user_ids = list(user_emails.keys()) for i in range(0, len(user_ids), batch_size): batch = user_ids[i:i+batch_size] job_ids = [] for user_id in batch: user_email = user_emails[user_id] job_id = start_export_job(user_id, user_email) if job_id: print(f"Started export job for {user_email} with Job ID: {job_id}") job_ids.append((user_email, job_id)) for user_email, job_id in job_ids: state = check_job_status(job_id) if state == 'COMPLETED': print(f"Export job for {user_email} completed successfully") else: print(f"Export job for {user_email} failed with state: {state}") def main(): export_mailboxes_in_batches(user_emails) if __name__ == "__main__": main() Replace the placeholders with your actual AWS WorkMail organization ID, user ID to email mappings, S3 bucket name, region, KMS key ARN, role name, and AWS account ID. Step 4: Run the Backup Script Ensure you have Boto3 installed: pip install boto3 Then, execute the Python script: python workmail_export.py Import Mailbox import boto3 import json import time # Configuration organization_id = 'your-organization-id' user_import_data = { 'user-id-1': 'user1@example.com', 'user-id-2': 'user2@example.com', 'user-id-3': 'user3@example.com', 'user-id-4': 'user4@example.com', 'user-id-5': 'user5@example.com', 'user-id-6': 'user6@example.com', 'user-id-7': 'user7@example.com', 'user-id-8': 'user8@example.com', 'user-id-9': 'user9@example.com', 'user-id-10': 'user10@example.com', 'user-id-11': 'user11@example.com', 'user-id-12': 'user12@example.com' # Add more user ID to email mappings as needed } s3_bucket_name = 's3.bucket.name' s3_object_prefix = 'workmail-backup/' # Prefix for S3 objects (folders) region = 'your-region' role_name = 'WorkMailImportRole' account_id = 'your-account-id' # Initialize AWS clients workmail = boto3.client('workmail', region_name=region) sts = boto3.client('sts', region_name=region) def start_import_job(entity_id, user_email): client_token = str(time.time()) # Unique client token role_arn = f"arn:aws:iam::{account_id}:role/{role_name}" s3_object_key = f"{s3_object_prefix}{user_email}/export.zip" try: response = workmail.start_mailbox_import_job( ClientToken=client_token, OrganizationId=organization_id, EntityId=entity_id, Description='Import job', RoleArn=role_arn, S3BucketName=s3_bucket_name, S3ObjectKey=s3_object_key ) return response['JobId'] except Exception as e: print(f"Failed to start import job for {entity_id}: {e}") return None def check_job_status(job_id): while True: try: response = workmail.describe_mailbox_import_job( OrganizationId=organization_id, JobId=job_id ) state = response.get('State', 'UNKNOWN') print(f"Job State: {state}") if state in ['COMPLETED', 'FAILED']: break except Exception as e: print(f"Error checking job status for {job_id}: {e}") time.sleep(30) # Wait for 30 seconds before checking again return state def import_mailboxes_in_batches(user_import_data, batch_size=10): user_ids = list(user_import_data.keys()) for i in range(0, len(user_ids), batch_size): batch = user_ids[i:i+batch_size] job_ids = [] for user_id in batch: user_email = user_import_data[user_id] job_id = start_import_job(user_id, user_email) if job_id: print(f"Started import job for {user_email} with Job ID: {job_id}") job_ids.append((user_email, job_id)) for user_email, job_id in job_ids: state = check_job_status(job_id) if state == 'COMPLETED': print(f"Import job for {user_email} completed successfully") else: print(f"Import job for {user_email} failed with state: {state}") def main(): import_mailboxes_in_batches(user_import_data) if __name__ == "__main__": main() By following these steps, you have set up an automated system to back up your AWS WorkMail mailboxes, handling up to 10 concurrent export jobs. This solution ensures that your emails are securely stored in an S3 bucket, organized by user email and date, and encrypted with a KMS key. This setup provides a robust and scalable backup strategy for your organization's email data. Github Repo: https://github.com/th3n00bc0d3r/AWS-WorkMail-Backup Feel free to get in touch on my LinkedIn, Also you can always buy me a coffee https://github.com/th3n00bc0d3r/AWS-WorkMail-Backup?embedable=true AWS WorkMail is a powerful email and calendar service that combines security, reliability, and ease of use. Its integration with AWS services and compatibility with existing email clients make it an attractive choice for businesses looking to streamline their email and calendar management while ensuring security and compliance. By leveraging AWS WorkMail, organizations can focus on their core activities without worrying about the complexities of managing email infrastructure. Setting up a reliable backup system for your AWS WorkMail is crucial to ensure that your emails are secure and easily recoverable. In this tutorial, I'll walk you through the process of automating AWS WorkMail backups for multiple users, handling up to 10 concurrent mailbox export jobs. We will use AWS services and Python to create a robust backup solution. Ah! I remember the time I was using the CPANEL mail server and then meddling with my own mail server. Those single-instance mail servers with limited access to IP addresses in the early 2010s were smooth, but as more and more people worldwide started exploiting these servers, Comon, we still get a lot of spam and marketing. Things were tough making sure your email did not land in the spam box. The choice between AWS WorkMail, a self-hosted mail server, and a cPanel mail server depends on several factors, including budget, technical expertise, scalability needs, and security requirements. AWS WorkMail is ideal for businesses seeking a scalable, secure, and managed email solution with minimal maintenance overhead. Self-Hosted Mail Server is suitable for organizations with in-house technical expertise and specific customization needs but involves higher costs and maintenance efforts. cPanel Mail Server offers a balanced approach for small to medium-sized businesses that need a user-friendly interface and are comfortable with some level of maintenance, benefiting from the bundled hosting services. AWS WorkMail is ideal for businesses seeking a scalable, secure, and managed email solution with minimal maintenance overhead. AWS WorkMail Self-Hosted Mail Server is suitable for organizations with in-house technical expertise and specific customization needs but involves higher costs and maintenance efforts. Self-Hosted Mail Server cPanel Mail Server offers a balanced approach for small to medium-sized businesses that need a user-friendly interface and are comfortable with some level of maintenance, benefiting from the bundled hosting services. cPanel Mail Server My main concerns were always the following; Security Security AWS WorkMail: Provides built-in security features, including encryption in transit and at rest, integration with AWS Key Management Service (KMS), and compliance with various regulatory standards. Self-Hosted Mail Server: Security depends entirely on the expertise of the administrator. Requires configuration for encryption, spam filtering, and regular security updates to protect against threats. cPanel Mail Server: Offers security features, but the responsibility for implementing and maintaining security measures falls on the user. cPanel provides tools for SSL/TLS configuration, spam filtering, and antivirus, but proper setup and regular updates are essential. AWS WorkMail : Provides built-in security features, including encryption in transit and at rest, integration with AWS Key Management Service (KMS), and compliance with various regulatory standards. AWS WorkMail Self-Hosted Mail Server : Security depends entirely on the expertise of the administrator. Requires configuration for encryption, spam filtering, and regular security updates to protect against threats. Self-Hosted Mail Server cPanel Mail Server : Offers security features, but the responsibility for implementing and maintaining security measures falls on the user. cPanel provides tools for SSL/TLS configuration, spam filtering, and antivirus, but proper setup and regular updates are essential. cPanel Mail Server Then came AWS to the rescue, and since using AWS from 2015, things have been a breeze, except when there came a case of backing up my AWS Account to move to a new account. Well, I have been searching over the internet to find a reliable solution because, as far as we speak as of now, AWS WorkMail doesn’t provide a straightforward way of backing up your email to your local computer or to S3, and it is understandable based on the security compliance, but still, I am still expecting that AWS should provide some GUI or a tool for that. Well while browsing, I did come across tools definitely PAID, so I decided to take on the path of developing my own tool of it. After rigorous testing, based on my experience with AWS, 9/10 problems always occur when one service is unable to communicate with the other service due to policy or IAM role issues. This is my contribution to all those people who are having a hard time taking backups of AWS Work Mail Accounts. How does the backup work? How does the backup work? We created an IAM Role in your account, giving the role access to export from AWS WorkMail, and then we attached a policy to the same role to allow it to access S3. We then create a KMS Key, which is given access to the IAM Role. The S3 bucket also needs access to the IAM role to function properly. Step 1: Configure AWS CLI First, ensure that the AWS CLI is installed and configured with appropriate credentials. Open your terminal and run: aws configure aws configure Follow the prompts to set your AWS Access Key ID, Secret Access Key, default region, and output format. Step 2: Set Up Required AWS Resources We need to create an IAM role, policies, an S3 bucket, and a KMS key. Save the following bash script as setup_workmail_export.sh . setup_workmail_export.sh #!/bin/bash # Configuration ROLE_NAME="WorkMailExportRole" POLICY_NAME="workmail-export" S3_BUCKET_NAME="s3.bucket.name" AWS_REGION="your-region" ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text) # Create Trust Policy cat trust-policy.json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": "export.workmail.amazonaws.com" }, "Action": "sts:AssumeRole", "Condition": { "StringEquals": { "sts:ExternalId": "$ACCOUNT_ID" } } } ] } EOF # Create IAM Role aws iam create-role --role-name $ROLE_NAME --assume-role-policy-document file://trust-policy.json # Create IAM Policy cat role-policy.json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "s3:*", "Resource": [ "arn:aws:s3:::$S3_BUCKET_NAME", "arn:aws:s3:::$S3_BUCKET_NAME/*" ] }, { "Effect": "Allow", "Action": [ "kms:Decrypt", "kms:GenerateDataKey" ], "Resource": "*" } ] } EOF # Attach the Policy to the Role aws iam put-role-policy --role-name $ROLE_NAME --policy-name $POLICY_NAME --policy-document file://role-policy.json # Create S3 Bucket aws s3api create-bucket --bucket $S3_BUCKET_NAME --region $AWS_REGION --create-bucket-configuration LocationConstraint=$AWS_REGION # Create Key Policy cat key-policy.json { "Version": "2012-10-17", "Id": "workmail-export-key", "Statement": [ { "Sid": "Enable IAM User Permissions", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::$ACCOUNT_ID:root" }, "Action": "kms:*", "Resource": "*" }, { "Sid": "Allow administration of the key", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::$ACCOUNT_ID:role/$ROLE_NAME" }, "Action": [ "kms:Create*", "kms:Describe*", "kms:Enable*", "kms:List*", "kms:Put*", "kms:Update*", "kms:Revoke*", "kms:Disable*", "kms:Get*", "kms:Delete*", "kms:ScheduleKeyDeletion", "kms:CancelKeyDeletion" ], "Resource": "*" } ] } EOF # Create the KMS Key and get the Key ID and ARN using Python for JSON parsing KEY_METADATA=$(aws kms create-key --policy file://key-policy.json) KEY_ID=$(python3 -c "import sys, json; print(json.load(sys.stdin)['KeyMetadata']['KeyId'])" s3_bucket_policy.json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::$ACCOUNT_ID:role/$ROLE_NAME" }, "Action": "s3:*" "Resource": [ "arn:aws:s3:::$S3_BUCKET_NAME", "arn:aws:s3:::$S3_BUCKET_NAME/*" ] } ] } EOF # Apply the Bucket Policy aws s3api put-bucket-policy --bucket $S3_BUCKET_NAME --policy file://s3_bucket_policy.json # Clean up temporary files rm trust-policy.json role-policy.json key-policy.json s3_bucket_policy.json # Display the variables required for the backup script cat <<EOF Setup complete. Here are the variables required for the backup script: # Print out the Variables organization_id = 'your-organization-id' user_id = 'your-user-id' s3_bucket_name = '$S3_BUCKET_NAME' s3_prefix = 'workmail-backup/' region = '$AWS_REGION' kms_key_arn = '$KEY_ARN' role_name = '$ROLE_NAME' account_id = '$ACCOUNT_ID' EOF #!/bin/bash # Configuration ROLE_NAME="WorkMailExportRole" POLICY_NAME="workmail-export" S3_BUCKET_NAME="s3.bucket.name" AWS_REGION="your-region" ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text) # Create Trust Policy cat trust-policy.json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Service": "export.workmail.amazonaws.com" }, "Action": "sts:AssumeRole", "Condition": { "StringEquals": { "sts:ExternalId": "$ACCOUNT_ID" } } } ] } EOF # Create IAM Role aws iam create-role --role-name $ROLE_NAME --assume-role-policy-document file://trust-policy.json # Create IAM Policy cat role-policy.json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Action": "s3:*", "Resource": [ "arn:aws:s3:::$S3_BUCKET_NAME", "arn:aws:s3:::$S3_BUCKET_NAME/*" ] }, { "Effect": "Allow", "Action": [ "kms:Decrypt", "kms:GenerateDataKey" ], "Resource": "*" } ] } EOF # Attach the Policy to the Role aws iam put-role-policy --role-name $ROLE_NAME --policy-name $POLICY_NAME --policy-document file://role-policy.json # Create S3 Bucket aws s3api create-bucket --bucket $S3_BUCKET_NAME --region $AWS_REGION --create-bucket-configuration LocationConstraint=$AWS_REGION # Create Key Policy cat key-policy.json { "Version": "2012-10-17", "Id": "workmail-export-key", "Statement": [ { "Sid": "Enable IAM User Permissions", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::$ACCOUNT_ID:root" }, "Action": "kms:*", "Resource": "*" }, { "Sid": "Allow administration of the key", "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::$ACCOUNT_ID:role/$ROLE_NAME" }, "Action": [ "kms:Create*", "kms:Describe*", "kms:Enable*", "kms:List*", "kms:Put*", "kms:Update*", "kms:Revoke*", "kms:Disable*", "kms:Get*", "kms:Delete*", "kms:ScheduleKeyDeletion", "kms:CancelKeyDeletion" ], "Resource": "*" } ] } EOF # Create the KMS Key and get the Key ID and ARN using Python for JSON parsing KEY_METADATA=$(aws kms create-key --policy file://key-policy.json) KEY_ID=$(python3 -c "import sys, json; print(json.load(sys.stdin)['KeyMetadata']['KeyId'])" s3_bucket_policy.json { "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "AWS": "arn:aws:iam::$ACCOUNT_ID:role/$ROLE_NAME" }, "Action": "s3:*" "Resource": [ "arn:aws:s3:::$S3_BUCKET_NAME", "arn:aws:s3:::$S3_BUCKET_NAME/*" ] } ] } EOF # Apply the Bucket Policy aws s3api put-bucket-policy --bucket $S3_BUCKET_NAME --policy file://s3_bucket_policy.json # Clean up temporary files rm trust-policy.json role-policy.json key-policy.json s3_bucket_policy.json # Display the variables required for the backup script cat <<EOF Setup complete. Here are the variables required for the backup script: # Print out the Variables organization_id = 'your-organization-id' user_id = 'your-user-id' s3_bucket_name = '$S3_BUCKET_NAME' s3_prefix = 'workmail-backup/' region = '$AWS_REGION' kms_key_arn = '$KEY_ARN' role_name = '$ROLE_NAME' account_id = '$ACCOUNT_ID' EOF Make the script executable and run it: chmod +x setup_workmail_export.sh ./setup_workmail_export.sh chmod +x setup_workmail_export.sh ./setup_workmail_export.sh Step 3: Write the Backup Script Now, let's write the Python script to export mailboxes in batches of 10. Save the following script as workmail_export.py . workmail_export.py import boto3 import json import time from datetime import datetime # Configuration organization_id = 'your-organization-id' user_emails = { 'user-id-1': 'user1@example.com', 'user-id-2': 'user2@example.com', 'user-id-3': 'user3@example.com', 'user-id-4': 'user4@example.com', 'user-id-5': 'user5@example.com', 'user-id-6': 'user6@example.com', 'user-id-7': 'user7@example.com', 'user-id-8': 'user8@example.com', 'user-id-9': 'user9@example.com', 'user-id-10': 'user10@example.com', 'user-id-11': 'user11@example.com', 'user-id-12': 'user12@example.com' # Add more user ID to email mappings as needed } s3_bucket_name = 's3.bucket.name' region = 'your-region' kms_key_arn = 'arn:aws:kms:your-region:your-account-id:key/your-key-id' role_name = 'WorkMailExportRole' account_id = 'your-account-id' # Get the current date current_date = datetime.now().strftime('%Y-%m-%d') # Set the S3 prefix with the date included s3_prefix_base = f'workmail-backup/{current_date}/' # Initialize AWS clients workmail = boto3.client('workmail', region_name=region) sts = boto3.client('sts', region_name=region) def start_export_job(entity_id, user_email): client_token = str(time.time()) # Unique client token role_arn = f"arn:aws:iam::{account_id}:role/{role_name}" s3_prefix = f"{s3_prefix_base}{user_email}/" try: response = workmail.start_mailbox_export_job( ClientToken=client_token, OrganizationId=organization_id, EntityId=entity_id, Description='Backup job', RoleArn=role_arn, KmsKeyArn=kms_key_arn, S3BucketName=s3_bucket_name, S3Prefix=s3_prefix ) return response['JobId'] except Exception as e: print(f"Failed to start export job for {entity_id}: {e}") return None def check_job_status(job_id): while True: try: response = workmail.describe_mailbox_export_job( OrganizationId=organization_id, JobId=job_id ) print(f"Full Response: {response}") # Log full response for debugging state = response.get('State', 'UNKNOWN') print(f"Job State: {state}") if state in ['COMPLETED', 'FAILED']: break except Exception as e : print(f"Error checking job status for {job_id}: {e}") time.sleep(30) # Wait for 30 seconds before checking again return state def export_mailboxes_in_batches(user_emails, batch_size=10): user_ids = list(user_emails.keys()) for i in range(0, len(user_ids), batch_size): batch = user_ids[i:i+batch_size] job_ids = [] for user_id in batch: user_email = user_emails[user_id] job_id = start_export_job(user_id, user_email) if job_id: print(f"Started export job for {user_email} with Job ID: {job_id}") job_ids.append((user_email, job_id)) for user_email, job_id in job_ids: state = check_job_status(job_id) if state == 'COMPLETED': print(f"Export job for {user_email} completed successfully") else: print(f"Export job for {user_email} failed with state: {state}") def main(): export_mailboxes_in_batches(user_emails) if __name__ == "__main__": main() import boto3 import json import time from datetime import datetime # Configuration organization_id = 'your-organization-id' user_emails = { 'user-id-1': 'user1@example.com', 'user-id-2': 'user2@example.com', 'user-id-3': 'user3@example.com', 'user-id-4': 'user4@example.com', 'user-id-5': 'user5@example.com', 'user-id-6': 'user6@example.com', 'user-id-7': 'user7@example.com', 'user-id-8': 'user8@example.com', 'user-id-9': 'user9@example.com', 'user-id-10': 'user10@example.com', 'user-id-11': 'user11@example.com', 'user-id-12': 'user12@example.com' # Add more user ID to email mappings as needed } s3_bucket_name = 's3.bucket.name' region = 'your-region' kms_key_arn = 'arn:aws:kms:your-region:your-account-id:key/your-key-id' role_name = 'WorkMailExportRole' account_id = 'your-account-id' # Get the current date current_date = datetime.now().strftime('%Y-%m-%d') # Set the S3 prefix with the date included s3_prefix_base = f'workmail-backup/{current_date}/' # Initialize AWS clients workmail = boto3.client('workmail', region_name=region) sts = boto3.client('sts', region_name=region) def start_export_job(entity_id, user_email): client_token = str(time.time()) # Unique client token role_arn = f"arn:aws:iam::{account_id}:role/{role_name}" s3_prefix = f"{s3_prefix_base}{user_email}/" try: response = workmail.start_mailbox_export_job( ClientToken=client_token, OrganizationId=organization_id, EntityId=entity_id, Description='Backup job', RoleArn=role_arn, KmsKeyArn=kms_key_arn, S3BucketName=s3_bucket_name, S3Prefix=s3_prefix ) return response['JobId'] except Exception as e: print(f"Failed to start export job for {entity_id}: {e}") return None def check_job_status(job_id): while True: try: response = workmail.describe_mailbox_export_job( OrganizationId=organization_id, JobId=job_id ) print(f"Full Response: {response}") # Log full response for debugging state = response.get('State', 'UNKNOWN') print(f"Job State: {state}") if state in ['COMPLETED', 'FAILED']: break except Exception as e : print(f"Error checking job status for {job_id}: {e}") time.sleep(30) # Wait for 30 seconds before checking again return state def export_mailboxes_in_batches(user_emails, batch_size=10): user_ids = list(user_emails.keys()) for i in range(0, len(user_ids), batch_size): batch = user_ids[i:i+batch_size] job_ids = [] for user_id in batch: user_email = user_emails[user_id] job_id = start_export_job(user_id, user_email) if job_id: print(f"Started export job for {user_email} with Job ID: {job_id}") job_ids.append((user_email, job_id)) for user_email, job_id in job_ids: state = check_job_status(job_id) if state == 'COMPLETED': print(f"Export job for {user_email} completed successfully") else: print(f"Export job for {user_email} failed with state: {state}") def main(): export_mailboxes_in_batches(user_emails) if __name__ == "__main__": main() Replace the placeholders with your actual AWS WorkMail organization ID, user ID to email mappings, S3 bucket name, region, KMS key ARN, role name, and AWS account ID. Step 4: Run the Backup Script Ensure you have Boto3 installed: pip install boto3 pip install boto3 Then, execute the Python script: python workmail_export.py python workmail_export.py Import Mailbox import boto3 import json import time # Configuration organization_id = 'your-organization-id' user_import_data = { 'user-id-1': 'user1@example.com', 'user-id-2': 'user2@example.com', 'user-id-3': 'user3@example.com', 'user-id-4': 'user4@example.com', 'user-id-5': 'user5@example.com', 'user-id-6': 'user6@example.com', 'user-id-7': 'user7@example.com', 'user-id-8': 'user8@example.com', 'user-id-9': 'user9@example.com', 'user-id-10': 'user10@example.com', 'user-id-11': 'user11@example.com', 'user-id-12': 'user12@example.com' # Add more user ID to email mappings as needed } s3_bucket_name = 's3.bucket.name' s3_object_prefix = 'workmail-backup/' # Prefix for S3 objects (folders) region = 'your-region' role_name = 'WorkMailImportRole' account_id = 'your-account-id' # Initialize AWS clients workmail = boto3.client('workmail', region_name=region) sts = boto3.client('sts', region_name=region) def start_import_job(entity_id, user_email): client_token = str(time.time()) # Unique client token role_arn = f"arn:aws:iam::{account_id}:role/{role_name}" s3_object_key = f"{s3_object_prefix}{user_email}/export.zip" try: response = workmail.start_mailbox_import_job( ClientToken=client_token, OrganizationId=organization_id, EntityId=entity_id, Description='Import job', RoleArn=role_arn, S3BucketName=s3_bucket_name, S3ObjectKey=s3_object_key ) return response['JobId'] except Exception as e: print(f"Failed to start import job for {entity_id}: {e}") return None def check_job_status(job_id): while True: try: response = workmail.describe_mailbox_import_job( OrganizationId=organization_id, JobId=job_id ) state = response.get('State', 'UNKNOWN') print(f"Job State: {state}") if state in ['COMPLETED', 'FAILED']: break except Exception as e: print(f"Error checking job status for {job_id}: {e}") time.sleep(30) # Wait for 30 seconds before checking again return state def import_mailboxes_in_batches(user_import_data, batch_size=10): user_ids = list(user_import_data.keys()) for i in range(0, len(user_ids), batch_size): batch = user_ids[i:i+batch_size] job_ids = [] for user_id in batch: user_email = user_import_data[user_id] job_id = start_import_job(user_id, user_email) if job_id: print(f"Started import job for {user_email} with Job ID: {job_id}") job_ids.append((user_email, job_id)) for user_email, job_id in job_ids: state = check_job_status(job_id) if state == 'COMPLETED': print(f"Import job for {user_email} completed successfully") else: print(f"Import job for {user_email} failed with state: {state}") def main(): import_mailboxes_in_batches(user_import_data) if __name__ == "__main__": main() import boto3 import json import time # Configuration organization_id = 'your-organization-id' user_import_data = { 'user-id-1': 'user1@example.com', 'user-id-2': 'user2@example.com', 'user-id-3': 'user3@example.com', 'user-id-4': 'user4@example.com', 'user-id-5': 'user5@example.com', 'user-id-6': 'user6@example.com', 'user-id-7': 'user7@example.com', 'user-id-8': 'user8@example.com', 'user-id-9': 'user9@example.com', 'user-id-10': 'user10@example.com', 'user-id-11': 'user11@example.com', 'user-id-12': 'user12@example.com' # Add more user ID to email mappings as needed } s3_bucket_name = 's3.bucket.name' s3_object_prefix = 'workmail-backup/' # Prefix for S3 objects (folders) region = 'your-region' role_name = 'WorkMailImportRole' account_id = 'your-account-id' # Initialize AWS clients workmail = boto3.client('workmail', region_name=region) sts = boto3.client('sts', region_name=region) def start_import_job(entity_id, user_email): client_token = str(time.time()) # Unique client token role_arn = f"arn:aws:iam::{account_id}:role/{role_name}" s3_object_key = f"{s3_object_prefix}{user_email}/export.zip" try: response = workmail.start_mailbox_import_job( ClientToken=client_token, OrganizationId=organization_id, EntityId=entity_id, Description='Import job', RoleArn=role_arn, S3BucketName=s3_bucket_name, S3ObjectKey=s3_object_key ) return response['JobId'] except Exception as e: print(f"Failed to start import job for {entity_id}: {e}") return None def check_job_status(job_id): while True: try: response = workmail.describe_mailbox_import_job( OrganizationId=organization_id, JobId=job_id ) state = response.get('State', 'UNKNOWN') print(f"Job State: {state}") if state in ['COMPLETED', 'FAILED']: break except Exception as e: print(f"Error checking job status for {job_id}: {e}") time.sleep(30) # Wait for 30 seconds before checking again return state def import_mailboxes_in_batches(user_import_data, batch_size=10): user_ids = list(user_import_data.keys()) for i in range(0, len(user_ids), batch_size): batch = user_ids[i:i+batch_size] job_ids = [] for user_id in batch: user_email = user_import_data[user_id] job_id = start_import_job(user_id, user_email) if job_id: print(f"Started import job for {user_email} with Job ID: {job_id}") job_ids.append((user_email, job_id)) for user_email, job_id in job_ids: state = check_job_status(job_id) if state == 'COMPLETED': print(f"Import job for {user_email} completed successfully") else: print(f"Import job for {user_email} failed with state: {state}") def main(): import_mailboxes_in_batches(user_import_data) if __name__ == "__main__": main() By following these steps, you have set up an automated system to back up your AWS WorkMail mailboxes, handling up to 10 concurrent export jobs. This solution ensures that your emails are securely stored in an S3 bucket, organized by user email and date, and encrypted with a KMS key. This setup provides a robust and scalable backup strategy for your organization's email data. Github Repo: https://github.com/th3n00bc0d3r/AWS-WorkMail-Backup Github Repo: https://github.com/th3n00bc0d3r/AWS-WorkMail-Backup https://github.com/th3n00bc0d3r/AWS-WorkMail-Backup Feel free to get in touch on my LinkedIn, Also you can always buy me a coffee Feel free to get in touch on my LinkedIn , Also you can always buy me a coffee LinkedIn buy me a coffee https://github.com/th3n00bc0d3r/AWS-WorkMail-Backup?embedable=true https://github.com/th3n00bc0d3r/AWS-WorkMail-Backup?embedable=true