The rationale behind this project stems from the various stability issues encountered with Harbor, other registries, prompting a transition away from it. Project Plan Set up ECR infrastructure using Terraform, This includes private repos creation. provider "aws" 
{region = "us-east-1" # Change to your AWS region# 
Optionally, specify a profile name if you're using AWS named profiles
# profile = "your-profile-name"}resource "aws_ecr_repository" "my_ecr_repo" {name = "my-ecr-repo" # Change this to your desired repo nameimage_tag_mutability = "MUTABLE"image_scanning_configuration {scan_on_push = true}
# If you need to create more repositories, you can either duplicate this block# and change the name, or use a more dynamic approach with `count` or `for_each`.# Optional: Enable lifecycle policy to manage image count or untagged imageslifecycle_policy {policy = jsonencode({rules = [{rulePriority = 1description = "Expire untagged images"selection = {==tagStatus = "untagged"countType = "imageCountMoreThan"countNumber = 100}action = {type = "expire"} }, ] })}} 2. Implement replication rules for cross-region and cross-account functionality. mutable-repos = ["apps/signatures",]
ecr_replication_destinations_rules = 
{lab1 = {region = "us-west-2"id = "<>"}
lab2 = {region = "us-east-1"id = "<>"}
stage1 = {region = "ap-southeast-1"id = "<>"}
stage2 = {region = "us-east-1"id = "<>"}
prod = {region = "us-west-2"id = "<>"}} 3. Once the Infrastructure is available, adapt application pipelines to push images and Helm charts to ECR along with Harbor. Activities To be Performed Prepare User “ecr_role” with necessary below permissions, facilitating automated image/chart pushes to ECR in the CI/CD section. Established ECR private repositories for all applications, replicating the setup across remaining environments to manage replication costs.
Updated and executed pipelines for all application components to ensure visibility of images in both locations (Harbor and ECR).
Provided implementation guidelines to the Application team, detailing necessary changes and their locations.
Transitioned to ECR from Harbor across all environments (Lab, Stage, Production). Moving Forward with Helm Our idea is to maintain the whole CD through ArgoCD. So for that, we need to add this as a repository in Argo. A new Challenge emerges with OCI calls in Helm implementation as it will not use HTTPs calls. Resolution Install External secrets in the cluster where we have Argo helm repo add external-secrets https://charts.external-secrets.iohelm install external-secrets \external-secrets/external-secrets \-n external-secrets \--create-namespace \--set installCRDs=true 2. Create AWS credentials in K8 secrets, or if you are using gitlab for pulling the data to argo, then you can configure the same under export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLEexport AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEYecho -n AWS_SECRET_ACCESS_KEY > ./secret-access-keyecho -n AWS_ACCESS_KEY_ID > ./access-key kubectl create secret generic awssm-secret --from-file=./access-key --from-file=./secret-access-key 3. The secret created in previous step, swasm-secret used as below apiVersion: generators.external-secrets.io/v1alpha1kind: ECRAuthorizationTokenmetadata: name: ecr-genspec: # specify aws region (mandatory) region: us-west-1 auth:   # 1: static credentials   # point to a secret that contains static credentials   # like AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY   secretRef:     accessKeyIDSecretRef:        name: "awssm-secret"    key: "access-key" secretAccessKeySecretRef: name: "awssm-secret" key: "secret-access-key" Challenges The predominant challenge involved handling Helm charts from ArgoCD when storing an OCI Helm repo in ECR, necessitating token rotation using “External Secrets Operator”. Achievements: Achieved zero downtime across all environments despite numerous redeployments and adherence to application team timelines. Progress made on the migration from Harbor to ECR, with no disruptions in operations. If this post was helpful, please do follow and click the clap 👏 button below to show your support. Follow me on Linked-in for further updates- https://www.linkedin.com/in/chaitanya-kanth-tummalachervu-296956121/ The rationale behind this project stems from the various stability issues encountered with Harbor, other registries, prompting a transition away from it. Project Plan Project Plan Set up ECR infrastructure using Terraform, This includes private repos creation. Set up ECR infrastructure using Terraform, This includes private repos creation. Set up ECR infrastructure using Terraform, This includes private repos creation. provider "aws" 
{region = "us-east-1" # Change to your AWS region# 
Optionally, specify a profile name if you're using AWS named profiles
# profile = "your-profile-name"}resource "aws_ecr_repository" "my_ecr_repo" {name = "my-ecr-repo" # Change this to your desired repo nameimage_tag_mutability = "MUTABLE"image_scanning_configuration {scan_on_push = true}
# If you need to create more repositories, you can either duplicate this block# and change the name, or use a more dynamic approach with `count` or `for_each`.# Optional: Enable lifecycle policy to manage image count or untagged imageslifecycle_policy {policy = jsonencode({rules = [{rulePriority = 1description = "Expire untagged images"selection = {==tagStatus = "untagged"countType = "imageCountMoreThan"countNumber = 100}action = {type = "expire"} }, ] })}} provider "aws" 
{region = "us-east-1" # Change to your AWS region# 
Optionally, specify a profile name if you're using AWS named profiles
# profile = "your-profile-name"}resource "aws_ecr_repository" "my_ecr_repo" {name = "my-ecr-repo" # Change this to your desired repo nameimage_tag_mutability = "MUTABLE"image_scanning_configuration {scan_on_push = true}
# If you need to create more repositories, you can either duplicate this block# and change the name, or use a more dynamic approach with `count` or `for_each`.# Optional: Enable lifecycle policy to manage image count or untagged imageslifecycle_policy {policy = jsonencode({rules = [{rulePriority = 1description = "Expire untagged images"selection = {==tagStatus = "untagged"countType = "imageCountMoreThan"countNumber = 100}action = {type = "expire"} }, ] })}} 2. Implement replication rules for cross-region and cross-account functionality. mutable-repos = ["apps/signatures",]
ecr_replication_destinations_rules = 
{lab1 = {region = "us-west-2"id = "<>"}
lab2 = {region = "us-east-1"id = "<>"}
stage1 = {region = "ap-southeast-1"id = "<>"}
stage2 = {region = "us-east-1"id = "<>"}
prod = {region = "us-west-2"id = "<>"}} mutable-repos = ["apps/signatures",]
ecr_replication_destinations_rules = 
{lab1 = {region = "us-west-2"id = "<>"}
lab2 = {region = "us-east-1"id = "<>"}
stage1 = {region = "ap-southeast-1"id = "<>"}
stage2 = {region = "us-east-1"id = "<>"}
prod = {region = "us-west-2"id = "<>"}} 3. Once the Infrastructure is available, adapt application pipelines to push images and Helm charts to ECR along with Harbor. Activities To be Performed Activities To be Performed Prepare User “ecr_role” with necessary below permissions, facilitating automated image/chart pushes to ECR in the CI/CD section. Prepare User “ecr_role” with necessary below permissions, facilitating automated image/chart pushes to ECR in the CI/CD section. Established ECR private repositories for all applications, replicating the setup across remaining environments to manage replication costs. Updated and executed pipelines for all application components to ensure visibility of images in both locations (Harbor and ECR). Provided implementation guidelines to the Application team, detailing necessary changes and their locations. Transitioned to ECR from Harbor across all environments (Lab, Stage, Production). Established ECR private repositories for all applications, replicating the setup across remaining environments to manage replication costs. Updated and executed pipelines for all application components to ensure visibility of images in both locations (Harbor and ECR). Provided implementation guidelines to the Application team, detailing necessary changes and their locations. Transitioned to ECR from Harbor across all environments (Lab, Stage, Production). Moving Forward with Helm Our idea is to maintain the whole CD through ArgoCD. So for that, we need to add this as a repository in Argo. A new Challenge emerges with OCI calls in Helm implementation as it will not use HTTPs calls. Resolution Install External secrets in the cluster where we have Argo helm repo add external-secrets https://charts.external-secrets.iohelm install external-secrets \external-secrets/external-secrets \-n external-secrets \--create-namespace \--set installCRDs=true Install External secrets in the cluster where we have Argo helm repo add external-secrets https://charts.external-secrets.iohelm install external-secrets \external-secrets/external-secrets \-n external-secrets \--create-namespace \--set installCRDs=true Install External secrets in the cluster where we have Argo helm repo add external-secrets https://charts.external-secrets.io helm install external-secrets \ external-secrets/external-secrets \ -n external-secrets \ --create-namespace \ --set installCRDs=true helm repo add external-secrets https://charts.external-secrets.io helm install external-secrets \ external-secrets/external-secrets \ -n external-secrets \ --create-namespace \ --set installCRDs=true 2. Create AWS credentials in K8 secrets, or if you are using gitlab for pulling the data to argo, then you can configure the same under export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY echo -n AWS_SECRET_ACCESS_KEY > ./secret-access-key echo -n AWS_ACCESS_KEY_ID > ./access-key kubectl create secret generic awssm-secret --from-file=./access-key --from-file=./secret-access-key export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY echo -n AWS_SECRET_ACCESS_KEY > ./secret-access-key echo -n AWS_ACCESS_KEY_ID > ./access-key kubectl create secret generic awssm-secret --from-file=./access-key --from-file=./secret-access-key 3. The secret created in previous step, swasm-secret used as below apiVersion: generators.external-secrets.io/v1alpha1 kind: ECRAuthorizationToken metadata: name: ecr-gen spec: # specify aws region (mandatory) region: us-west-1 auth: # 1: static credentials # point to a secret that contains static credentials # like AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY secretRef: accessKeyIDSecretRef: name: "awssm-secret" key: "access-key" secretAccessKeySecretRef: name: "awssm-secret" key: "secret-access-key" kind: ECRAuthorizationToken metadata: name: ecr-gen spec: # specify aws region (mandatory) region: us-west-1 auth: # 1: static credentials # point to a secret that contains static credentials # like AWS_ACCESS_KEY_ID / AWS_SECRET_ACCESS_KEY secretRef: accessKeyIDSecretRef: name: "awssm-secret" key: "access-key" secretAccessKeySecretRef: name: "awssm-secret" key: "secret-access-key" Challenges The predominant challenge involved handling Helm charts from ArgoCD when storing an OCI Helm repo in ECR, necessitating token rotation using “External Secrets Operator”. Achievements: Achieved zero downtime across all environments despite numerous redeployments and adherence to application team timelines. Progress made on the migration from Harbor to ECR, with no disruptions in operations. If this post was helpful, please do follow and click the clap 👏 button below to show your support. If this post was helpful, please do follow and click the clap 👏 button below to show your support. Follow me on Linked-in for further updates- https://www.linkedin.com/in/chaitanya-kanth-tummalachervu-296956121/ https://www.linkedin.com/in/chaitanya-kanth-tummalachervu-296956121/

Read My Stories

Migrating Container Images and Helm Charts from Harbor to ECR

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

Delivering Cloud Infrastructure the GitOps Way with Crossplane and ArgoCD

Getting Started with GitOps 2.0 Using ArgoCD & Codefresh

How to Handle Kubernetes Secrets with ArgoCD and Sops

How to Optimize Kubernetes and Argo Workflow Spending

Managing Multi-Environment Kubernetes Deployments with GitOps and Argo CD

Delivering Cloud Infrastructure the GitOps Way with Crossplane and ArgoCD

Getting Started with GitOps 2.0 Using ArgoCD & Codefresh

How to Handle Kubernetes Secrets with ArgoCD and Sops

How to Optimize Kubernetes and Argo Workflow Spending

Managing Multi-Environment Kubernetes Deployments with GitOps and Argo CD

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps