When building Python applications with Poetry in a Docker container, we sometimes encounter issues accessing private packages stored in the Google Artifact Registry (GAR). Locally, this challenge arises because docker build cannot directly handle the Google Cloud credentials in the same way as our CI/CD pipeline, where we leverage service accounts and the Kaniko action for secure builds. To resolve this for local development, we'll configure Google Cloud credentials within Docker using Docker Compose, enabling secure access to the GAR repository during the build process. Non-Containerized Pre-requirements Python ^3.12
Poetry ^1.7.1
Google Cloud SDK ^489.0.0
Keyring ^24.0.0
keyrings.google-artifactregistry-auth ^1.0.0
Read access to your GAR Python repository
Logged into GCP (gcloud auth application-default login) Setup Keyring Setup
Install keyring support for Google Artifact Registry:
pip install keyring
pip install keyrings.google-artifactregistry-auth Connect Poetry to your GAR repo
Access to the private repository in the Google Artifact Registry can be managed through Poetry. First, configure a custom source in Poetry for the GAR repository by running:
poetry source add --priority=explicit <PACKAGE_NAME> https://<REGION>-python.pkg.dev/<PROJECT>/<REGISTRY>/simple Be sure to append /simple to the repository URL for compatibility. Now you can install packages from your private repo: poetry add --source <PACKAGE_NAME> some-package@1.0.0 Notes In some cases, accessing the repository may require setting an explicit OAuth token for authentication in Poetry. Use the following command to configure this globally in Poetry: poetry config http-basic.<PACKAGE_NAME> oauth2accesstoken $(gcloud auth print-access-token) Containerized Pre-requirements Docker ^20.10
Docker Compose ^2.0
Google Cloud SDK ^489.0.0
Read access to your GAR Python repository
Logged into GCP (gcloud auth application-default login) Setup Secrets Configuration
First, define a secret in docker-compose.yaml using the local path to your credentials file:
secrets:
  gcloud_credentials:
    file: ~/.config/gcloud/application_default_credentials.json



We define gcloud_credentials.file as a consistent path for Unix-like environments in docker-compose.yaml:


This configuration securely passes the credentials file from your local machine to the build context without exposing sensitive data.




Dockerfile Adjustments
In the Dockerfile, we handle credentials with the following setup:
ARG GOOGLE_APPLICATION_CREDENTIALS
ENV GOOGLE_APPLICATION_CREDENTIALS=${GOOGLE_APPLICATION_CREDENTIALS}

RUN --mount=type=secret,id=gcloud_credentials \
    mkdir -p $(dirname ${GOOGLE_APPLICATION_CREDENTIALS}) && \
    cp /run/secrets/gcloud_credentials ${GOOGLE_APPLICATION_CREDENTIALS}



--mount=type=secret,id=gcloud_credentials: Securely mounts the credentials during the build process.


GOOGLE_APPLICATION_CREDENTIALS: Specifies the credential file's path within the container.




Service Configurations
Here, gcloud_credentials is the secret mounted at build time, as specified in the secrets configuration.
some-service:
  build:
    context: .
    dockerfile: ./Dockerfile.local
    args:
      GOOGLE_APPLICATION_CREDENTIALS: /tmp/application_default_credentials.json
    secrets:
      - gcloud_credentials Usage Just run: docker-compose up --build Conclusion This approach allows local Docker builds to access private GAR resources securely, ensuring that credentials are handled appropriately and remain protected. When building Python applications with Poetry in a Docker container, we sometimes encounter issues accessing private packages stored in the Google Artifact Registry (GAR). Locally, this challenge arises because docker build cannot directly handle the Google Cloud credentials in the same way as our CI/CD pipeline, where we leverage service accounts and the Kaniko action for secure builds. To resolve this for local development, we'll configure Google Cloud credentials within Docker using Docker Compose, enabling secure access to the GAR repository during the build process. docker build Docker Non-Containerized Pre-requirements Python ^3.12 Poetry ^1.7.1 Google Cloud SDK ^489.0.0 Keyring ^24.0.0 keyrings.google-artifactregistry-auth ^1.0.0 Read access to your GAR Python repository Logged into GCP (gcloud auth application-default login) Python ^3.12 Poetry ^1.7.1 Google Cloud SDK ^489.0.0 Keyring ^24.0.0 keyrings.google-artifactregistry-auth ^1.0.0 Read access to your GAR Python repository Logged into GCP ( gcloud auth application-default login ) gcloud auth application-default login Setup Keyring Setup
Install keyring support for Google Artifact Registry:
pip install keyring
pip install keyrings.google-artifactregistry-auth Keyring Setup Install keyring support for Google Artifact Registry: pip install keyring
pip install keyrings.google-artifactregistry-auth Keyring Setup Keyring Setup Install keyring support for Google Artifact Registry: keyring pip install keyring
pip install keyrings.google-artifactregistry-auth pip install keyring
pip install keyrings.google-artifactregistry-auth Connect Poetry to your GAR repo
Access to the private repository in the Google Artifact Registry can be managed through Poetry. First, configure a custom source in Poetry for the GAR repository by running:
poetry source add --priority=explicit <PACKAGE_NAME> https://<REGION>-python.pkg.dev/<PROJECT>/<REGISTRY>/simple Connect Poetry to your GAR repo Access to the private repository in the Google Artifact Registry can be managed through Poetry. First, configure a custom source in Poetry for the GAR repository by running: poetry source add --priority=explicit <PACKAGE_NAME> https://<REGION>-python.pkg.dev/<PROJECT>/<REGISTRY>/simple Connect Poetry to your GAR repo Connect Poetry to your GAR repo Access to the private repository in the Google Artifact Registry can be managed through Poetry. First, configure a custom source in Poetry for the GAR repository by running: poetry source add --priority=explicit <PACKAGE_NAME> https://<REGION>-python.pkg.dev/<PROJECT>/<REGISTRY>/simple poetry source add --priority=explicit <PACKAGE_NAME> https://<REGION>-python.pkg.dev/<PROJECT>/<REGISTRY>/simple Be sure to append /simple to the repository URL for compatibility. Be sure to append /simple to the repository URL for compatibility. /simple Now you can install packages from your private repo: poetry add --source <PACKAGE_NAME> some-package@1.0.0 poetry add --source <PACKAGE_NAME> some-package@1.0.0 Notes In some cases, accessing the repository may require setting an explicit OAuth token for authentication in Poetry. Use the following command to configure this globally in Poetry: poetry config http-basic.<PACKAGE_NAME> oauth2accesstoken $(gcloud auth print-access-token) poetry config http-basic.<PACKAGE_NAME> oauth2accesstoken $(gcloud auth print-access-token) Containerized Pre-requirements Docker ^20.10 Docker Compose ^2.0 Google Cloud SDK ^489.0.0 Read access to your GAR Python repository Logged into GCP (gcloud auth application-default login) Docker ^20.10 Docker Compose ^2.0 Google Cloud SDK ^489.0.0 Read access to your GAR Python repository Logged into GCP ( gcloud auth application-default login ) gcloud auth application-default login Setup Secrets Configuration
First, define a secret in docker-compose.yaml using the local path to your credentials file:
secrets:
  gcloud_credentials:
    file: ~/.config/gcloud/application_default_credentials.json



We define gcloud_credentials.file as a consistent path for Unix-like environments in docker-compose.yaml:


This configuration securely passes the credentials file from your local machine to the build context without exposing sensitive data. Dockerfile Adjustments
In the Dockerfile, we handle credentials with the following setup:
ARG GOOGLE_APPLICATION_CREDENTIALS
ENV GOOGLE_APPLICATION_CREDENTIALS=${GOOGLE_APPLICATION_CREDENTIALS}

RUN --mount=type=secret,id=gcloud_credentials \
    mkdir -p $(dirname ${GOOGLE_APPLICATION_CREDENTIALS}) && \
    cp /run/secrets/gcloud_credentials ${GOOGLE_APPLICATION_CREDENTIALS}



--mount=type=secret,id=gcloud_credentials: Securely mounts the credentials during the build process.


GOOGLE_APPLICATION_CREDENTIALS: Specifies the credential file's path within the container. Service Configurations
Here, gcloud_credentials is the secret mounted at build time, as specified in the secrets configuration.
some-service:
  build:
    context: .
    dockerfile: ./Dockerfile.local
    args:
      GOOGLE_APPLICATION_CREDENTIALS: /tmp/application_default_credentials.json
    secrets:
      - gcloud_credentials Secrets Configuration First, define a secret in docker-compose.yaml using the local path to your credentials file: secrets:
  gcloud_credentials:
    file: ~/.config/gcloud/application_default_credentials.json We define gcloud_credentials.file as a consistent path for Unix-like environments in docker-compose.yaml:


This configuration securely passes the credentials file from your local machine to the build context without exposing sensitive data. Secrets Configuration Secrets Configuration First, define a secret in docker-compose.yaml using the local path to your credentials file: docker-compose.yaml secrets:
  gcloud_credentials:
    file: ~/.config/gcloud/application_default_credentials.json secrets:
  gcloud_credentials:
    file: ~/.config/gcloud/application_default_credentials.json We define gcloud_credentials.file as a consistent path for Unix-like environments in docker-compose.yaml: This configuration securely passes the credentials file from your local machine to the build context without exposing sensitive data. We define gcloud_credentials.file as a consistent path for Unix-like environments in docker-compose.yaml: We define gcloud_credentials.file as a consistent path for Unix-like environments in docker-compose.yaml : gcloud_credentials.file docker-compose.yaml This configuration securely passes the credentials file from your local machine to the build context without exposing sensitive data. This configuration securely passes the credentials file from your local machine to the build context without exposing sensitive data. Dockerfile Adjustments In the Dockerfile, we handle credentials with the following setup: ARG GOOGLE_APPLICATION_CREDENTIALS
ENV GOOGLE_APPLICATION_CREDENTIALS=${GOOGLE_APPLICATION_CREDENTIALS}

RUN --mount=type=secret,id=gcloud_credentials \
    mkdir -p $(dirname ${GOOGLE_APPLICATION_CREDENTIALS}) && \
    cp /run/secrets/gcloud_credentials ${GOOGLE_APPLICATION_CREDENTIALS} --mount=type=secret,id=gcloud_credentials: Securely mounts the credentials during the build process.


GOOGLE_APPLICATION_CREDENTIALS: Specifies the credential file's path within the container. Dockerfile Adjustments Dockerfile Adjustments In the Dockerfile, we handle credentials with the following setup: ARG GOOGLE_APPLICATION_CREDENTIALS
ENV GOOGLE_APPLICATION_CREDENTIALS=${GOOGLE_APPLICATION_CREDENTIALS}

RUN --mount=type=secret,id=gcloud_credentials \
    mkdir -p $(dirname ${GOOGLE_APPLICATION_CREDENTIALS}) && \
    cp /run/secrets/gcloud_credentials ${GOOGLE_APPLICATION_CREDENTIALS} ARG GOOGLE_APPLICATION_CREDENTIALS
ENV GOOGLE_APPLICATION_CREDENTIALS=${GOOGLE_APPLICATION_CREDENTIALS}

RUN --mount=type=secret,id=gcloud_credentials \
    mkdir -p $(dirname ${GOOGLE_APPLICATION_CREDENTIALS}) && \
    cp /run/secrets/gcloud_credentials ${GOOGLE_APPLICATION_CREDENTIALS} --mount=type=secret,id=gcloud_credentials: Securely mounts the credentials during the build process. GOOGLE_APPLICATION_CREDENTIALS: Specifies the credential file's path within the container. --mount=type=secret,id=gcloud_credentials: Securely mounts the credentials during the build process. --mount=type=secret,id=gcloud_credentials : Securely mounts the credentials during the build process. --mount=type=secret,id=gcloud_credentials GOOGLE_APPLICATION_CREDENTIALS: Specifies the credential file's path within the container. GOOGLE_APPLICATION_CREDENTIALS : Specifies the credential file's path within the container. GOOGLE_APPLICATION_CREDENTIALS Service Configurations Here, gcloud_credentials is the secret mounted at build time, as specified in the secrets configuration. some-service:
  build:
    context: .
    dockerfile: ./Dockerfile.local
    args:
      GOOGLE_APPLICATION_CREDENTIALS: /tmp/application_default_credentials.json
    secrets:
      - gcloud_credentials Service Configurations Service Configurations Here, gcloud_credentials is the secret mounted at build time, as specified in the secrets configuration. gcloud_credentials some-service:
  build:
    context: .
    dockerfile: ./Dockerfile.local
    args:
      GOOGLE_APPLICATION_CREDENTIALS: /tmp/application_default_credentials.json
    secrets:
      - gcloud_credentials some-service:
  build:
    context: .
    dockerfile: ./Dockerfile.local
    args:
      GOOGLE_APPLICATION_CREDENTIALS: /tmp/application_default_credentials.json
    secrets:
      - gcloud_credentials Usage Just run: docker-compose up --build docker-compose up --build Conclusion This approach allows local Docker builds to access private GAR resources securely, ensuring that credentials are handled appropriately and remain protected. Docker

Walkthroughs, tutorials, guides, and tips. This story will teach you how to do something new or how to do something better.

Setting up Yandex Cloud Provider with Terraform and Terragrunt

Accessing Private Google Artifact Registry with Poetry: Local and Docker Setup

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Fetching Large Logs from Loki in Kubernetes

104 Stories To Learn About Continuous Integration

139 Stories To Learn About Cicd

15 of the Best Continuous Delivery Tools

5 Best Microservices CI/CD Tools You Need to Check Out

53 Stories To Learn About Continuous Deployment

Fetching Large Logs from Loki in Kubernetes

104 Stories To Learn About Continuous Integration

139 Stories To Learn About Cicd

15 of the Best Continuous Delivery Tools

5 Best Microservices CI/CD Tools You Need to Check Out

53 Stories To Learn About Continuous Deployment

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps