In this tutorial, I will demonstrate how to leverage OAuth2 Proxy to restrict access to your services exclusively to your GitHub organization's members, teams, or yourself. Kubernetes If you're like me then you might have a long list of self-hosted third-party services running inside of your Kubernetes clusters, such as , Sentry.io, Elastic, and Jaeger. Some of these services may have their own mechanism for authentication, some can be configured with an external OAuth provider, and some won't have any authentication mechanism at all. Grafana How can we easily expose our services over the internet with a proper audit trail and without letting any unauthorized users in? One surprisingly simple solution is to create a GitHub OAuth app and configure your ingresses to use a proxy that permits access based on which teams a user is part of. GitHub That's a lot of words, but I'll show you that setting it up is just minutes of work, and the result is protected internal services with just two ingress annotations: nginx.ingress.kubernetes.io/auth-signin: https://oauth2.symbiosis.host/oauth2/start?rd=https://$host$uri nginx.ingress.kubernetes.io/auth-url: https://oauth2.symbiosis.host/oauth2/auth Requirements A Kubernetes cluster Helm Kubectl NGINX ingress controller (to generate TLS certs for OAuth2 Proxy, optional) Cert-manager What is OAuth2 Proxy? is a reverse proxy that authenticates users for a whole range of different Oauth identity providers, such as Keycloak, , GitHub, OpenID Connect, . OAuth2 Proxy Google and more Different providers have different configurations. What makes this powerful in the case of GitHub is the ability to allow or reject requests based on properties such as GitHub organization membership, team membership, email domain, and others. A running Oauth2 Proxy will expose endpoints for signing in, signing out, callbacks used by the OAuth provider, and more. These endpoints can in turn be used to configure NGINX Ingress to ensure that requests to a specific ingress are only performed by a user that is properly signed in and has the proper authority. However, before we can install and configure OAuth2 Proxy we first have to configure the OAuth provider. Creating an OAuth app on GitHub Sign in to GitHub and browse to , fill in the name and page of your application. Settings → Developer settings → OAuth Apps → New OAuth App The authorization callback URL is the URL that GitHub will redirect to when the authorization is finished. This URL will not be reachable as we haven't configured the OAuth2 Proxy ingress yet. However, it should have the following structure, with replaced by your domain name: example.com https://example.com/oauth2/callback Once the app is created, generate a client secret and save the generated value. We will use this value soon to configure OAuth2 Proxy. Installing OAuth2 Proxy with Helm I've used Helm to install OAuth2 Proxy and found it to be pretty convenient. Check out the for the full list of parameters. Helm chart Below is an example file that permits access to any user that is part of the "example-org" GitHub organization, with TLS certificates generated by cert-manager. values.yaml config: clientID: example-client-id # Replace with your GitHub OAuth app client ID configFile: | provider = "github" scope = "user:email read:org" github_org = "example-org" # Replace with your GitHub org name, or github_team=team_name to limit access to a specific team email_domains = [ "*" ] # Replace with [ "example.com" ] to limit access to users with specific email domain cookie_domains = [ "example.com" ] # Replace with domain names that the proxy is allowed to redirect to after auth, prepend . for wildcards, i.e. ".example.com" whitelist_domains = [ "example.com" ] # Same as above ingress: enabled: true path: /oauth2 pathType: Prefix className: nginx annotations: acme.cert-manager.io/http01-edit-in-place: "true" cert-manager.io/cluster-issuer: letsencrypt # Replace with name of your cert-manager ClusterIssuer kubernetes.io/tls-acme: "true" hosts: - example.com # Replace with host address for the oauth2-proxy ingress tls: - hosts: - example.com # Same as above secretName: oauth2-tls Make sure to replace , and with your GitHub OAuth app client ID, GitHub organization, and domain name. example-client-id example-org example.com You may have to edit the cert-manager annotations based on your own configuration, for example by using the annotation for namespaced certificate issuers. cert-manager.io/issuer We also need to configure a cookie secret that is used by OAuth2 Proxy to encrypt and decrypt user session cookies. Any base64 encoded string is valid, a secure base64 encoded string can be generated using with the following command: openssl openssl rand -base64 32 | head -c 32 | base64 Finally, we can add the OAuth2 Proxy helm repository and install the chart with our values file, cookie, and client secrets: helm repo add oauth2-proxy https://oauth2-proxy.github.io/manifests helm install oauth2-proxy oauth2-proxy/oauth2-proxy --file values.yaml --set config.cookieSecret=example-cookie-secret --set config.clientSecret=example-client-secret oauth2-proxy without cert-manager If you don't have cert-manager installed you can either create the TLS certificate manually or disable TLS altogether: config: clientID: example-client-id # Replace with your GitHub OAuth app client ID configFile: | github_org = "example-org" # Replace with your GitHub org name, or github_team=team_name to limit access to a specific team scope = "user:email read:org" email_domains = [ "*" ] # Replace with [ "example.com" ] to limit access to users with specific email domain provider = "github" cookie_secure = false cookie_domains = [ "example.com" ] # Replace with domain names that the proxy is allowed to redirect to after auth whitelist_domains = [ "example.com" ] # Same as above ingress: enabled: true path: /oauth2 pathType: Prefix className: nginx hosts: - example.com # Replace with host address for the oauth2-proxy ingress Note that you need to change the schema of the callback and auth URLs to , both in the GitHub OAuth configuration and in the annotations below. http:// Configuring our ingress to use OAuth2 Proxy Now that OAuth2 Proxy is up and running, we can configure our ingress to protect our internal services with GitHub OAuth. Add the following annotations to your ingress: nginx.ingress.kubernetes.io/auth-signin: https://example.com/oauth2/start?rd=https://$host$uri nginx.ingress.kubernetes.io/auth-url: https://example.com/oauth2/auth Make sure to replace with your own domain name. example.com The first annotation will redirect unauthenticated requests to the OAuth2 Proxy login page. After the user logs in and authorizes the application, they will be redirected back to the original requested URL thanks to the query parameter that forwards the redirect URL to the proxy. auth-signin rd The second annotation specifies the authentication URL that is used to verify the user's session. auth-url Use the commands below if you don't have a service to expose yet in order to launch a protected hello-world application. Make sure to replace with your domain name. example.com kubectl create deployment web --image=gcr.io/google-samples/hello-app:1.0 kubectl expose deployment web --port=8080 kubectl create ingress web --class nginx '--rule=example.com/=web:8080' \ --annotation 'nginx.ingress.kubernetes.io/auth-signin=https://example.com/oauth2/start?rd=https://$host$uri' \ --annotation 'nginx.ingress.kubernetes.io/auth-url=https://example.com/oauth2/auth' Accessing an endpoint protected by OAuth2 Proxy will redirect you to a GitHub OAuth sign-in page. Make sure to grant access to the organization that you are authenticating against for OAuth2 Proxy to be able to find it and verify your authority. If everything is set up correctly you should be redirected to your protected endpoint after signing in. Limitations Unfortunately, OAuth2 Proxy cannot be configured to allow specific GitHub teams or organizations to access a resource on the ingress level. If you wish to limit access to ingress1 only to team1 while also limiting access to ingress2 only to team2 you will have to configure multiple OAuth2 Proxy instances and GitHub OAuth applications. Your services are still exposed on the internet. A security vulnerability in GitHub OAuth, the OAuth2 Proxy, or the configuration might expose your service to the whole wide world. Protecting internal services behind a private network is always a good idea. Putting it all together To conclude, we've set up a GitHub OAuth app that we've configured with OAuth2 Proxy to allow access to layer 7 ingresses only to the members of our GitHub organization or team. We can use the aforementioned annotations to tag any ingress that we wish to only protect authenticated users. Oauth2-proxy isn't the silver bullet for securing internal services, rather it is an easy way to secure many different services or UIs without much work. It can also act as a first line of defense even for services that are only routable on a private network, or services that offer their own authentication mechanism, like Grafana. You can check out OAuth2 Proxy's for more tips, tricks and config parameters. official documentation Also published here.
