Skip to main content

Ingress

The ingress system of kubernetes is specifically designed to route external HTTP and HTTPS traffic into the cluster. It is composed of the ingress resource itself and an ingress controller which implements the needed logic. We deploy the HAProxy ingress controller by default in every nine Managed GKE cluster. You can control various features by adding annotations to your ingress object.

Details​

The HAProxy ingress controller allows to route external HTTP/HTTPS traffic into the cluster.

Availability​

The HAProxy ingress controller is available as standard with nine Managed GKE.

warning

The nginx ingress controller is deprecated and will be removed in a future release. Please migrate your Ingress resources to use the HAProxy ingress controller. Refer to the HAProxy Ingress Features section for the available annotations.

Usage​

The basic usage and structure of a ingress resource is documented in the official kubernetes documentation. To use automatic generated Lets Encrypt certificates for TLS termination please refer to automatic TLS certificates.

DNS Setup​

Wildcard DNS domain​

We provide an automatic created DNS wildcard "apps" domain for you. It is meant for quick application tests in development. You can use any hostname of that wildcard zone in your ingress resources. DNS is already set up. You will find the application wildcard domain in the Ingress info on runway.

Ingress DNS​

We also provide a DNS name which will always point to your HAProxy ingress controller's IP. You can use it to point your own domain hostnames to nine Managed GKE. It can be found at the same place as your wildcard apps domain in the Ingress info on runway.

To use it just create a CNAME record in your own domain and point it to our provided ingress DNS.

IngressClass​

To make use of our ingress controller, you can set the ingressClassName field in your Ingress resource to haproxy. Alternatively you can also omit the field, since it is set as the default class.

The deprecated nginx ingress controller is still available using ingressClassName: nginx but will be removed in a future release.

Access Logs​

The access logs of your Ingress requests can be viewed in your Grafana Instance in the Loki Explore view. The Ingress logs are available under the label app_kubernetes_io_name="haproxy-ingress". To only get the logs of a specific Ingress instance, you can filter by using the additional label ingress. The label is in the schema of <namespace>-<ingress-name>-<backend-port>. Here's an example query to get all the logs of the Ingress frontend with the port 80 in the namespace shop-prod:

{app_kubernetes_io_name="haproxy-ingress", ingress="shop-prod-frontend-80"}

Additionally the Ingress logs can be filtered by these labels:

  • method the HTTP method of a request
  • status the HTTP status code of the request

For more information on the usage of Loki, refer to the specific support article.

HAProxy Ingress Features​

The HAProxy ingress controller provides many features like rate limiting, IP whitelisting, temporary or permanent redirects, etc. All of the configuration keys which can be used to control those features can be found in the official HAProxy ingress controller documentation.

Documentation for the most used features can be found below.

Basic authentication​

You can add basic authentication to your ingress resource by providing the credentials in a kubernetes secret. Here are some instructional steps:

  1. set some env variables for easier processing

    USERNAME=<YOUR USERNAME>
    SECRET_NAMESPACE=<THE NAMESPACE FOR THE SECRET>
    INGRESS_NAMESPACE=<THE NAMESPACE OF YOUR INGRESS RESOURCE>
    INGRESS=<THE NAME OF YOUR INGRESS RESOURCE>

  2. create the kubernetes secret which contains the credentials for basic auth. It can also be created in a different namespace than your ingress resource is stored. You will need the mkpasswd tool installed locally (can be found in the whois package in Debian/Ubuntu).

    kubectl create secret generic basic-auth-secret --namespace=$SECRET_NAMESPACE --from-literal=auth=$USERNAME:$(mkpasswd -m sha-512)

  3. add some annotations to your ingress object

    kubectl --namespace=$INGRESS_NAMESPACE annotate ingress $INGRESS haproxy-ingress.github.io/auth-secret=$SECRET_NAMESPACE/basic-auth-secret
    kubectl --namespace=$INGRESS_NAMESPACE annotate ingress $INGRESS haproxy-ingress.github.io/auth-realm='Authentication required'

Rate limiting​

You have various ways of putting rate limits on your ingresses. You can limit requests per second using the haproxy-ingress.github.io/limit-rps annotation or limit concurrent connections using the haproxy-ingress.github.io/limit-connections annotation. All available options are documented in the official HAProxy ingress docs.

Temporary and persistent redirects​

To enable a redirect to another URL for your ingress you can use the following annotation:

haproxy-ingress.github.io/redirect-to: <YOUR URL>

The redirect will use the HTTP status code of 302 (temporary) by default. If you want to change the status code, for example to 301 for a permanent redirect, use:

haproxy-ingress.github.io/redirect-to-code: "301"

HTTPS redirect​

If TLS is enabled for the given ingress, the HAProxy ingress controller will automatically redirect to the equivalent HTTPS URL of the ingress. To disable this redirect use:

haproxy-ingress.github.io/ssl-redirect: "false"

IP whitelisting​

You can whitelist the IP addresses which are allowed to connect to your ingress resource. You can specify them in CIDR notation in the following annotation:

haproxy-ingress.github.io/allowlist-source-range: <YOUR CIDR RANGE>

Custom default backend​

The default backend is responsible for showing a 404 error page if a request arrives on the HAProxy ingress controller for which no ingress rule was specified. You can create your own custom default backend (+ kubernetes service) and refer to it on your ingress object.

The default backend only has 2 requirements:

  • it needs to serve a 404 page/code on the path /
  • it needs to serve a 200 HTTP code on the path /healthz

Once you built and deployed your default backend service in the same namespace as your ingress resource you can refer to it via the following annotation on your ingress:

haproxy-ingress.github.io/default-backend: <SERVICE NAME OF YOUR DEFAULT BACKEND>

SLI Probe​

You may find a service at sli-probe.apps-customer.<domain>.ninegcp.ch. This service is responsible for monitoring the provided ingress instance from an outside perspective, helping to detect ingress failures as early as possible.

Deprecated: Nginx Ingress Controller​

warning

The nginx ingress controller is deprecated and will be removed in a future release. Please migrate your Ingress resources to the HAProxy ingress controller documented above.

Show deprecated nginx ingress documentation

IngressClass​

To use the deprecated nginx ingress controller, set the ingressClassName field in your Ingress resource to nginx.

Access Logs​

The nginx ingress logs are available under the Loki label app_kubernetes_io_name="ingress-nginx". Example query:

{app_kubernetes_io_name="ingress-nginx", ingress="shop-prod-frontend-80"}

Nginx Ingress Features​

All available annotations can be found in the official nginx ingress controller documentation.

Basic authentication​

kubectl --namespace=$INGRESS_NAMESPACE annotate ingress $INGRESS nginx.ingress.kubernetes.io/auth-type=basic
kubectl --namespace=$INGRESS_NAMESPACE annotate ingress $INGRESS nginx.ingress.kubernetes.io/auth-secret=$SECRET_NAMESPACE/basic-auth-secret
kubectl --namespace=$INGRESS_NAMESPACE annotate ingress $INGRESS nginx.ingress.kubernetes.io/auth-realm='Authentication required'

Rate limiting​

All available options are documented in the official nginx ingress docs.

Temporary and persistent redirects​

# Temporary redirect (HTTP 302)
nginx.ingress.kubernetes.io/temporal-redirect: <YOUR URL>
# Permanent redirect (HTTP 301)
nginx.ingress.kubernetes.io/permanent-redirect: <YOUR URL>

HTTPS redirect​

nginx.ingress.kubernetes.io/ssl-redirect: "false"

IP whitelisting​

nginx.ingress.kubernetes.io/whitelist-source-range: <YOUR CIDR RANGE>

Caching​

nginx.ingress.kubernetes.io/proxy-buffering: "on"
nginx.ingress.kubernetes.io/configuration-snippet: |
  proxy_cache static-cache;
  proxy_cache_valid 10m;
  proxy_cache_use_stale error timeout updating http_404 http_500 http_502 http_503 http_504;
  proxy_cache_bypass $http_x_purge;
  add_header X-Cache-Status $upstream_cache_status;

Custom default backend​

nginx.ingress.kubernetes.io/default-backend: <SERVICE NAME OF YOUR DEFAULT BACKEND>

Custom error pages​

nginx.ingress.kubernetes.io/custom-http-errors: <ERROR CODES> # for example: "404,415,503"

More information can be found in the official documentation.