Ingress

The ingress system of kubernetes is specifically designed to route external HTTP and HTTPS traffic into the cluster. It is composed of the ingress resource itself and a ingress controller which implements the needed logic. We decided to deploy the nginx ingress controller by default in every nine Managed GKE cluster. You can control various features by adding annotations to your ingress object.

Details

The nginx ingress controller allows to route external HTTP/HTTPS traffic into the cluster.

Availability

The nginx ingress controller is available as standard with nine Managed GKE.

Usage

The basic usage and structure of a ingress resource is documented in the official kubernetes documentation. To use automatic generated Lets Encrypt certificates for TLS termination please refer to automatic SSL certificates.

DNS Setup

Wildcard DNS domain

We provide an automatic created DNS wildcard "apps" domain for you. It is meant for quick application tests in development. You can use any hostname of that wildcard zone in your ingress resources. DNS is already set up. You will find the application wildcard domain in the Ingress info on runway.

Ingress DNS

We also provide a DNS name which will always point to your nginx ingress controllers IP. You can use it to point your own domain hostnames to nine Managed GKE. It can be found at the same place as your wildcard apps domain in the Ingress info on runway.

To use it just create a CNAME record in your own domain and point it to our provided ingress DNS.

IngressClass

To make use of our ingress controller, you can use the ingressClassName field in your Ingress resource to to nginx. Alternatively you can also omit the field, since it is set as the default class.

Access Logs

The access logs of your Ingress requests can be viewed in your Grafana Instance in the Loki Explore view. The Ingress logs are available under the label app_kubernetes_io_name="ingress-nginx". To only get the logs of a specific Ingress instance, you can filter by using the additional label ingress. The label is in the schema of <namespace>-<ingress-name>-<backend-port>. Here's an example query to get all the logs of the Ingress frontend with the port 80 in the namespace shop-prod:

{app_kubernetes_io_name="ingress-nginx", ingress="shop-prod-frontend-80"}

Additionally the Ingress logs can be filtered by these labels:

• method the HTTP method of a request
• status the HTTP status code of the request

For more information on the usage of Loki, refer to the specific support article.

Nginx ingress features

The nginx ingress controller provides many features like rate limiting, IP white listing, custom default backends, temporary or permanent redirects, etc. All of the annotations which can be used to control those features can be found in the official nginx ingress controller documentation.

Documentation for the most used features can be found below.

Basic authentification

You can add basic authentication to your ingress resource by providing the credentials in a kubernetes secret. Here are some instructional steps:

1. set some env variables for easier processing

   USERNAME=<YOUR USERNAME>
   SECRET_NAMESPACE=<THE NAMESPACE FOR THE SECRET>
   INGRESS_NAMESPACE=<THE NAMESPACE OF YOUR INGRESS RESOURCE>
   INGRESS=<THE NAME OF YOUR INGRESS RESOURCE>

2. create the kubernetes secret which contains the credentials for basic auth. It can also be created in a different namespace than your ingress resource is stored. You will need the mkpasswd tool installed locally (can be found in the whois package in Debian/Ubuntu).

   kubectl create secret generic basic-auth-secret --namespace=$SECRET_NAMESPACE --from-literal=auth=$USERNAME:$(mkpasswd -m sha-512)

3. add some annotations to your ingress object

   kubectl --namespace=$INGRESS_NAMESPACE annotate ingress $INGRESS nginx.ingress.kubernetes.io/auth-type=basic
   kubectl --namespace=$INGRESS_NAMESPACE annotate ingress $INGRESS nginx.ingress.kubernetes.io/auth-secret=$SECRET_NAMESPACE/basic-auth-secret
   kubectl --namespace=$INGRESS_NAMESPACE annotate ingress $INGRESS nginx.ingress.kubernetes.io/auth-realm='Authentication required'

Rate limiting

You have various ways of putting rate limits on your ingresses. All available options are documented in the official nginx ingress docs.

Temporary and persistent redirects

To enable a temporary redirect to another URL for your ingress you can use the following annotation:

nginx.ingress.kubernetes.io/temporal-redirect: <YOUR URL>

The redirect will use the HTTP status code of 302.

If you want to have a permanent redirect you can use:

nginx.ingress.kubernetes.io/permanent-redirect: <YOUR URL>

HTTPS redirect

If TLS is enabled for the given ingress, the nginx ingress controller will automatically redirect to the equivalent HTTPS URL of the ingress. To disable this redirect use:

nginx.ingress.kubernetes.io/ssl-redirect: "false"

IP whitelisting

You can whitelist the IP addresses which are allowed to connect to your ingress resource. You can specify them in CIDR notation in the following annotation:

nginx.ingress.kubernetes.io/whitelist-source-range: <YOUR CIDR RANGE>

Caching

The nginx ingress controllers allows to enable basic caching of backend resources, which can be particularly useful for static content. You can enable caching on specific ingress resources by setting the following annotations in your ingress definition (metadata.annotations):

nginx.ingress.kubernetes.io/proxy-buffering: "on"
nginx.ingress.kubernetes.io/configuration-snippet: |
  proxy_cache static-cache;
  proxy_cache_valid 10m;
  proxy_cache_use_stale error timeout updating http_404 http_500 http_502 http_503 http_504;
  proxy_cache_bypass $http_x_purge;
  add_header X-Cache-Status $upstream_cache_status;

In this example, the cache is invalidated after 10 minutes, and only HTTP status codes 200, 301 and 302 are cached. If another behaviour should be desired, proxy_cache_valid can also take a list of status codes in front of the time:

proxy_cache_valid 404 1m;

would cache 404 responses for one minute. There is also the special code any, which can be specified to cache any responses. Additionally, multiple proxy_cache_valid statements can be added on one ingress to specify different cache times for different status codes.

The default cache size is 100MB. Should you have other requirements, please do not hesitate to get in contact with us.

To check that the caching worked, you can use cURL to inspect the header of the returned resource. For that, you will need to execute the command twice, and at the second request, the resource should be cached:

curl --head <URL>

should print the header x-cache-status: HIT.

If you want to enable caching only for a sub-path of your application (for example an endpoint /static), you will need to create two separate ingress resources. To further configure caching, you can use the proxy_cache options that are valid for location blocks. These options can be found here.

Custom default backend

The default backend is responsible for showing a 404 error page if a request arrives on the nginx ingress controller for which no ingress rule was specified. You can create an own custom default backend (+ kubernetes service) and refer to it on your ingress object.

The default backend only has 2 requirements:

• it needs to serve a 404 page/code on the path /
• it needs to serve a 200 HTTP code on the path /healthz

The implementation of the default nginx ingress controller backend can be found here.

Once you built and deployed your default backend service in the same namespace as your ingress resource you can refer to it via the following annotation on your ingress:

nginx.ingress.kubernetes.io/default-backend: <SERVICE NAME OF YOUR DEFAULT BACKEND>

Additional custom error pages

To be able to additionally display custom error pages on the default backend (for example if the service your ingress points to is not available) you can use the following annotation:

nginx.ingress.kubernetes.io/custom-http-errors: <ERROR CODES> # for example: "404,415,503"

The nginx ingress controller will forward error information via certain HTTP headers to your default backend, which then can return the best possible error representation. More information about this can be found in the official documentation. An example of a default backend which can display custom error pages can be found here.

SLI Probe

You may find a service at sli-probe.apps-customer.<domain>.ninegcp.ch. This service is responsible for monitoring the provided ingress instance from an outside perspective, helping to detect ingress failures as early as possible.

Documentation and Links

• the official kubernetes ingress documentation
• all available ingress annotations of the nginx ingress controller