Skip to main content

Static Egress

The static egress feature allows your Pods to have a static, predictable outgoing IP address. This IP address can then be used in remote systems to allow/deny traffic emitted from the selected Pods.

Details​

By default, Pods running in NKE clusters will use the IP address of the node they are running on as source IP when initiating connections to external (out of cluster) systems. As the IP addresses of nodes can change and also because Pods can be scheduled onto different nodes, restricting traffic from Pods at external systems is very hard. The only possibility so far was to allow traffic from all NKE subnets which are in use.

The static egress feature overcomes this limitation. It allows Pods, which have a specific label set, to use a static egress IP. All egress traffic made by these Pods will be routed via one node in the NKE cluster and be emitted with a static source IP. All other Pods (which don't have the specific label set) in the cluster will continue to use the IP of the node they are running on for egressing traffic. This way, traffic from specific Pods in the cluster can be allowed/denied on external systems (firewalls, etc).

If the node which was selected for the static egress traffic is getting replaced during our maintenance window, a new node will be automatically selected and static egress traffic will be routed over the new node. Only nodes from the "nine node pool" will be selected for static egress traffic.

Availability​

The static egress feature is currently available for the following products:

  • NKE clusters
  • vClusters
  • Deploio applications

Usage​

The configuration of the static egress feature differs slightly, depending on the product in use.

NKE clusters​

To enable the feature on NKE clusters, you will first need to enable the static egress runtime on the NKE cluster itself. The runtime will select the egressing node from the "nine" node pool and prepare all network configurations. Please find the corresponding commands for enabling the runtime below. Enabling the runtime is a requirement for static egress to work in NKE clusters.

Enabling the Static Egress Runtime​

  1. Make sure that you have kubectl and nctl installed.

  2. Authenticate with our API using nctl:

    nctl auth login

  3. List all your clusters to find your desired NKE cluster.

    nctl get clusters -A

  4. Once you found the name of your NKE cluster you can use kubectl with the nineapis.ch context to enable the runtime:

    kubectl  --context=nineapis.ch patch kubernetescluster <NAME OF NKE CLUSTER> -n <PROJECT OF CLUSTER> \
    --type=merge -p '{"spec":{"forProvider":{"nke":{"staticEgress":{"enabled":true}}}}}'

Create a Static Egress Resource​

After the runtime got enabled, a static egress resource which targets your cluster can be created. This will result in the automatic creation of a static egress IP and a corresponding Pod label. You can then add this label to all the Pods which should make use of the static egress IP.

  1. Make sure that you followed the API authentication steps from above and have kubectl and nctl installed.

  2. List all your clusters.

    nctl get clusters -A

  3. Create the following static egress resource definition in a static-egress.yaml file and make sure to replace the content of the placeholders with your corresponding values.

    apiVersion: networking.nine.ch/v1alpha1
    kind: StaticEgress
    metadata:
      name: my-static-egress
      namespace: <REPLACE WITH PROJECT OF CLUSTER>
    spec:
      forProvider:
        disabled: false
        target:
          group: infrastructure.nine.ch
          kind: KubernetesCluster
          name: <REPLACE WITH NAME OF CLUSTER>

  4. Use nctl to apply the file to the API.

    nctl apply -f static-egress.yaml

  5. After a few seconds an IP address and a Pod label should be selected.

    $> kubectl --context nineapis.ch get staticegress my-static-egress -n <NAME OF PROJECT> -o yaml
    ...
    status:
        atProvider:
          address: <SELECTED EGRESS IP>
          selectionLabel:
            name: networking.nine.ch/static-egress
            value: <VALUE OF POD LABEL>

Assigning the Static Egress Label to Pods​

Lets assume the static egress configuration selected the label networking.nine.ch/static-egress: production-egress for static egress pod selection. If you already have a Kubernetes Deployment called "production" which deploys your production application, you can attach the label to all running pods of that Deployment by:

  1. creating a file label-patch.yaml with the following content:
spec:
  template:
    metadata:
      labels:
        networking.nine.ch/static-egress: production-egress
  1. and patching your Kubernetes Deployment "production":
kubectl patch deploy production --patch-file=label-patch.yaml

This will rollout new pods which will all have the special static egress label attached. All egressing traffic of those Pods will now make use of the static egress IP.

If you remove the label again, the pods will be using the default node IPs instead.

vClusters​

The usage of static egress for vClusters is very similar to the NKE cluster one. The only difference is that you do not need to enable the static egress runtime (as the runtime gets automatically enabled). You can just create static egress resources and attach the selected Kubernetes label to your Pods as described in the example.

Deploio applications​

The static-egress configuration for Deploio applications is very easy. Just create a static egress resource and your Deploio application will use the selected IP for egress communication.

Create a Static Egress Resource​

  1. Make sure that you have kubectl and nctl installed.

  2. Authenticate with the API using nctl:

    nctl auth login

  3. List all your Deploio applications.

    nctl get apps -A

  4. Create the following static egress resource definition in a deploio-static-egress.yaml file and make sure to replace the content of the placeholders with your corresponding values.

    apiVersion: networking.nine.ch/v1alpha1
    kind: StaticEgress
    metadata:
      name: my-deploio-static-egress
      namespace: <REPLACE WITH PROJECT OF APPLICATION>
    spec:
      forProvider:
        disabled: false
        target:
          group: apps.nine.ch
          kind: Application
          name: <REPLACE WITH NAME OF APPLICATION>

  5. Use nctl to apply the file to the API.

    nctl apply -f deploio-static-egress.yaml

  6. After a few seconds an IP address should be selected and your Deploio application will use it automatically for egress traffic (no further configuration required).

    $> kubectl --context nineapis.ch get staticegress my-deploio-static-egress -n <NAME OF PROJECT> -o yaml
    ...
    status:
        atProvider:
          address: <SELECTED EGRESS IP>

Disable a Static Egress Resource​

You can temporarily disable a static egress resource. Contrary to the deletion of a static egress resource, this will not delete the automatically selected egress IP address which can be reused after you activated the static egress again.

  1. Make sure that you have kubectl and nctl installed.

  2. Authenticate with the API using nctl:

    nctl auth login

  3. List all your static egress resources to select the one which you want to temporarily disable.

    nctl get all --kinds staticegress -A

  4. Patch the static egress resource with kubectl.

    kubectl  --context=nineapis.ch patch staticegress <NAME OF RESOURCE> -n <PROJECT OF RESOURCE> \
    --type=merge -p '{"spec":{"forProvider":{"disabled":true}}}'

Remove a Static Egress Resource​

If you want to remove the static egress feature again, you can just delete the corresponding static egress resource from the API. This will also delete the automatically selected egress IP address.

  1. Make sure that you have kubectl and nctl installed.

  2. Authenticate with the API using nctl:

    nctl auth login

  3. List all your static egress resources to select the one which you want to delete.

    nctl get all --kinds staticegress -A

  4. Delete the static egress resource with kubectl.

    kubectl --context nineapis.ch delete staticegress <NAME OF RESOURCE> -n <PROJECT OF RESOURCE>