ArgoCD

Argo CD is a service that allows you to continously deploy applications to NKE clusters by using a gitops workflow.

Details

For customers who need to continously deploy application code, Argo CD provides:

• declarative and version controlled application deployments
• automation and traceability via gitOps workflow
• support for Helm, kustomize and jsonnet application declarations
• a web UI for visualizing kubernetes resources
• webhook integration to fully automate deployments on git operations
• a command line interface application
• audit trails for application events and API calls
• parameter overrides of Helm/ksonnet declarations (simplifies development deployments)
• a grafana metrics dashboard

Availability

Argo CD is available as an optional service for NKE. It can be deployed using Cockpit and can connect with any number of configured NKE clusters.

Usage

Before starting to use Argo CD it is important to understand how a typical workflow should look in the end. Argo CD supports a continous deployment by utilizing a gitops workflow. For that to work it recommends to separate application code from application configuration (Helm charts, kustomize files, etc.). The separation should happen by using 2 different git repositories. Although it is technically possible to use 1 git repository, best practises advise strongly against doing so.

When using 2 separate git repositories, one possible production deployment workflow with Argo CD could look like:

1. a developer creates a pull request/merge request to get some application code changes merged into the master branch
2. after merging of the changes happened (and all tests passed), a tag will be created by the developer signaling that a new productive version of the application should be build
3. a CI pipeline starts. It executes the following steps:

• it builds, tags and pushes a new application container image.
• it creates a commit in the configuration git repository, specifying the new image version to be used (for example by changing the content of the values.yaml in a Helm chart)
• it pushes the commit

1. (optional) a git webhook signalises Argo CD to check for new commits in the configuration repository
2. Argo CD deploys the new image version of the container

Argo CD is not connected to the application source code repository in any way. It only connects to the configuration git repository (read only permissions are sufficient).

If there are any problems with the deployed version, a rollback can be initiated by reverting the commit in the configuration git repository. ArgoCD will then deploy the previous version of the image.

For a further separation of access it is also possible to not directly commit to the configuration git repository within the pipeline. Instead a pull request/merge request will be created which needs to be approved before the new image version should be deployed. With this it is possible to give developers access to the code repository without granting permissions in the configuration repository.

Requirements

To be able to use Argo CD (for production deployments) with a gitops workflow you will need at least the following:

• the URL to your Argo CD installation (see Login)
• a kubernetes namespace where Argo CD can deploy to (see here)
• a git repository with the configuration of your application (called the config repo). This can be:
  • a kustomize application
  • a Helm chart
  • a directory of plain yaml manifests
  • a jsonnet application
• a CI tool/service for:
  • automatically building container images
  • doing changes to the configuration repository (optional)

We at Nine are preferring Helm charts as we are using them in the company ourselves. An example Helm application configuration which deploys a guestbook application can be found in the argo project github namespace in the helm-guestbook directory.

Permissions

The current authorization concept permits all configured NKE users with one of the following roles with full access to all Argo CD applications and projects in their installation of Argo CD:

• admin
• user

Users with the role "view" are only permitted to see configured Argo CD applications and projects, but are not authorized to change them.

Login

Argo CD provides a web user interface as well as a cli application to interact with it.

Web UI

You can find the URL in Cockpit.

You can login with your Cockpit account credentials after clicking on Login via Nine in the web UI.

CLI

The CLI application can be downloaded on the help page in the Argo CD web interface (you will find a link to the help page in the navigation menu on the left side).

To login via the cli application, please follow these steps:

1. execute argocd login <Argo CD URL> --sso locally in a terminal on your machine
2. argocd will open a browser page so that you can enter your Nine Cockpit credentials
3. after a successful authentication you can use argocd locally on the cli as an authenticated user

Argo CD opens a local port (8085 by default) on your machine to be able to authenticate via single sign on. If that port is already in use by another application, please choose a different port by the using the --sso-port argument.

Configuration resources in Argo CD

Argo CD introduces 3 kubernetes resources: Applications, Projects and ApplicationSets.

Applications: The Application CRD is the Kubernetes resource object representing a deployed application instance in an environment. It is defined by two key pieces of information:

• a source reference to the desired state in the configuration Git (repository, revision, path, environment)
• a destination reference to the target cluster and namespace.

It basically describes which configuration state should be deployed to which namespace in your cluster.

Projects: The AppProject CRD is the Kubernetes resource object representing a logical grouping of applications. It is defined by the following key pieces of information:

• a sourceRepos reference to the configuration repositories that applications within the project can pull manifests from
• a destinations reference to clusters and namespaces that applications within the project can deploy into
• a roles list of entities with definitions of their access to resources within the project

ApplicationSets: ApplicationSets allow you to automate the creation of applications and improve multi-cluster support.

ApplicationSets can only be created via the CLI.

Check the official documentation on how to use ApplicationSets.

If you have problems getting ApplicationSets to work, please contact support.

App of Apps principle

Both CRDs (Applications and Projects) can be used in git configuration repositories directly. Specifying the Argo CD applications/projects itself in git (versus creating them via Web UI or cli application) can be used to make use of the "App of Apps Principle".

In a "app of apps" deployment you specify one "root" application in git which points to other ArgoCD application definitions. Those definitions may then either point again to ArgoCD applications or create other Kubernetes resources like Deployments, etc.

Please be aware that your ArgoCD installation is running on an external cluster and not your NKE cluster. When deploying ArgoCD application custom resources you therefore need to use a special target namespace in combination with the 'in-cluster' target server. You can find the value of the special namespace in the "Access information" pane of your ArgoCD instance in cockpit.

Here is an example of a root ArgoCD application which will deploy the guestbook example application from ArgoCD.

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: guestbook
  namespace: <special namespace value>
  finalizers:
    - resources-finalizer.argocd.argoproj.io
spec:
  destination:
    namespace: <special namespace value>
    name: in-cluster
  project: default
  source:
    path: guestbook
    repoURL: https://github.com/argoproj/argocd-example-apps
    targetRevision: HEAD

Deploying your application with Argo CD

Namespace creation

To be able to deploy your application you will need a kubernetes namespace where your application should be deployed to. By default Argo CD does not have the permissions to deploy to any namespace in cluster. You explicitly need to annotate the deployment namespace where your application needs to be deployed with nine.ch/argo-admin="true".

The following example creates the namespace my-application and annotates it to permit access with Argo CD:

kubectl create namespace my-application
kubectl annotate namespace/my-application nine.ch/argo-admin="true"

Argo CD now has permissions to deploy into that namespace:

$> kubectl describe rolebinding namespace-admins -n my-application

Name:         namespace-admins
<...>
Subjects:
  Kind            Name                           Namespace
  ----            ----                           ---------
  ServiceAccount  argocd-c3182374-mcj28sd        default
<...>

Creating an Argo CD application

An Argo CD application basically describes which application configuration should be deployed to which namespace of your cluster.

You should first configure the git configuration repositpory via Argo CD. Afterwards you can use it when configuring the Argo CD application.

You can configure the application and the repository either via the web UI or by using the CLI application. The following documentation describes how to create the application via the web UI.

A note about SSH

Although it is possible to connect your config git repository via SSH, we recommend to use HTTPS. If you want to use SSH you need to add the public host key of your git provider to Argo CD as described in the Argo CD documentation. This is not needed when using the HTTPS protocol. Please make sure that you are using a trusted TLS certificate when using HTTPS.

Steps

Follow these steps to configure a repository via the web UI:

1. Login to the web UI (see Login)
2. Click on the gear icon in the menu on the left ("Manage your repositories,projects,settings")
3. Click on Repositories
4. Click on Connect repo using https
5. You can now enter the repository details

Item	Description	Example
Repository URL	the https URL to your config git repository	https://gitlab.com/example/my-application-config
Username	the username to access the repository (please use access tokens and not personal credentials)	argocd
Password	the password to access the repository (please use access tokens and not personal credentials	3macm32449asdnf243rt
TLS client certificate	an optional TLS client certificate in PEM format which you use for authentication with your git repository
TLS client certificate key	an optional TLS client certificate key in PEM format which you use for authentication with your git repository
skip server verification	check this box if Argo CD should not verify the TLS certificate of your HTTPS connection
Enable LFS support	check this if you used git large file support in your repository

After you registered the git repository you can now configure your Argo CD application

1. Click on New Application (upper left of the screen) on the main page of Argo CD
2. You can now enter the application details

Item	Description	Example
Application Name	the name of your application	my-application
Project	the project your application is part of (see Projects for more information)	default
Sync Policy	choose between Manual or Automatic synchonisation. With Manual synchronisation you will have to trigger a sync manual via Web UI or CLI. Automatic synchronisation will check your git repo every 3 minutes (or immediately when using webhooks from your git provider)	Automatic
Source	the source of the configuration git repository (you should be able to select the created repository from the first step)	https://gitlab.com/example/my-application-config
Revision	this specifies either the branch, tag or commit you want to use in the configuration repository. This can be used to create different environments of your application.	development
Path	please specify . here if all your files (Helm chart, kustomize files, etc) are in the root of the git repository. Otherwise you can specify the sub directory.	.
Cluster	the cluster where to deploy	your-cluster-name
Namespace	the namespace where all the resources should be deployed to (see Namespace creation)	my-application
Type	the type of the configuration (plain yaml files, kustomize, Helm, etc).	Helm
include subdirectories	if subdirectories should also be included

If you use Helm charts as configuration type it is possible to set multiple value.yaml files which will be merged in the given order.

Parameter overrides

Sometimes a separate configuration repository is not really needed or just too much effort. This might be the case in development/testing environments where one wants to:

• have faster iteration cycles
• use upstream Helm charts without forking them into an own git repository

You might also want to set secrets directly in Argo CD without committing them into the configuration git.

For those use cases Argo CD provides so called parameter overrides. Parameter overrides are only possible for applications which use Helm charts or ksonnet configurations.

By overriding parameters (for example in a Helm chart) we are providing configuration information directly to Argo CD, without committing to a configuration repository.

One possible workflow for a Helm configuration application would look like:

1. a developer creates an Argo CD application which uses a upstream Helm chart (either hosted in a git repository or in a Helm chart repository)
2. development of application code happens in a feature branch
3. once the developer pushes new changes to the application code git repository, a CI pipeline starts which executes the following tasks:

• build and push a new container image
• tell Argo CD to use the new container image by using the argocd cli command to set a parameter override in the values.yaml of the upstream Helm chart
• sync the application configuration state by executing argocd app sync for your application

Parameter overrides for an application can be set via the cli application or via the web interface.

Advanced topics

Projects

Projects in Argo CD are used to logical group applications. You can find more information about them in the Argo CD documentation. Due to some restrictions in NKE it is currently not possible to define RBAC rules for projects.

One use case for projects is to create roles. With roles for example you can permit access to Argo CD applications from a CI/CD pipeline (to sync for example), by using the JWT token assigned to a role.

Here is an example to create a role called cicd allowed to sync all applications in the default project:

argocd proj role create default cicd
argocd proj role create-token default cicd # save this token somewhere
argocd proj role add-policy default cicd -a sync -o '*' -p 'allow'

In your pipeline you can then sync applications with

argocd app sync <app name> --auth-token <your auth token>

Webhooks

With webhooks your git provider can immediately notify Argo CD about changes in the configuration git repository. Without webhooks Argo CD will check for new commits every 3 minutes. You need to create the webhook in your git providers settings. The URL and predefined secrets can be found in Cockpit.

Documentation and links

• Argo CD Architecture
• Argo CD documentation for developers