Make a Scalable CI/CD Pipeline for Kubernetes With GitHub and Argo CD

Modern software development isn’t just about writing code; it’s about meeting the demands of speed, scalability and reliability in a rapidly changing technological landscape. Users expect frequent updates and seamless performance, and organizations are under pressure to deliver all this without compromising stability.

That’s why CI/CD pipelines have become essential. By automating and streamlining development, testing and deployment, these pipelines not only speed up delivery cycles but also minimize manual errors and boost overall software quality.

If you’re looking to stay ahead, this tutorial is your roadmap to mastering CI/CD in a way that’s practical and highly relevant for today’s cloud native environments. I’ll walk you through building your own scalable CI/CD pipeline using GitHub Actions for continuous integration and Argo CD for continuous deployment.

With this guide, you’ll create an end-to-end solution to automate the build, test and deployment processes for a real-world use case: a Kubernetes-based Node.js REST API that returns the current time. By the end, you won’t just understand CI/CD pipelines; you’ll know how to implement them effectively and take your development workflow to the next level.

Setting Up the Environment

Before diving into the implementation, ensure you have the following:

• Kubernetes cluster: A working Kubernetes cluster (e.g., Minikube, AKS EKS or Google GKE).
• GitHub repository: A repository containing your application’s source code.
• Container registry: Access to a Docker container registry (e.g., Docker Hub or AWS ECR).
• Command-line interface (CLI) tools: kubectl, Docker and Argo CD CLI installed.

This tutorial uses the following real-world example:

• Application: A Node.js REST API that returns the current time. This application was chosen for its simplicity and clarity, making it an ideal example for demonstrating CI/CD workflows. Its lightweight structure allows you to focus on pipeline automation without the complexities of managing a large codebase, while still being representative of real-world deployment scenarios.
• Repository: kubernetes-ci-cd-demo
• Container Registry: Docker Hub (<Your-dockerhub-username>/current-time-api).

To follow along, start by forking the repository and cloning it to your local computer. This will allow you to make changes to the code and push them back to your GitHub account.

Implementing CI With GitHub Actions

Step 1: GitHub Workflow File

The GitHub Actions workflow is defined within the .github/workflows/ directory of your repository. This directory contains the CI configuration files. Inside this directory, you typically create a file named ci-pipeline.yml to define your CI pipeline.

When changes are pushed to the repository, it triggers the GitHub Actions workflow. The workflow consists of a job called build-and-push. This job performs the following tasks:

1. Check out code: It checks out the latest version of the code from the repository.
2. Set up Node.js environment: It configures the Node.js environment required to run the project.
3. Install dependencies: It installs all necessary project dependencies.
4. Run tests: It executes tests to ensure that the code is running correctly and without errors.
5. Log Into Docker hub: It logs into Docker Hub to allow for image building and pushing.
6. Build and push Docker image: It builds a Docker image for the current application (in this case, a time API) and pushes the image to Docker Hub.

This Docker image will later be deployed to a Kubernetes cluster, which is managed by an Argo CD continuous deployment pipeline.

Step 2: Define the CI Pipeline

Here are the contents of the workflow:

Integrating Argo CD for Continuous Deployment

Step 1: Install Argo CD in Your Kubernetes Cluster

Argo CD is a declarative, GitOps continuous delivery tool for Kubernetes. It enables automated deployments and version control of your Kubernetes manifests by syncing them with your Git repository. Installing Argo CD gives you a powerful platform for managing and automating application rollouts and lifecycles within your Kubernetes cluster. To begin, execute the following commands:

To verify that Argo CD was successfully installed on your Kubernetes cluster, run the kubectl command:

kubectl get all -n argocd

The command will display all the resources in the argocd namespace. Here is an example of the results.

Step 2: Access the Argo CD User Interface (UI)

Now that you have Argo CD installed, it’s time to expose the Argo CD UI so that you can access it through the web. To do this, run the following kubectl command:

kubectl port-forward svc/argocd-server -n argocd 8080:443

You should see the following on the terminal. You must keep this terminal running continuously to serve the site.

Access the Argo CD UI using the URL provided for your environment (https://127.0.0.1:8080 in the example above). Note: Your access URL may be different from this, depending on your specific setup and configuration. Make sure to use the correct URL provided for your environment.

This will be the first page you see when you open the URL in your browser.

Step 3: Configure Argo CD

1. Log in to Argo CD:

After launching the Argo CD UI, the next step is to log in for the first time. The default username in Argo CD is admin. To retrieve the password for the admin user, you can run the following command. Ensure that you have the Argo CD CLI installed on your computer before executing this command.

argocd admin initial-password -n argocd

Once you have the password and admin username, you can log in to the Argo CD UI. After logging in, it is recommended to change the default password to one that is easier for you to remember.

2. Add Your GitHub Repository to Argo CD:

The next step is to add your GitHub repository to Argo CD. You can configure it either through the Argo CD CLI or directly via the web UI. I will guide you through the process using the web UI.

1. Navigate to the Settings section in Argo CD.
2. Under the Repositories tab, you will find options to add a new repository.
3. Here, you can configure your GitHub repository by providing the necessary details.

This is what the Repositories section looks like in the Argo CD UI.

If the setup is successful, you should see a green check mark next to your GitHub repository, indicating that it has been connected successfully. If there is an issue, you will see a connection failure error message, indicating that the GitHub repository failed to connect.

The next optional step at this stage is the cluster configuration. By default, the cluster where Argo CD is installed is automatically configured during installation. Any additional Kubernetes clusters should be added through the Cluster Configuration section in the UI.

Keep in mind that most of these configurations can also be done using the Argo CD CLI, so you can choose to use either the CLI or the web UI — both options achieve the same goal.

Step 4: Create an Application in Argo CD

Define an application in a YAML file. This application definition serves as the bridge between your GitHub repository and your Kubernetes cluster within the Argo CD workflow. It tells Argo CD what to deploy, where to deploy it and which configurations to use. The application YAML specifies the repository URL, branch and path for the Kubernetes manifests, as well as the destination cluster and namespace. This ensures that your deployment process is automated and repeatable. Here’s the YAML definition:

To create your first application on the Argo CD platform, use the kubectl apply command at the end of this step. Before running the command, make sure to update the repoURL and server values to match your correct GitHub repository URL and the correct cluster URL.

This command will create the application on Argo CD. Ensure that the repository URL points to your GitHub repository, and the server URL corresponds to the Kubernetes cluster that you want the application to deploy.

kubectl apply -f app.yaml

Step 5: Deploy the Node.js REST API

The last step is pushing the sample Kubernetes manifest to your repository.

This manifest is a YAML configuration file that defines the Kubernetes resources necessary to deploy your application. It specifies a Deployment and outlines the desired state for your pods, such as the number of replicas, container images and port configurations.

By pushing this file to your repository, you enable Argo CD to monitor the repository for changes and automatically deploy or update your application based on the latest configurations. Here’s the example manifest:

Argo CD will automatically detect the changes and deploy the application. Here is an example of how the application looks.

Best Practices for CI/CD With Kubernetes

By leveraging GitHub Actions for CI and Argo CD for CD, you’ve built a robust, scalable pipeline tailored for Kubernetes deployments. This setup streamlines your development process, allowing for faster iterations without sacrificing reliability. The Node.js REST API application serves as a practical example of how this pipeline can be applied in real-world projects.

Here are some best practices for doing CI/CD with Kubernetes:

1. Secrets management: Use tools like HashiCorp Vault or Kubernetes Secrets to manage sensitive information securely. HashiCorp Vault provides robust methods for securing, storing and tightly controlling access to secrets such as API keys, passwords and certificates. Kubernetes Secrets, on the other hand, allow you to store and manage sensitive information directly within the Kubernetes ecosystem. Integrating these tools into your CI/CD pipeline ensures that sensitive data is protected throughout the pipeline’s lifecycle. For example, you can use GitHub Actions to reference these secrets securely without hardcoding them into your repository. This approach minimizes risks and improves compliance with security best practices.
2. Monitoring: Implement observability with tools like Prometheus and Grafana to monitor your pipeline.
3. Testing: Include unit tests and integration tests in your CI pipeline.
4. Rollback strategies: Use Argo CD’s built-in rollback features to handle failed deployments.

Conclusion

From here, you can take it even further. Consider exploring advanced techniques like blue-green deployments or canary releases to push your pipeline’s capabilities to the next level.

To explore how CI/CD tools such as GitHub can transform your projects, read Andela’s step-by-step guide Deploying React on AWS Amplify with Terraform, and bring your seamless cloud-native applications to life.

The post Make a Scalable CI/CD Pipeline for Kubernetes With GitHub and Argo CD appeared first on The New Stack.

Automate the build, test and deployment processes for a Kubernetes-based Node.js REST API, and take your development workflow to the next level.