Build a GitOps pipeline with this Kubernetes, Jenkins X tutorial
With GitOps, development and operations teams look to extend the benefits of version-controlled code beyond software development and into infrastructure management.
As part of a CI/CD pipeline, GitOps helps teams maintain a single and centralized source of truth for an application and its resources, spanning from the development stage to deployment. This discipline ensures consistency and stability throughout a DevOps workflow -- even in the most complex and distributed IT environments.
Organizations ramp up with GitOps in different ways, but generally GitOps relies on a continuous integration engine, cloud host and version control tool.
Follow along the video for a Jenkins X tutorial to create a GitOps workflow with this version of the Jenkins CI/CD tool built specifically for cloud and Kubernetes deployments, alongside Google Kubernetes Engine (GKE) and GitHub as a version-control repository.
To get started, download the Jenkins X binaries and command line jx, then start to build a Kubernetes cluster with the jx create cluster command. Since this tutorial uses GKE, specify "gke" as your argument, which tells Jenkins the cloud or on-premises instance on which the cluster is being set up. Continue to enter the required parameters to create the Kubernetes cluster, including the Google Compute Engine zone and machine type, which determines the size of the nodes Kubernetes generates as it scales. Not sure what to specify? Use a question mark to access documentation. After the cloud host spins up this cluster, set up an ingress controller for the kube-system namespace, and add a domain -- which, in this case, is an IP address on the nip.io domain rather than a custom one.
Next, the Jenkins X tutorial walks through the necessary steps to establish an API token for GitHub, so that Jenkins X can make changes to the code repository there. This setup requires at least repository-level permissions on your GitHub account to delegate to the CI/CD server.
Finally, walk through the steps to provision a Jenkins server: Create staging and production environment files in GitHub using Jenkins X commands:
- jx console to open up the Jenkins X console;
- jx create quickstart to create a quickstart -- a premade app -- which is golang-http-test in this tutorial -- and send the generated code to Git and Jenkins;
- jx get applications to view applications, their versions and other information, such as their pod counts and URL; and
- jx promote to move a version of an app along the CI/CD pipeline to a specific environment, such as from staging to production.
Transcript - Build a GitOps pipeline with this Kubernetes, Jenkins X tutorial
Hello. Today we're going to go over how to set up a GitOps deployment workflow using Jenkins X, Google Cloud Platform and Kubernetes.
So first of all, I'm here in my Google Cloud Platform, and I'm going to go ahead and open up a Cloud Shell instance. And let's put that into a new window to make sure we have enough space to work on here. All right, now, first things first, I want to make sure that I'm in my project that I want to do here, which is the "GitOps Demo Project." Perfect. Now, next we're going to go through and get the Jenkins X binaries. Jenkins X is an orchestrator that we're going use to run this whole project, and the command-line tool for it is Jenkins X. They're fairly easy, it's just downloading the command line and then adding it to our bin directory. But after that, we can test and see if it worked by running JX, which is going to tell us we do have a working Jenkins X command line, and we can clear that off now and start working on creating our Kubernetes cluster.
So the command to do that is "jx create cluster," and here it's expecting an argument -- usually the cloud provider or on-premises instance you're setting up in your Kubernetes cluster. Since we are on the Google Kubernetes Engine, we're going to use "gke." And there's a little bit of a gotcha when we're using Google Cloud Platform: We want to make sure we put in the "skip login" flag as well, and that'll save us some headache later on. So let's go ahead and run that. All right, now that it's run a little bit, it's asking us what type of cluster we want to create. We're going to pick "Zonal," for this case. And as far as which zone we want to deploy it to, I'm in the Eastern United States, so we're going to go ahead and pick the "us-east1-cloud zone." And the Google cloud machine type is the size of the nodes that you want Kubernetes to generate when it scales up and scales down. It's worth noting that any one of these options, if you type in a question mark, it will tell you a little bit more about it, in some cases even give you a link to further documentation. For this demo purpose, we're going to go ahead and just use the defaults for everything, including the three minimum nodes and five maximum nodes. We're not going to use Preemtible VMs, or access the Google Cloud Storage or Google Container Registry, and we're not going to enable Kaniko for building container images.
So now we've run through all of those, it'll prompt us a little bit later on for some next steps, but for now, let's just let the cluster create in the background. Okay. Now that that's run for a little while, it's going to ask us if we want to set up an ingress controller for our kube-system namespace. We definitely want to do this. This is how we're going to interact with not only Jenkins X, but our application as well. Next up we're being prompted to add a domain -- this is if we have a custom domain we want to use for our application. If we don't, it will actually give us an IP address on the nip.io domain, which is what I'm going to choose for this. Next we're being prompted for an API token for GitHub, and we're being given a URL to run here. So I'm going to go ahead and do that in another window, and then bring it up here. So this is my personal GitHub account, and this is what that URL redirects to. It's a personal access token that has already got these permissions filled out. And what we need to be able to do is generate a token here and give that to Jenkins X so that it can make modifications to GitHub on our behalf. I'm going to go ahead and generate the token, and then copy it off, and then I will go back to my Jenkins X installation and enter it from there. Okay. I've got my API token and I'm entering it now. So this does mean that in order to set this up you have to have the ability to delegate at least repo level permissions on your GitHub account. And, yes. I'm going to select the defaults and use GitHub as the pipeline's Git server.
Next, when we select Jenkins installation type, we want to pick our static Jenkins server and Jenkins files. All right, as for the default workload build pack, we're going to go with the Kubernetes Workloads: Automated CI+CD with GitOps Promotion. All right, now that all that's been provisioned, it's given us an admin password for our Jenkins instance. We're going to go ahead and copy that off. And next, it's asking us to generate an API token that's logged into the Jenkins server. So let's go ahead and go into "Admin," and enter that password. There we go. So now we have our Jenkins server. Let's go back here and select the organization where we want to create our environment files. We're going to select "devbyaccident," because that is my GitHub account. All right. And it looks like that's been provisioned. Let's go to my GitHub and see if anything's changed. So I'm back in my GitHub account here, and it does have the two environment folders -- there's "Staging" and "Production," with the "Samuraibead," which is the name that GK auto-generated for us.
All right, so let's go back to our Cloud Shell and see what else we could do. Okay. So I'm back on my Cloud Shell window, I'm going to go ahead and type in that command down there. We're going to start a new microservice from a quickstart, so that is "jx create quickstart." And we'll go ahead and use the Git username we set up earlier, the same organization, and we'll go ahead and call this a "golang-http-test." And out of our choices here, we have several that we can pick from, but we will pick our "golang-http" endpoint, and we will go ahead and initialize it and stick with the default message which is "Initial Import." Now we'll let this run for a minute. All right, now that that's done, let's go ahead and see if we can run some of these basic commands. I'll run a "jx console," which will pull up our Jenkins console here. All right, and we see we have our golang-http-test environment, and we have staging and production environments. Let's go ahead and see if we can get the activity for that. I'm going to go back to my Cloud Shell here and run this first pipeline command here. We're actually pending, so let's let that run for a little bit and see if we can come back to it after it's done.
All right. It looks like all our deployments have succeeded, so I've gone ahead and gotten out of there. We can also run a "jx get applications," to see that we do have our golang-http-test version 1 in staging, we have one pod, and we have the URL for it as well. But it's not just enough to put it in staging -- let's actually see if we can get this into production. So, if I run a list directory, I see I have my golang-http-test, let's go ahead and get into that. And then I'll run a "jx promote golang-http-test." All right, so let's break this down a little bit. We have our "jx promote golang-http-test," which is our application, we have the version 0.0.1, which is the version we're going to promote into production, and we want to specify the environment, which is production. So let's go ahead and run that. Okay. So, we've entered the username and password for my GitHub account. We've entered the password for my account, which since I have two-factor authentication on, is a personal access token, so that's something to keep in mind. Let's go ahead and wait for this to finish. Actually, let's do one better, let's go ahead and go into Jenkins and take a look at it. So you see, in my production environment, it's open to pull request, and it's now validating the environment and getting ready to deploy.
All right, and that's it. That's how we run a GitOps workflow from Jenkins X in the Google Cloud Engine. Thanks for watching.