Jenkins

Used as an open-source scheduler ..

Jenkins is an open-source automation server that enables developers to build, test, and deploy software efficiently. It's mainly used for Continuous Integration (CI) and Continuous Delivery (CD), automating the parts of software development related to building, testing, and deploying, facilitating continuous improvement and collaboration.

The following section is for Reference only.

Jenkins + plugins have been installed and configured.

Docker is a platform designed to help developers build, share, and run container applications.

Docker Compose is a tool for defining and running multi-container applications.

Run update & upgrade (optional).

sudo apt update -y && sudo apt upgrade -y

Create a Jenkins folder.

cd
mkdir ~/Jenkins

Create docker-compose.yml

cd
cd ~/Jenkins
nano docker-compose.yml

services:
  jenkins:
    image: jenkins/jenkins:lts
    restart: always
    privileged: true
    user: root
    ports:
      - 9080:8080
      - 50000:50000
    container_name: jenkins-1
    volumes:
      - /home/pentaho/Jenkins/jenkins_configuration:/var/jenkins_home
      - /var/run/docker.sock:/var/run/docker.sock

image          — lts will download latest version.
ports          — defines ports: 9080 (www) and 50000 (api)
container_name — the name of our container: jenkins-1
volumes        — these are the virtual volumes used by the container.

Save.

CTRL + O
Enter
CTRL + X

Create jenkins container.

cd
cd ~/Jenkins
sudo docker compose up -d

[+] Running 13/13
 ✔ jenkins Pulled                                                         10.3s 
   ✔ ca4e5d672725 Pull complete                                            4.1s 
   ✔ 5c2c33065c08 Pull complete                                            6.4s 
   ✔ 92488061783e Pull complete                                            6.6s 
   ✔ 283fc5f22098 Pull complete                                            6.6s 
   ✔ a9c4c37656d4 Pull complete                                            6.7s 
   ✔ ba8c648ced13 Pull complete                                            7.5s 
   ✔ 1cf1dfaee3c9 Pull complete                                            7.5s 
   ✔ 94f0d5472c4d Pull complete                                            7.6s 
   ✔ f8b0d06461dd Pull complete                                            8.5s 
   ✔ 43dde4536f7b Pull complete                                            8.6s 
   ✔ cdaf12c3ce9b Pull complete                                            8.6s 
   ✔ 70d4bec61dab Pull complete                                            8.6s 
[+] Running 2/2
 ✔ Network jenkins_default  Created                                        0.2s 
 ✔ Container jenkins-1        Started                                        3.6s

Verify the logs

sudo docker compose logs --follow

jenkins  | 2024-08-14 09:49:09.249+0000 [id=47]	INFO	jenkins.install.SetupWizard#init: 
jenkins  | 
jenkins  | *************************************************************
jenkins  | *************************************************************
jenkins  | *************************************************************
jenkins  | 
jenkins  | Jenkins initial setup is required. An admin user has been created and a password generated.
jenkins  | Please use the following password to proceed to installation:
jenkins  | 
jenkins  | 3d56d7ddc87c459ebaafe1117f7f56e7
jenkins  | 
jenkins  | This may also be found at: /var/jenkins_home/secrets/initialAdminPassword
jenkins  | 
jenkins  | *************************************************************
jenkins  | *************************************************************
jenkins  | *************************************************************

Make a note of the password: 3d56d7ddc87c459ebaafe1117f7f56e7
You can also retrieve the password with:

cd
sudo cat /var/lib/jenkins/secrets/initialAdminPassword

Configure Jenkins

Access Jenkins UI.

Copy and paste the password.

Click 'Continue' & keep the default option 'Install suggested plugins'.

Create Admin User.

Username

admin

Password

Welcome123!

Full name

system admin

Save & Finish. You are now able to access Jenkins with the default admin account.

This section is for Reference only.

Best Practice is to configure a specific system account - Jenkins

The Jenkins architecture is designed for distributed build environments. It allows us to use different environments for each build project balancing the workload among multiple agents running jobs in parallel.

The Jenkins controller is the original node in the Jenkins installation. The Jenkins controller administers the Jenkins agents and orchestrates their work, including scheduling jobs on agents and monitoring agents. Agents may be connected to the Jenkins controller using either local or cloud computers.

The agents require a Java installation and a network connection to the Jenkins controller.

Generate an SSH key. This key will allow the controller to access the agent via SSH.

cd
ssh-keygen -f ~/.ssh/jenkins_agent

Generating public/private rsa key pair.
Enter passphrase (empty for no passphrase): 
Enter same passphrase again: 
Your identification has been saved in /home/pentaho/.ssh/jenkins_agent
Your public key has been saved in /home/pentaho/.ssh/jenkins_agent.pub
The key fingerprint is:
SHA256:w+logzz6LDa6zZqmBNqbjLBAmF8Tw06vIuBwYFReA1U pentaho@pentaho
The key's randomart image is:
+---[RSA 3072]----+
| ..oo+.E         |
|. ... .          |
|.. .=            |
|oo o + . .       |
|*.. + . S        |
|B+ o + o .       |
|=o+ = + .        |
|+X+* o .         |
|@=B+o            |
+----[SHA256]-----+

This command creates two files:

jenkins_agent - private key.

jenkins-agent.pub - public key.

Navigate to your Jenkins dashboard.

Go to Manage Jenkins option in main menu and click on Credentials button.

From the drop down select: Add Credentials

Fill in the form:

Kind

SSH Username with private key

jenkins

Description

jenkins ssh key

Username

jenkins - jenkins' user already exists by default in the jenkins_agent-1 container.

Private Key

select Enter directly and press the Add button to insert the content of your private key file at ~/.ssh/jenkins_agent

Paraphrase

fill your passphrase used to generate the SSH key pair.

Click on 'Create'.

Jenkins Agent on Docker

Run the command to start your first agent.

docker run -d --rm --name=jenkins_agent-1 -p 22:22 \
-e "JENKINS_AGENT_SSH_PUBKEY=[your-public-key]" \
jenkins/ssh-agent:alpine-jdk17

• Replace the tag [your-public-key] for your own SSH public key.

• Your public key value in this example could be found by issuing : cat ~/.ssh/jenkins_agent_key.pub on the machine your created it. Do not add the square brackets [] around the key value

• The value of [your-public-key] MUST include the full contents of your .pub file, including the ssh-XXXX prefix.

• Ex: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAAQQCo9+BpMRYQ/dL3DS2CyJxRF+j6ctbT3/Qp84+KeFhnii7NT7fELilKUSnxS30WAvQCCo2yU1orfgqr41mM70MB

• If your machine already has a ssh server running on the 22 port (if you logged onto this machine thanks to the ssh command, that’s the case), you should use another port for the docker command, such as -p 4444:22

or you can modify the docker-compose.yml to include the agent and public key.

Bring the jenkins-1 container down.

cd
cd ~/Jenkins
docker-compose down

Edit the docker-compose.yml

cd
cd ~/Jenkins
nano docker-compose.yml

Add the ~/.ssh/jenkins_agent.pub - the public key that will authentiacte the connection to the agent.

services:
  jenkins:
    image: jenkins/jenkins:lts
    restart: always
    privileged: true
    user: root
    ports:
      - 9080:8080
      - 50000:50000
    container_name: jenkins-1
    volumes:
      - /home/pentaho/Jenkins/jenkins_configuration:/var/jenkins_home
      - /var/run/docker.sock:/var/run/docker.sock
  agent:
    image: jenkins/ssh-agent:jdk17
    privileged: true
    user: root
    container_name: jenkins_agent-1
    expose:
      - 22
    environment:
      - JENKINS_AGENT_SSH_PUBKEY=ssh-rsa
AAAAB3NzaC1yc2EAAAADAQABAAABgQCfQR3NKeoP+OkRbXqDmltwfxq6vYYrjFP7+kiKHrqB/xIuHbrcWGrRpb65qaPXstzxZLopiN1lkdn0kCeJAHwe38tHB+CJESqjvGemq0hRPAY/YGT1DqyP0l5IQAck6NGRC6VM7Gto/vGTgJQF5eNh+hPa+4wfTvkzfwy5gB5rJN9zfqW08uYNurqOJCHoCBb0FnyRVCO3WzWReib+A5YVP880jOLLR3jhyNkJEjsDH1yFPPc5sS11bHcK9yl8NZeVsi9KpPFCOYSuZCmBai+uXXX+vMTwQ95k4qUMorQqV8ctZ1U8b7PlcuBlv0OSz+LyxOBELPxHYuoRGn4T5bWiQCUakSzaTmsstsyRFQXL5tG+TZwGxzvsbyGNUY7ybvh08tr4a8XowkctrdLEJKKr3n+Meag1FpHwEI0c+X96hL3RyC6+pAOTeW1ZbljydNTW59LM47nzrqmGn/mZF6PHSlmjfy9IUWZuHT1E0pTfq+ro+XNfNHcnhnqzIu7v5OM= pentaho@pentaho

[+] Running 9/9
 ✔ agent Pulled                                                            6.6s 
   ✔ ca4e5d672725 Already exists                                           0.0s 
   ✔ e5f960337628 Pull complete                                            0.4s 
   ✔ a05e468af172 Pull complete                                            4.0s 
   ✔ e60e3fc02759 Pull complete                                            4.0s 
   ✔ 4f4fb700ef54 Pull complete                                            4.0s 
   ✔ dee10b008112 Pull complete                                            4.9s 
   ✔ 670563ebccf3 Pull complete                                            4.9s 
   ✔ 1b4075dda320 Pull complete                                            5.0s 
[+] Running 3/3
 ✔ Network jenkins_default    Created                                      0.1s 
 ✔ Container jenkins          Started                                      1.5s 
 ✔ Container jenkins_agent-1  Started                                      1.5s

Check that the Jenkins Docker Agent is up and running.

Configure the Agent

This is a very basic setup - everything is installed on a single server, i.e. from the master node we can deploy a new slave node & run our Jobs using the deployed Agent.

Navigate to your Jenkins dashboard.
Log in as user: Jenkins

Go to Manage Jenkins option in main menu and click on Nodes button.

Or click on the Set Up agent message.

Click on New Node button.

Enter the following details:

Let's run through a couple of Jenkins scenarios:

Periodic Build - demonstrates the first stages of the CI/CD pipeline. Developers commit to a Github repository which, periodically, triggers a 'build'.

The 'build' executes a shell 'PUT' script which uploads the Jobs & Transformations into the Pentaho Repository.

Commit Build - a Github commit triggers a 'build'.

You could simply set a CRON schedule to trigger the 'build' process ..

Log into Jenkins.

Username

admin

Password

Welcome123!

Click on 'Create a Job' or 'New Item'.
Provide a name - Periodic Build - for the item and then choose: 'Freestyle project' & click: 'OK'

Set the following project options - execute every 5 mins:

Section

Option

Value

Source Management

Git

Repository URL

https://github.com/jporeilly/Jenkins-Github

Branches to build

Branch Specifier

Build Triggers

Build periodically

Schedule

H/5 * * * *

Build Steps

Execute shell

Command

curl -H "Content-Type: application/xml" -d @Job1.kjb -X PUT -u admin:password -i http://pentaho.pentaho.lab:8080/pentaho/api/repo/files/:public:Jenkins:Job1.kjb

curl -H "Content-Type: application/xml" -d @Transformation1.ktr -X PUT -u admin:password -i http://pentaho.pentaho.lab:8080/pentaho/api/repo/files/:public:Jenkins:Transformation1.ktr

curl -H "Content-Type: application/xml" -d @Transformation2.ktr -X PUT -u admin:password -i http://pentaho.pentaho.lab:8080/pentaho/api/repo/files/:public:Jenkins:Transformation2.ktr

After specifying the Repository URL, and the error "Failed to connect to repository..." is displayed - indicates Git has not been installed.

If the Repository is private, Jenkins wil validate the Github credentials when pulling the source code.

Source Code Management

Build Triggers

Build Steps

Before you click Save, just check:

• Pentaho Server is up and running.

• Create a Public/Jenkins folder & is empty.

cd
cd/opt/pentaho/server/pentaho-server/
sudo ./start-pentaho.sh

Check the 'Build Status' in the Dashboard.

Click on a build

View the 'Console Ouput'.

Finally .. check the Pentaho Repository.

In CI/CD we want to trigger the build once we commit our code to git.

There are at least two approaches to achieving this:

You could have Jenkins continuously polling the git repository for changes or more efficiently, have Github tell Jenkins about any commits that happen.

Log into Jenkins.

Username

admin

Password

Welcome123!

Click on 'New Item'.

Provide a name - Commit Build - for the item and then choose: 'Freestyle project' & click: 'OK'

Set the following project options - Git commit:

Section

Option

Value

Source Management

Git

Repository URL

https://github.com/jporeilly/Jenkins-Github

Branches to build

Branch Specifier

Build Triggers

Build periodically

Schedule

H/5 * * * *

Build

Execute shell

Command

curl -H "Content-Type: application/xml" -d @Job1.kjb -X PUT -u admin:password -i http://pentaho.pentaho.lab:8080/pentaho/api/repo/files/:public:Jenkins:Job1.kjb

curl -H "Content-Type: application/xml" -d @Transformation1.ktr -X PUT -u admin:password -i http://pentaho.pentaho.lab:8080/pentaho/api/repo/files/:public:Jenkins:Transformation1.ktr

curl -H "Content-Type: application/xml" -d @Transformation2.ktr -X PUT -u admin:password -i http://pentaho.pentaho.lab:8080/pentaho/api/repo/files/:public:Jenkins:Transformation2.ktr

After specifying the Repository URL, and the error "Failed to connect to repository..." is displayed - indicates Git has not been installed.

If the Repository is private, Jenkins wil validate the Github credentials when pulling the source code.

PreviousRESTful API NextGenAI

Last updated 10 days ago