jenkins整合碼云和阿里云私有鏡像，k8s部署和可視化

1. cicd流水線

版本

Docker 20.10.7

Harbor 2.3.0

https://github.com/goharbor/harbor/releases/tag/v2.3.0

Jenkins 2.289.1

Redhat Jenkins Packages

https://archives.jenkins-ci.org/windows-stable/2.289.1/

GitLab 14.3.2

https://packages.gitlab.com/app/gitlab/gitlab-ce/search?q=14.3.2&filter=all&filter=all&dist=

單節(jié)點服務(wù)器要求:

2核，8g，40gb

或者

Docker 20.10.7

Jenkins 2.289.1

阿里云私人鏡像庫

碼云

1.1 docker安裝

yum install -y yum-utils device-mapper-persistent-data lvm2

yum-config-manager --add-repo http://mirrors.aliyun.com/dockerce/linux/centos/docker-ce.repo

yum makecache fast

sudo yum install -y yum-utils
sudo yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

yum list docker-ce --showduplicates | sort -r


sudo yum install -y docker-ce-20.10.7 docker-ce-cli-20.10.7 containerd.io


sudo systemctl start docker
sudo systemctl enable docker

sudo docker --version

關(guān)閉docker

sudo systemctl stop docker.socket
sudo systemctl stop docker.service


sudo systemctl status docker

設(shè)置鏡像庫

mkdir -p /etc/docker

tee /etc/docker/daemon.json <<-'EOF'
{
  "registry-mirrors": [
    "https://dockerpull.com",
    "https://dockerproxy.net",
    "https://docker.m.daocloud.io",
    "https://docker.ketches.cn",
    "https://do.nark.eu.org"
  ]
}
EOF

systemctl daemon-reload
systemctl restart docker

systemctl enable docker

docker -v
docker version
docker info

1.2 Harbor(HTTPS)安裝(如果服務(wù)器頂不住，可以換成阿里云私人鏡像倉庫)

解壓harbor

echo "192.168.49.159 harbor.lagouedu.com" >> /etc/hosts

cat /etc/hosts

cd /data

tar zxf harbor-offline-installer-v2.3.0.tgz

cd harbor/

mkdir -p ssl

cd ssl

獲得證書頒發(fā)機構(gòu)

在生產(chǎn)環(huán)境中，應(yīng)該從CA官方獲取證書。在測試或開發(fā)環(huán)境中，可以生成自己的CA。若要生成CA證
書，請運行以下命令。

cd /data/harbor/ssl

創(chuàng)建CA根證書
openssl genrsa -out ca.key 4096


openssl req -x509 -new -nodes -sha512 -days 3650 -subj "/C=TW/ST=Beijing/L=Beijing/O=example/OU=Personal/CN=harbor.lagouedu.com" -key ca.key -out ca.crt

獲取服務(wù)器證書
openssl genrsa -out harbor.lagouedu.com.key 4096

openssl req -sha512 -new -subj \
"/C=TW/ST=Beijing/L=Beijing/O=example/OU=Personal/CN=harbor.lagouedu.com" \
-key harbor.lagouedu.com.key -out harbor.lagouedu.com.csr

cat > v3.ext <<-EOF
authorityKeyIdentifier=keyid,issuer
basicConstraints=CA:FALSE
keyUsage = digitalSignature, nonRepudiation, keyEncipherment, dataEncipherment
extendedKeyUsage = serverAuth
subjectAltName = @alt_names
[alt_names]
DNS.1=harbor.lagouedu.com
EOF


openssl x509 -req -sha512 -days 3650 -extfile v3.ext -CA ca.crt -CAkey ca.key \
-CAcreateserial -in harbor.lagouedu.com.csr -out harbor.lagouedu.com.crt


openssl x509 -inform PEM -in harbor.lagouedu.com.crt -out harbor.lagouedu.com.cert

mkdir -p /etc/docker/certs.d/harbor.lagouedu.com/

cp harbor.lagouedu.com.cert /etc/docker/certs.d/harbor.lagouedu.com/
cp harbor.lagouedu.com.key /etc/docker/certs.d/harbor.lagouedu.com/
cp ca.crt /etc/docker/certs.d/harbor.lagouedu.com/

systemctl daemon-reload
systemctl restart docker

vi harbor.yml

注釋掉http的配置信息

hostname: harbor.lagouedu.com
https:
port: 443
certificate: /data/harbor/ssl/harbor.lagouedu.com.crt
private_key: /data/harbor/ssl/harbor.lagouedu.com.key

安裝harbor

docker pull goharbor/prepare:v2.3.0
cd /data/harbor
./prepared
./install.sh

訪問UI

C:\Windows\System32\drivers\etc
192.168.198.101 harbor.lagouedu.com
https://harbor.lagouedu.com/

上傳鏡像

docker-100服務(wù)器：
將harbor服務(wù)器端生成的ca.crt文件復(fù)制到/etc/pki/ca-trust/source/anchors/中。
執(zhí)行命令更新ca證書授權(quán)：update-ca-trust
重啟docker服務(wù)：
systemctl restart docker
echo "192.168.49.159 harbor.lagouedu.com" >> /etc/hosts
docker login harbor.lagouedu.com
admin
Harbor12345

docker load -i nginx.1.19.3.alpine.tar
docker tag nginx:1.19.3-alpine harbor.lagouedu.com/lagouedu/nginx:v1
docker push harbor.lagouedu.com/lagouedu/nginx:v1

1.3 Harbor(HTTP)安裝(如果服務(wù)器頂不住，可以換成阿里云私人鏡像倉庫)

解壓harbor

echo "192.168.49.159 harbor.lagouedu.com" >> /etc/hosts

cat /etc/hosts

cd /data

tar zxf harbor-offline-installer-v2.3.0.tgz

cd harbor/

修改配置文件
vi harbor.yml


修改私服鏡像地址
hostname: 192.168.49.159
修改鏡像地址訪問端口號
port: 5000
harbor管理員登錄系統(tǒng)密碼
harbor_admin_password: Harbor12345
修改harbor映射卷目錄
data_volume: /data/harbor


安裝harbor
執(zhí)行啟動腳本,經(jīng)過下述3個步驟后，成功安裝harbor私服
./install.sh
準備安裝環(huán)境：檢查docker版本和docker-compose版本
加載harbor需要的鏡像
準備編譯環(huán)境
啟動harbor。通過docker-compose方式啟動服務(wù)

google瀏覽器訪問harbor私服

http://192.168.49.159:5000
username: admin
password: Harbor12345

啟動，關(guān)閉harbor

啟動
docker-compose up -d
關(guān)閉
docker-compose down

配置harbor私服

jenkinsagent-154服務(wù)器配置docker登錄harbor私服信息。
配置私服

vi /etc/docker/daemon.json

"insecure-registries":["192.168.49.159:5000"]

重啟docker服務(wù)：

systemctl daemon-reload
systemctl restart docker

1.4 GitLab安裝(如果服務(wù)器頂不住，可以換成碼云或者GitHub)

安裝

yum -y install policycoreutils openssh-server openssh-clients postfix

可以選擇下載成rpm包到指定路徑，不安裝

yum install --downloadonly --downloaddir=/path/to/download policycoreutils openssh-server openssh-clients postfix

systemctl enable sshd && sudo systemctl start sshd
systemctl enable postfix && systemctl start postfix

rpm -i gitlab-ce-14.3.2-ce.0.el7.x86_64.rpm

vim  /etc/gitlab/gitlab.rb

修改gitlab訪問地址和端口，默認為80，我們不進行修改。
external_url 'http://192.168.66.152'
//external_url 'http://<你的服務(wù)器地址或域名>:11000'

gitlab-ctl reconfigure
gitlab-ctl restart

啟動和停止

#啟動服務(wù)
# gitlab-ctl start
#停止服務(wù)
# gitlab-ctl stop
#重啟服務(wù)
# gitlab-ctl restart
#狀態(tài)
#gitlab-ctl status
#監(jiān)控
#gitlab-ctl  tailunicorn 監(jiān)控unicorn日志
#gitlab-ctl  tail

登錄gitlab

登錄gitlab：用戶名默認為root。第一次登錄需要設(shè)置密碼。本教程將密碼設(shè)置為12345678

username：root
password:12345678

1.5 Jenkins安裝(Linux)

https://www.oracle.com/java/technologies/downloads/#java8

安裝jenkins，jdk，git，maven

需要在jenkins上安裝相應(yīng)的jdk，git，maven

sudo yum install -y curl-devel expat-devel gettext-devel openssl-devel zlib-devel gcc perl-ExtUtils-MakeMaker

sudo yum -y remove git

sudo yum install wget

wget https://download.oracle.com/otn/java/jdk/8u261-b12/a4634525489241b9a9e1aa73d9e118e6/jdk-8u421-linux-x64.tar.gz

wget https://repo.maven.apache.org/maven2/org/apache/maven/apache-maven/3.6.3/apache-maven-3.6.3-bin.tar.gz

wget https://www.kernel.org/pub/software/scm/git/git-2.28.0.tar.gz

tar -zxvf jdk-8u421-linux-x64.tar.gz -C /opt

tar -zxf apache-maven-3.6.3-bin.tar.gz -C /opt
mv /opt/apache-maven-3.6.3 /opt/maven

tar -zxvf git-2.28.0.tar.gz

cd git-2.28.0
./configure --prefix=/opt/git
make && sudo make install

vi /etc/profile
export PATH
export JAVA_HOME=/opt/jdk1.8.0_421
export PATH=$JAVA_HOME/bin:$JAVA_HOME/jre/bin:$PATH
export CLASSPATH=.:$JAVA_HOME/lib:$JAVA_HOME/jre/lib
export MAVEN_HOME=/opt/maven
export PATH=$MAVEN_HOME/bin:$PATH
export PATH=$PATH:/opt/git/bin

source /etc/profile

java -version

mvn -version

git --version

ln -s /opt/jdk1.8.0_421/bin/java /usr/bin/
ln -s /opt/maven/bin/mvn /usr/bin/
ln -s /opt/git/bin/git /usr/bin

mkdir -p /data/maven/repository

c

設(shè)置本地倉庫目錄
<localRepository>/data/maven/repository</localRepository>

<mirror>
    <id>nexus-aliyun</id>
    <mirrorOf>*</mirrorOf>
    <name>Nexus aliyun</name>
    <url>http://maven.aliyun.com/nexus/content/groups/public</url>
</mirror>


maven工程JDK8編譯配置


<profile>
    <id>jdk-1.8</id>
    <activation>
        <activeByDefault>true</activeByDefault>
        <jdk>1.8</jdk>
    </activation>
    <properties>
        <maven.compiler.source>1.8</maven.compiler.source>
        <maven.compiler.target>1.8</maven.compiler.target>
        <maven.compiler.compilerVersion>1.8</maven.compiler.compilerVersion>
    </properties>
</profile>

需要開啟全局代理，部分命令無法開啟，也可以使用 proxychains 配置 Docker 和 Jenkins 等軟件的命令

地址是windows宿主機代理的網(wǎng)絡(luò)地址和可訪問ip

sudo vi /etc/profile

export http_proxy="http://root:root@192.168.1.104:9530"
export https_proxy="http://root:root@192.168.1.104:9530"
export ftp_proxy="http://root:root@192.168.1.104:9530"


source /etc/profile

curl http://www.google.com
wget http://www.google.com

上傳centos服務(wù)器進行安裝：
rpm -ivh jenkins-2.289.1-1.1.noarch.rpm

whereis jenkins

vi /etc/init.d/jenkins

/opt/jdk1.8.0_421/bin/java

修改Jenkins配置

vi /etc/sysconfig/jenkins

修改內(nèi)容如下：

JENKINS_USER="root"
JENKINS_PORT="8888"

JENKINS_JAVA_OPTIONS="-Dhudson.model.DownloadService.noSignatureCheck=true"

修改Jenkins插件安裝版本為當(dāng)前版本

或者在圖形界面上設(shè)置，參照

https://www.haxi.cc/2022/11/

https://mirrors.jenkins.io/updates/dynamic-stable-2.289.1/update-center.json?skipSignatureCheck=true

vi /var/lib/jenkins/hudson.model.UpdateCenter.xml

編輯文件：打開文件并將 <url> 字段替換為新地址：

<sites>
  <site>
    <id>default</id>
    <url>https://mirrors.jenkins.io/updates/dynamic-stable-2.289.1/update-center.json</url>
  </site>
</sites>

啟動Jenkins服務(wù)

sudo systemctl daemon-reload

systemctl start jenkins
systemctl enable jenkins

停止jenkins

systemctl stop jenkins

打開瀏覽器訪問
http://192.168.49.159:8888

獲取密碼

獲取并輸入admin賬戶密碼
cat /var/lib/jenkins/secrets/initialAdminPassword

root

插件安裝

瀏覽器訪問
http://192.168.49.159:8888/jnlpJars/jenkins-cli.jar

在界面開啟

java -jar /opt/jenkins-cli.jar -s http://192.168.49.159:8888 \
install-plugin workflow-aggregator pipeline-stage-step docker-plugin docker-workflow \
credentials credentials-binding git gitlab-plugin \
gitee mailer email-ext timestamper matrix-auth thinBackup \
ssh-slaves build-timeout  generic-webhook-trigger

tail -n 100 /var/log/jenkins/jenkins.log

日志最后沒有繼續(xù)輸出后，重啟Jenkins

systemctl restart jenkins

1.6 Jenkins安裝(windows)

https://www.jenkins.io/download/thank-you-downloading-windows-installer-stable/

https://www.yangshaofeng.com/home/Detail?id=3364CF3F300A4E0E8A21D2FA26848176

需要提前在windows上安裝好jdk,maven,git，并開啟vpn

安裝jenkins.msi包

進入安裝目錄，修改jenkins.xml 配置文件

訪問jenkins

http://localhost:8080

選擇默認插件安裝

1.7 碼云和阿里云鏡像倉庫創(chuàng)建

碼云倉庫

登錄碼云，新建git倉庫

私人鏡像庫

1.首先登陸阿里云，進入控制臺，打開主菜單，找到“容器鏡像服務(wù)”

2.倉庫實例有個人版和企業(yè)版，個人版是免費，企業(yè)版收費

3.進入個人實例，可以看到這個倉庫的摘要信息

4.進入倉庫，這個時候鏡像為空，你可以自己push上來docker鏡像

5.可以給鏡像用命名空間進行分類，個人版命名空間限制3個也夠用了

6.在本機把我自己的進項上傳上來，命令很簡單，主要docker login 和docker push

7.還可以查看鏡像的詳情

1.8 腳本編寫

點進去

編寫pipeline腳本

碼云用戶名和密碼

新增憑據(jù)

gitee的api令牌:

05bf6dbb7ea13e272c6d05a89e4dff5d

進入片段生成器，

生成groovy語句

將生成的拉取語句放到流水線腳本中

(gitlab)版本

設(shè)置git提交，自動觸發(fā)Jenkinsflie執(zhí)行，jenkinsfile在根目錄下

(gitee)版本

1.9 推送基礎(chǔ)鏡像到阿里云

docker pull openjdk:8-alpine3.9

docker login registry.cn-hangzhou.aliyuncs.com -u aliyun0291282216 -p Citygis@1613

docker tag openjdk:8-alpine3.9 registry.cn-hangzhou.aliyuncs.com/dddd56656/openjdk:8-alpine3.9

docker push registry.cn-hangzhou.aliyuncs.com/dddd56656/openjdk:8-alpine3.9

1.10 測試能否成功打包

mvn clean package -Dmaven.test.skip=true jib:build -DsendCredentialsOverHttp=true

1.11 測算代碼提交觸發(fā)容器運行

刪除多余路徑，修改為指定路徑

提交代碼，觸發(fā)jenkins執(zhí)行pipeline

2. k8s部署和編排

2.1 k8s部署

master節(jié)點至少2核2G

不使用rancher,rancher更新跟不上進度

Kubernetes Dashboard v2.8.2

sealos_5.0.0-beta4_linux_amd64.tar.gz
kubernetes 1.27.10

https://github.com/labring/sealos/releases/tag/v5.0.0-beta4

https://github.com/kubernetes/kubernetes/tags?after=v1.30.0-alpha.3

新增三臺節(jié)點

192.168.49.159 linux159(原docker的cicd服務(wù)器)

新增的三臺k8s節(jié)點

192.168.49.160 linux160
192.168.49.161 linux161
192.168.49.162 linux162

設(shè)置集群

systemctl stop firewalld && systemctl disable firewalld
systemctl stop NetworkManager && systemctl disable NetworkManager

setenforce 0
sed -i s/SELINUX=enforcing/SELINUX=disabled/ /etc/selinux/config

swapoff -a
sed -ri 's/.swap./#&/' /etc/fstab

yum install chrony -y
systemctl enable chronyd --now
chronyc sources

升級內(nèi)核(舊版本只能手動下載)

yum install -y wget

wget http://mirrors.coreix.net/elrepo-archive-archive/kernel/el7/x86_64/RPMS/kernel-lt-devel-5.4.203-1.el7.elrepo.x86_64.rpm
wget http://mirrors.coreix.net/elrepo-archive-archive/kernel/el7/x86_64/RPMS/kernel-lt-headers-5.4.203-1.el7.elrepo.x86_64.rpm
wget http://mirrors.coreix.net/elrepo-archive-archive/kernel/el7/x86_64/RPMS/kernel-lt-5.4.203-1.el7.elrepo.x86_64.rpm

yum install -y perl

rpm -Uvh *.rpm

rpm -qa | grep kernel

awk -F' '$1=="menuentry " {print $2}' /etc/grub2.cfg

grub2-set-default 0

所有服務(wù)器都要

拿sealos_5.0.0-beta4_linux_amd64.tar.gz里面的sealos，授權(quán)并移動到/usr/bin目錄中

cd /opt

chmod +x sealos && mv sealos /usr/bin

單master多node:

sealos所在服務(wù)器需要

sealos pull registry.cn-shanghai.aliyuncs.com/labring/kubernetes:v1.27.10 registry.cn-shanghai.aliyuncs.com/labring/helm:v3.9.4 registry.cn-shanghai.aliyuncs.com/labring/cilium:v1.14.7

sealos run --force registry.cn-shanghai.aliyuncs.com/labring/kubernetes:v1.27.10 registry.cn-shanghai.aliyuncs.com/labring/helm:v3.9.4 registry.cn-shanghai.aliyuncs.com/labring/cilium:v1.14.7
--masters 192.168.49.160
--nodes 192.168.49.161,192.168.49.162 -u root -p 123456

如果失敗: sealos reset --force

2.2 dashboard安裝和連接k8s

Kubernetes Dashboard v2.7.0

helm repo add kubernetes-dashboard https://kubernetes.github.io/dashboard/
helm repo update

helm show chart kubernetes-dashboard/kubernetes-dashboard

本地環(huán)境

卸載

helm uninstall kubernetes-dashboard --namespace kube-system

helm install kubernetes-dashboard kubernetes-dashboard/kubernetes-dashboard \
  --version 6.0.8 \
  --namespace kube-system \
  --set replicaCount=1 \
  --set service.port=443 \
  --set service.targetPort=8443 \
  --set service.type=NodePort \
  --set service.nodePort=30001 \
  --set image.repository=dockerproxy.net/kubernetesui/dashboard \--set image.name=dashboard \
  --set image.tag=v2.7.0 \
  --set image.pullPolicy=IfNotPresent

kubectl get pods -n kube-system

kubectl describe pod kubernetes-dashboard-5948b5f5d7-whlqw -n kube-system

kubectl get svc -n kube-system

kubectl create sa dashboard -n kube-system
kubectl create clusterrolebinding dashboard-cluster-admin \
  --clusterrole=cluster-admin \
  --serviceaccount=kube-system:dashboard

echo "
apiVersion: v1
kind: Secret
metadata:
  name: dashboard-sec
  namespace: kube-system
  annotations:
    kubernetes.io/service-account.name: \"dashboard\"
type: kubernetes.io/service-account-token
" > dashboard-sec.yaml

kubectl apply -f dashboard-sec.yaml


kubectl get secret -n kube-system


kubectl describe secret/dashboard-sec -n kube-system | tail -n 1

2.1 dashboard使用

創(chuàng)建命名空間

posted @ 2025-01-14 10:37 $Traitor$ 閱讀(9) 評論(0) 收藏舉報

刷新頁面返回頂部

jenkins整合碼云和阿里云私有鏡像，k8s部署和可視化

1. cicd流水線

版本

1.1 docker安裝

1.2 Harbor(HTTPS)安裝(如果服務(wù)器頂不住，可以換成阿里云私人鏡像倉庫)

解壓harbor

獲得證書頒發(fā)機構(gòu)

安裝harbor

訪問UI

上傳鏡像

1.3 Harbor(HTTP)安裝(如果服務(wù)器頂不住，可以換成阿里云私人鏡像倉庫)

解壓harbor

啟動，關(guān)閉harbor

配置harbor私服

1.4 GitLab安裝(如果服務(wù)器頂不住，可以換成碼云或者GitHub)

安裝

啟動和停止

登錄gitlab

1.5 Jenkins安裝(Linux)

安裝jenkins，jdk，git，maven

插件安裝

1.6 Jenkins安裝(windows)

1.7 碼云和阿里云鏡像倉庫創(chuàng)建

碼云倉庫

私人鏡像庫

1.8 腳本編寫

(gitlab)版本

(gitee)版本

1.9 推送基礎(chǔ)鏡像到阿里云

1.10 測試能否成功打包

1.11 測算代碼提交觸發(fā)容器運行

2. k8s部署和編排

2.1 k8s部署

2.2 dashboard安裝和連接k8s

2.1 dashboard使用

公告