golang程序通過(guò)docker打包到harbor的方式進(jìn)行jenkins自動(dòng)化發(fā)布
需求描述：
公司內(nèi)部的git倉(cāng)庫(kù) golang 服務(wù)端代碼需要以docker打包的方式發(fā)布到外網(wǎng)多臺(tái)服務(wù)器中，作為api接口服務(wù)啟動(dòng)，然后nginx反向代理到這幾臺(tái)機(jī)器中
如果不以docker的方式啟動(dòng)直接編譯發(fā)布啟動(dòng)也是可以的，以docker啟動(dòng)方式主要是為后續(xù)上k8s等類似的平臺(tái)做技術(shù)鋪墊
大概的思路：
1.jenkins拉取git代碼通過(guò)rsync推送到遠(yuǎn)端服務(wù)器中(因?yàn)槭莋olang代碼，依賴的go模塊在公司內(nèi)部網(wǎng)絡(luò)git，需要在公司內(nèi)部編譯生成可執(zhí)行的文件，推送到外網(wǎng)進(jìn)行docker打包)
2.在遠(yuǎn)端搭建docker的倉(cāng)庫(kù)harbor服務(wù)器
3.通過(guò)jenkins的方式調(diào)取遠(yuǎn)端服務(wù)器中的腳本進(jìn)行 doker 的打包并推送到harbor中，然后拉取harbor中的鏡像啟動(dòng)服務(wù)
架構(gòu)圖
1.在遠(yuǎn)端服務(wù)器中安裝docker，在其中一臺(tái)服務(wù)器中安裝harbor作為鏡像服務(wù)器
# 安裝docker
yum install -y yum-utils
yum-config-manager --add-repo https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo
yum install -y docker-ce
環(huán)境準(zhǔn)備
mkdir -p /data/www/vhosts/docker/storage/logs
chown -R apache.users /data/www/vhosts/docker/
2.修改docker配置(每個(gè)服務(wù)端的ip端最好不同)
mkdir /etc/docker
# vim /etc/docker/daemon.json
{ "graph": "/data/docker", "storage-driver": "overlay2", "insecure-registries": ["registry.access.redhat.com","quay.io","172.30.0.122:1800"], "registry-mirrors": ["https://q2gr04ke.mirror.aliyuncs.com"], "bip": "10.30.136.1/24", "exec-opts": ["native.cgroupdriver=systemd"], "live-restore": true }
"bip": "10.30.162.1/24" 說(shuō)明：
最好不要和公司內(nèi)部或者vpc的網(wǎng)段沖突
服務(wù)器IP：1.2.17.136/172.30.0.70
對(duì)應(yīng)
10.30.136.1/24
# 啟動(dòng)docker
systemctl start docker
systemctl enable docker
# 安裝 harbor 倉(cāng)庫(kù)
參考：http://www.rzrgm.cn/reblue520/p/13615972.html
修改harbor倉(cāng)庫(kù)的地址
hostname: 172.30.0.122
port: 1800
harbor_admin_password: pass
# 遠(yuǎn)程需要部署docker 安裝包api接口的服務(wù)器上 通過(guò)apache用戶登錄docker harbor
# 生成config.json 這個(gè)秘鑰文件，這樣就可以在遠(yuǎn)程服務(wù)器中執(zhí)行 pull push 命令而不需要輸入賬號(hào)密碼直接執(zhí)行了
# more /data/www/.docker/config.json { "auths": { "172.30.0.122:1800": { "auth": "efadYXJib3IyMDIxYWd2W4=" } } }
3.將apache用戶添加可以運(yùn)行docker的權(quán)限
sudo usermod -aG docker apache
4.編寫遠(yuǎn)程腳本腳本
mkdir /usr/local/worksh/jeninks_task/
# cat /usr/local/worksh/jeninks_task/vidiar_search_docker_build_restart.sh
#!/bin/bash ENV=$1 TAG=$2 # 復(fù)制環(huán)境變量 cd /data/www/vhosts/vidair-search.chinasoft.com/httpdocs cp .env.$ENV .env # 執(zhí)行權(quán)限 chmod 755 vidair-search.exe # 生成docker image cd /data/www/vhosts/vidair-search.chinasoft.com/httpdocs docker build -t 172.30.0.122:1800/todd/vidair-search:$TAG . # 推送到docker倉(cāng)庫(kù) cd /data/www/vhosts/vidair-search.chinasoft.com/httpdocs docker push 172.30.0.122:1800/todd/vidair-search:$TAG # 通知遠(yuǎn)程機(jī)器重新拉取image并且啟動(dòng)container docker pull 172.30.0.122:1800/todd/vidair-search:${TAG} docker stop vidair-search docker rm -f vidair-search docker container run -p 3012:3001 -it -d -v /data/www/vhosts/docker/storage/logs:/app/storage/logs -v /nasplatform:/nasplatform -v /nasuser:/nasuser -v /tmpdata:/tmpdata --name=vidair-search 172.30.0.122:1800/todd/vidair-search:${TAG}
# 添加可執(zhí)行權(quán)限
chmod +x /usr/local/worksh/jeninks_task/vidiar_search_docker_build_restart.sh
chown -R apache.users /usr/local/worksh/jeninks_task
# Dockerfile
# more Dockerfile
From centos RUN mkdir -p /app/storage/logs WORKDIR /app COPY vidair-search.exe /app/vidair-search.exe COPY .env /app/.env EXPOSE 3001 CMD ["/app/vidair-search.exe"]
# 修改 Dockerfile 這樣可以在不能啟動(dòng) docker 容器成功的情況下進(jìn)入容器中排查問(wèn)題
# more Dockerfile
From centos RUN mkdir -p /app/storage/logs WORKDIR /app COPY vidair-search.exe /app/vidair-search.exe COPY .env /app/.env EXPOSE 3001 # CMD ["/app/vidair-search.exe"] CMD ["/bin/bash","-c","tail -f /dev/null"]
5.配置jenkins發(fā)布環(huán)境
git倉(cāng)庫(kù)相關(guān)配置
jekins shell腳本
#!/bin/bash # 此腳本功能為根據(jù)構(gòu)建時(shí)選擇的參數(shù)，同步 /data/www/vhosts/vidair-search.chinasoft.com.prod 下的文件同步到遠(yuǎn)程中轉(zhuǎn)機(jī)器 # 2021.05.21 初始化腳本 #非apache用戶運(yùn)行腳本，則退出 if [ `whoami` != "apache" ];then echo "only apache can run me" exit 1 fi echo "xxx" ## 1.定義變量 dir_name=bak.$(date +%Y-%m-%d-%H-%M-%S) project_dir=/data/www/vhosts/vidair-search.chinasoft.com.prod ## 2.備份代碼函數(shù) function func_project_backup(){ cp -a $project_dir/ /data/data_backup/vidair-search.chinasoft.com.prod_$dir_name } #func_project_backup ## 3.判斷代碼發(fā)布目錄變量是否為空 if [ ! $project_dir ];then echo "$project_dir IS NULL ,shell exit!!!!" exit 1 fi ## 2.判斷同步狀態(tài) function func_rsync_status(){ if [[ $? == 0 || $? == 23 ]];then rsync_edit=1 else rsync_edit=0 echo "`date` 同步到本地目標(biāo)失敗! " exit 1 fi } ## 4.同步到本地待發(fā)路徑 function func_rsync_project_local(){ echo "xxxxxxxxxxxxxx同步待發(fā)目錄開(kāi)始xxxxxxxxxxxxxxxxxx" cd $WORKSPACE /usr/local/bin/rsync -vau -progress --delete --exclude='.git/' --exclude='.gitignore' --exclude='*.log' $WORKSPACE/ $project_dir/httpdocs/ func_rsync_status echo "xxxxxxxxxxxxxx同步待發(fā)目錄完成xxxxxxxxxxxxxxxxxx" } func_rsync_project_local # 執(zhí)行編譯(必須在公司內(nèi)部編譯，因?yàn)榫幾g打包依賴內(nèi)網(wǎng)git倉(cāng)庫(kù)) cd $project_dir/httpdocs/ && chmod +x ./init.production.sh && ./init.production.sh production chown -R apache.users $project_dir/ sleep 1 ## 5.推送代碼到遠(yuǎn)程中轉(zhuǎn)機(jī)并發(fā)布(發(fā)布到線上) echo "------------------------------------ rsync start prod -----------------------------------------" chown -R apache.users $project_dir/ sleep 1 /bin/bash /usr/local/worksh/jeninks_task/online_video.vidair-search.chinasoft.com.prod.sh echo "------------------------------------ rsync done prod -----------------------------------------" ## 7.通過(guò)插件執(zhí)行遠(yuǎn)程服務(wù)器中的腳本生成docker鏡像，然后推送到倉(cāng)庫(kù)中，最后推送到目標(biāo)服務(wù)器中，并啟動(dòng)
涉及的 init.production.sh 腳本內(nèi)容
#!/bin/bash -l ENV=$1 # 復(fù)制環(huán)境變量 cp .env.$ENV .env # 處理依賴 go mod tidy go mod vendor # 構(gòu)建啟動(dòng)程序 GOARCH=amd64 GOOS=linux go build -ldflags "-s -w" -o vidair-search.exe main.go # 執(zhí)行權(quán)限 chmod 755 vidair-search.exe
發(fā)布代碼腳本
# cat /usr/local/worksh/jeninks_task/online_video.vidair-search.chinasoft.com.prod.sh
#!/bin/bash ############################################# ## 設(shè)置變量和GET請(qǐng)求過(guò)來(lái)的變量 ## GET請(qǐng)求傳過(guò)來(lái)的文件所在目錄,目錄路徑寫全路徑了 #dir=$1 passfile="/data/www/.rsync/pass.oneline-video.vidair-search.chinasoft.com.prod" # 非apache用戶運(yùn)行腳本，則退出 if [ `whoami` != "apache" ];then echo " only apache can run me" exit 1 fi # 判斷同步狀態(tài) function func_rsync_status(){ if [[ $? == 0 || $? == 23 ]];then rsync_edit=1 else rsync_edit=0 echo "`date` 同步到目標(biāo)失敗! " exit 1 fi } # 判斷目錄是否為空函數(shù) function func_is_empty_dir(){ return `ls -A $1|wc -w` } # 代碼發(fā)目錄 project_dir="/data/www/vhosts/vidair-search.chinasoft.com.prod/" # 判斷待發(fā)目錄是否為空，為空則退出 if func_is_empty_dir $project_dir then echo " $project_dir is empty , exit!!!!" exit 1 else echo " $project_dir 可以發(fā)布" fi ## 設(shè)置變量，目標(biāo)服務(wù)器 server_ip_list="1.1.1.1 1.1.1.2" # src directory src_directory="vidair-search.chinasoft.com.prod" # dst directory dst_directory="vidair-search.chinasoft.com" exclude_list="--exclude=.svn --exclude=.git --exclude=.gitignore --exclude=*.log --exclude=.gitattributes --exclude=.env" function vidair_rsyn_prod() { # rsync ip_list for ip in ${server_ip_list} do echo "####################rsync ${ip} start################################" rsync -zavP --bwlimit=1000 ${exclude_list} --password-file=${passfile} /data/www/vhosts/${src_directory}/ apache@${ip}::apache/data/www/vhosts/${dst_directory}/ func_rsync_status echo "################### rsync ${ip} end #######################" done } vidair_rsyn_prod exit 0
編譯時(shí)的發(fā)布參數(shù)
# 通過(guò) docker-compose 啟動(dòng)docker容器
# more /usr/local/worksh/jeninks_task/vidiar_search_docker_build_restart.sh
#!/bin/bash ENV=$1 TAG=$2 # 復(fù)制環(huán)境變量 cd /data/www/vhosts/vidair-search.chinasoft.com/httpdocs cp .env.$ENV .env # 執(zhí)行權(quán)限 chmod 755 vidair-search.exe # 生成docker image cd /data/www/vhosts/vidair-search.chinasoft.com/httpdocs docker build -t 172.30.0.122:1800/vidair/search-service:$TAG . docker build -t 172.30.0.122:1800/vidair/search-service-worker:$TAG . # 推送到docker倉(cāng)庫(kù) cd /data/www/vhosts/vidair-search.chinasoft.com/httpdocs docker push 172.30.0.122:1800/vidair/search-service:$TAG docker push 172.30.0.122:1800/vidair/search-service-worker:$TAG # 通知遠(yuǎn)程機(jī)器重新拉取image并且啟動(dòng)container #docker pull 172.30.0.122:1800/vidair/vidair-search:${TAG} #docker stop vidair-search #docker rm -f vidair-search #docker container run -p 3012:3001 -it -d -v /data/www/vhosts/docker/storage/logs:/app/storage/logs -v /nasplatform:/nasplatform -v /nasuser:/nasuser -v /tmpdata:/tmpdata --name=vidair-search 172.30.0.122:1800/vidair/vidair-search:${TAG} #docker-compose -f /data/www/vhosts/vidair-search.chinasoft.com/httpdocs/docker-compose.production.yml up --scale worker=3 -d TAG=${TAG} docker-compose -f /data/www/vhosts/vidair-search.chinasoft.com/httpdocs/docker-compose.production.yml up --scale worker=3 -d
# compose文件
# more /data/www/vhosts/vidair-search.chinasoft.com/httpdocs/docker-compose.production.yml
version: '3' services: web: image: 172.30.0.122:1800/vidair/search-service:${TAG} container_name: vidair-search-container command: ["/app/vidair-search.exe"] ports: - "3012:3001" volumes: - "./storage/logs:/app/storage/logs" - "/nasplatform:/nasplatform" - "/nasuser:/nasuser" - "/tmpdata:/tmpdata" worker: image: 172.30.0.122:1800/vidair/search-service-worker:${TAG} command: ["/app/vidair-search.exe","worker"] volumes: - "./storage/logs:/app/storage/logs" - "/nasplatform:/nasplatform" - "/nasuser:/nasuser" - "/tmpdata:/tmpdata"
容器情況情況
# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES f554731ffe77 172.30.0.122:1800/vidair/search-service-worker:1.0.0 "/app/vidair-search.…" 36 minutes ago Up 36 minutes 3001/tcp httpdocs_worker_3 512c5ba84297 172.30.0.122:1800/vidair/search-service:1.0.0 "/app/vidair-search.…" 36 minutes ago Up 36 minutes 0.0.0.0:3012->3001/tcp, :::3012->3001/tcp vidair-search-container 0921a7e16cfc 172.30.0.122:1800/vidair/search-service-worker:1.0.0 "/app/vidair-search.…" 36 minutes ago Up 36 minutes 3001/tcp httpdocs_worker_2 9e353c484e80 172.30.0.122:1800/vidair/search-service-worker:1.0.0 "/app/vidair-search.…" 36 minutes ago Up 36 minutes 3001/tcp httpdocs_worker_1
浙公網(wǎng)安備 33010602011771號(hào)