项目以前一直用Jenkins+svn的组合，近期在部门推广git后，就尝试这用gitlab自带的ci，下面简单记录一下踩坑之路。

这并不是一篇详细的安装文档，gitlab的官方文档已经很详细

gitlab-runner的安装

gitlab和gitlab-runner的安装都是通过docker安装的

docker run -d --name gitlab-runner --restart always \
    -v /srv/gitlab-runner/config:/etc/gitlab-runner \
    -v /var/run/docker.sock:/var/run/docker.sock \
    gitlab/gitlab-runner:latest

注册runner

# docker exec -it gitlab-runner gitlab-ci-multi-runner register
Runtime platform                                    arch=amd64 os=linux pid=60 revision=6c154264 version=11.11.0
Running in system-mode.                            
                                                   
Please enter the gitlab-ci coordinator URL (e.g. https://gitlab.com/):
#这里输入的是gitlab的地址
https://gitlab.xxxxxx.com
Please enter the gitlab-ci token for this runner:
#这里输入的是gitlab-ci的token，注意全局runner、群组runner和单个项目的runner的token不同,我这里是用的全局runner
fZzjV_xxxxxxxxxxxxxxxxxx
Please enter the gitlab-ci description for this runner:
# runner的描述
[f72c791415ce]: node-runner
Please enter the gitlab-ci tags for this runner (comma separated):
# runner的tag，很重要的属性，gitlab-ci是通过tag来判断使用哪个runner的
node
Registering runner... succeeded                     runner=fZzjV_tw
Please enter the executor: parallels, ssh, docker-ssh+machine, kubernetes, docker, docker-windows, docker-ssh, shell, virtualbox, docker+machine:
# 我这里选择的docker
docker
Please enter the default Docker image (e.g. ruby:2.1):
# docker的镜像
node:8
Runner registered successfully. Feel free to start it, but if it's running already the config should be automatically reloaded! 

我的token用的全局token，CI的时候始终无法找到runner，后来才发现是要在全局配置将runner分配给具体的项目

分配之后在每个项目的CI界面上显示如下

本地executor镜像

在执行CI的时候发现每次都会重新从docker hub上下载镜像，效率较低

Running with gitlab-runner 11.11.0 (6c154264)
  on maven-runner s5YTytLY
Using Docker executor with image maven:latest ...
Pulling docker image maven:latest ...
ERROR: Preparation failed: Error response from daemon: Get https://registry-1.docker.io/v2/library/maven/manifests/latest: Get https://auth.docker.io/token?scope=repository%3Alibrary%2Fmaven%3Apull&service=registry.docker.io: dial tcp: lookup auth.docker.io: no such host (executor_docker.go:172:13s)
Will be retried in 3s ...

...

$ mvn compile
[INFO] Scanning for projects...
Downloading from central

docker-executor可以指定pull_policy

always 默认值永远从线上拉取
never 永远不从线上拉取，只使用本地镜像
if-not-present 如果本地不存在镜像，则从线上拉取

修改/srv/gitlab-runner/config/config.toml（docker的映射地址），找到相应runner，在runners.docker加入pull_policy = if-not-present


[[runners]]
  name = "maven-runner"
  url = "https://gitlab.xxxxx.com"
  token = "xxxxx"
  executor = "docker"
  [runners.custom_build_dir]
  [runners.docker]
    pull_policy = “if-not-present”

重启gitlab-runner，再次执行CI，可以看到控制台的执行已经改变

Running with gitlab-runner 11.11.0 (6c154264)
  on maven-runner s5YTytLY
Using Docker executor with image maven:latest ...
Using locally found image version due to if-not-present pull policy
Using docker image sha256:cafa0008b7356891041c00a8f4a0d7cce29107e1e6de42fb2c468a7806959cde for maven:latest ...
Running on runner-s5YTytLY-project-1-concurrent-0 via f72c791415ce...
Reinitialized existing Git repository in /builds/tabao-rd/tabao-api/.git/
Fetching changes...
warning: redirecting to https://xxxxx.git/
From http://xxxx
   2e2e4c7..c9182b7  develop    -> origin/develop
Checking out c9182b76 as develop...

部署到远程服务器

因为我是用的是docker镜像，而使用SSH公钥访问服务器的方式需要上传公钥，最终还是要输入密码，所以我们这里采用ssh输入密码的方式实现，需要现在安装sshpass 1.在CI的环境变量处设置服务器变量 2.修改部署配置

test_deploy:
  stage: deploy
  script:
  - echo "Deploy to staging server ${aliyun_server_53_ip}"
  - apt-get --quiet update --yes
  - apt-get --quiet install --yes sshpass
  - echo "delete old version"
  - sshpass -p ${aliyun_server_53_password} ssh -o StrictHostKeyChecking=no ${aliyun_server_53_username}@${aliyun_server_53_ip} "rm -rf /alidata/server/tabao-api/*"
  - echo "upload $CI_PROJECT_DIR/target/microservice-tabao-api-1.0.jar"
  - sshpass -p ${aliyun_server_53_password} scp -o StrictHostKeyChecking=no $CI_PROJECT_DIR/target/microservice-tabao-api-1.0.jar ${aliyun_server_53_username}@${aliyun_server_53_ip}:/alidata/server/tabao-api/
  only:
  - develop
  tags:
  - maven

这样处理后，每次apt-get非常慢，还需要增加国内镜像

test_deploy:
  stage: deploy
  script:
  #国外源很慢，改为使用国内源，maven用的Debian
  - cp /etc/apt/sources.list /etc/apt/sources.listbak
  #删除该源配置文件
  - rm -f /etc/apt/sources.list
  #生成国内的源配置文件
  - echo "deb http://mirrors.163.com/debian/ stretch main non-free contrib" >> /etc/apt/sources.list
  - echo "deb http://mirrors.163.com/debian/ stretch-updates main non-free contrib" >> /etc/apt/sources.list
  - echo "deb http://mirrors.163.com/debian/ stretch-backports main non-free contrib" >> /etc/apt/sources.list
  - echo "deb-src http://mirrors.163.com/debian/ stretch main non-free contrib" >> /etc/apt/sources.list
  - echo "deb-src http://mirrors.163.com/debian/ stretch-updates main non-free contrib" >> /etc/apt/sources.list
  - echo "deb-src http://mirrors.163.com/debian/ stretch-backports main non-free contrib" >> /etc/apt/sources.list
  - echo "deb http://mirrors.163.com/debian-security/ stretch/updates main non-free contrib" >> /etc/apt/sources.list
  - echo "deb-src http://mirrors.163.com/debian-security/ stretch/updates main non-free contrib" >> /etc/apt/sources.list
  - echo "Deploy to staging server ${aliyun_server_53_ip}"
  - apt-get --quiet update --yes
  - apt-get --quiet install --yes sshpass
  - echo "delete old version"
  - sshpass -p ${aliyun_server_53_password} ssh -o StrictHostKeyChecking=no ${aliyun_server_53_username}@${aliyun_server_53_ip} "rm -rf /alidata/server/tabao-api/*"
  - echo "upload $CI_PROJECT_DIR/target/microservice-tabao-api-1.0.jar"
  - sshpass -p ${aliyun_server_53_password} scp -o StrictHostKeyChecking=no $CI_PROJECT_DIR/target/microservice-tabao-api-1.0.jar ${aliyun_server_53_username}@${aliyun_server_53_ip}:/alidata/server/tabao-api/
  only:
  - develop
  tags:
  - maven

maven的仓库配置

我使用的maven的docker镜像进行构建，默认使用的是官方仓库，如果想改成公司仓库就有点困难，网上搜了一下，找到一个解决方案将setting.xml放到.gitlab-ci.yml同级目录然后在.gitlab-ci.yml增加变量

variables:
  MAVEN_CLI_OPTS: "-s ./settings.xml --batch-mode"

在mvn命令后增加参数

- mvn $MAVEN_CLI_OPTS clean package

docker-executor的映射

使用镜像有个缺点就是每次都要重新下载依赖，看了下文档，反向可以在docker-executor里配置映射

[[runners]]
  name = "maven-runner"
  url = "https://gitlab.tabaosmart.com"
  token = "s5YTytLYrBsyyeZyDGsz"
  executor = "docker"
  [runners.custom_build_dir]
  [runners.docker]
    pull_policy = "if-not-present"
    tls_verify = false
    image = "maven:latest"
    privileged = false
    disable_entrypoint_overwrite = false
    oom_kill_disable = false
    disable_cache = false
    volumes = ["/cache", "/srv/gitlab-ci/maven-repository:/root/.m2"]

spring boot项目的乱码

通过ci部署spring boot后，发现入参产生乱码，但是在服务器上重启应用后又没有乱码，猜测是因为runner的ssh编码问题，修改应用的启动脚本，增加-Dfile.encoding=utf-8

一个maven项目的`.gitlab-ci.yml`

stages:
- build
- deploy

variables:
  MAVEN_CLI_OPTS: "-s ./settings.xml --batch-mode"
  MAVEN_OPTS: "-Dmaven.repo.local=.m2/repository"

cache:
  paths:
  - .m2/repository/
  - target/

test_build:
  stage: build
  script:
  - mvn $MAVEN_CLI_OPTS clean package
  - ls -ld $CI_PROJECT_DIR/target/
  only:
  - release
  tags:
  - maven

test_deploy:
  stage: deploy
  script:
  #国外源很慢，改为使用国内源，maven用的Debian
  - cp /etc/apt/sources.list /etc/apt/sources.listbak
  #删除该源配置文件
  - rm -f /etc/apt/sources.list
  #生成国内的源配置文件
  - echo "deb http://mirrors.163.com/debian/ stretch main non-free contrib" >> /etc/apt/sources.list
  - echo "deb http://mirrors.163.com/debian/ stretch-updates main non-free contrib" >> /etc/apt/sources.list
  - echo "deb http://mirrors.163.com/debian/ stretch-backports main non-free contrib" >> /etc/apt/sources.list
  - echo "deb-src http://mirrors.163.com/debian/ stretch main non-free contrib" >> /etc/apt/sources.list
  - echo "deb-src http://mirrors.163.com/debian/ stretch-updates main non-free contrib" >> /etc/apt/sources.list
  - echo "deb-src http://mirrors.163.com/debian/ stretch-backports main non-free contrib" >> /etc/apt/sources.list
  - echo "deb http://mirrors.163.com/debian-security/ stretch/updates main non-free contrib" >> /etc/apt/sources.list
  - echo "deb-src http://mirrors.163.com/debian-security/ stretch/updates main non-free contrib" >> /etc/apt/sources.list
  - echo "Deploy to staging server ${aliyun_server_53_ip}"
  - apt-get --quiet update --yes
  - apt-get --quiet install --yes sshpass
  - echo "delete old version"
  - sshpass -p ${aliyun_server_53_password} ssh -o StrictHostKeyChecking=no ${aliyun_server_53_username}@${aliyun_server_53_ip} "rm -rf /alidata/server/tabao-api/*"
  - echo "upload $CI_PROJECT_DIR/target/microservice-tabao-api-1.0.jar"
  - sshpass -p ${aliyun_server_53_password} scp -o StrictHostKeyChecking=no $CI_PROJECT_DIR/target/microservice-tabao-api-1.0.jar ${aliyun_server_53_username}@${aliyun_server_53_ip}:/alidata/server/tabao-api/microservice-tabao-api-1.0.jar
  - echo "upload $CI_PROJECT_DIR/target/lib"
  - sshpass -p ${aliyun_server_53_password} scp -o StrictHostKeyChecking=no -r $CI_PROJECT_DIR/target/lib ${aliyun_server_53_username}@${aliyun_server_53_ip}:/alidata/server/tabao-api
  - sshpass -p ${aliyun_server_53_password} ssh -o StrictHostKeyChecking=no ${aliyun_server_53_username}@${aliyun_server_53_ip} "ls /alidata/server/tabao-api"
  - echo "run $CI_PROJECT_DIR/target/microservice-tabao-api-1.0.jar"
  - sshpass -p ${aliyun_server_53_password} ssh -o StrictHostKeyChecking=no ${aliyun_server_53_username}@${aliyun_server_53_ip} "/alidata/gitlab-ci/shell/tabao-api.sh"
  only:
  - release
  tags:
  - maven

prod_build:
  stage: build
  script:
  - mvn $MAVEN_CLI_OPTS clean package
  - ls -ld $CI_PROJECT_DIR/target/
  only:
  - master
  tags:
  - maven

prod_deploy:
  stage: deploy
  script:
  #国外源很慢，改为使用国内源，maven用的Debian
  - cp /etc/apt/sources.list /etc/apt/sources.listbak
  #删除该源配置文件
  - rm -f /etc/apt/sources.list
  #生成国内的源配置文件
  - echo "deb http://mirrors.163.com/debian/ stretch main non-free contrib" >> /etc/apt/sources.list
  - echo "deb http://mirrors.163.com/debian/ stretch-updates main non-free contrib" >> /etc/apt/sources.list
  - echo "deb http://mirrors.163.com/debian/ stretch-backports main non-free contrib" >> /etc/apt/sources.list
  - echo "deb-src http://mirrors.163.com/debian/ stretch main non-free contrib" >> /etc/apt/sources.list
  - echo "deb-src http://mirrors.163.com/debian/ stretch-updates main non-free contrib" >> /etc/apt/sources.list
  - echo "deb-src http://mirrors.163.com/debian/ stretch-backports main non-free contrib" >> /etc/apt/sources.list
  - echo "deb http://mirrors.163.com/debian-security/ stretch/updates main non-free contrib" >> /etc/apt/sources.list
  - echo "deb-src http://mirrors.163.com/debian-security/ stretch/updates main non-free contrib" >> /etc/apt/sources.list
  - echo "Deploy to staging server ${aliyun_server_68_ip}"
  - apt-get --quiet update --yes
  - apt-get --quiet install --yes sshpass
  - echo "delete old version"
  - sshpass -p ${aliyun_server_68_password} ssh -o StrictHostKeyChecking=no ${aliyun_server_68_username}@${aliyun_server_68_ip} "rm -rf /alidata/server/tabao-api/*"
  - echo "upload $CI_PROJECT_DIR/target/microservice-tabao-api-1.0.jar"
  - sshpass -p ${aliyun_server_68_password} scp -o StrictHostKeyChecking=no $CI_PROJECT_DIR/target/microservice-tabao-api-1.0.jar ${aliyun_server_68_username}@${aliyun_server_68_ip}:/alidata/server/tabao-api/microservice-tabao-api-1.0.jar
  - echo "upload $CI_PROJECT_DIR/target/lib"
  - sshpass -p ${aliyun_server_68_password} scp -o StrictHostKeyChecking=no -r $CI_PROJECT_DIR/target/lib ${aliyun_server_68_username}@${aliyun_server_68_ip}:/alidata/server/tabao-api
  - sshpass -p ${aliyun_server_68_password} ssh -o StrictHostKeyChecking=no ${aliyun_server_68_username}@${aliyun_server_68_ip} "ls /alidata/server/tabao-api"
  - echo "run $CI_PROJECT_DIR/target/microservice-tabao-api-1.0.jar"
  - sshpass -p ${aliyun_server_68_password} ssh -o StrictHostKeyChecking=no ${aliyun_server_68_username}@${aliyun_server_68_ip} "/alidata/gitlab-ci/shell/tabao-api.sh"
  only:
  - master
  tags:
  - maven

一个前端项目的`.gitlab-ci.yml`

stages:
- build
- deploy

test_build:
  stage: build
  script:
  - npm config set registry https://registry.npm.taobao.org
  - npm config set chromedriver_cdnurl https://npm.taobao.org/mirrors/chromedriver
  #- npm config set sass_binary_site https://npm.taobao.org/mirrors/node-sass
  - npm install --registry=https://registry.npm.taobao.org
  - npm run build
  - ls -l $CI_PROJECT_DIR/
  only:
  - release
  tags:
  - node
  artifacts:
    expire_in: 1 week
    paths:
    - dist

test_deploy:
  stage: deploy
  script:
  #国外源很慢，改为使用国内源，node用的Debian
  - cp /etc/apt/sources.list /etc/apt/sources.listbak
  #删除该源配置文件
  - rm -f /etc/apt/sources.list
  #生成国内的源配置文件
  - echo "deb http://mirrors.163.com/debian/ stretch main non-free contrib" >> /etc/apt/sources.list
  - echo "deb http://mirrors.163.com/debian/ stretch-updates main non-free contrib" >> /etc/apt/sources.list
  - echo "deb http://mirrors.163.com/debian/ stretch-backports main non-free contrib" >> /etc/apt/sources.list
  - echo "deb-src http://mirrors.163.com/debian/ stretch main non-free contrib" >> /etc/apt/sources.list
  - echo "deb-src http://mirrors.163.com/debian/ stretch-updates main non-free contrib" >> /etc/apt/sources.list
  - echo "deb-src http://mirrors.163.com/debian/ stretch-backports main non-free contrib" >> /etc/apt/sources.list
  - echo "deb http://mirrors.163.com/debian-security/ stretch/updates main non-free contrib" >> /etc/apt/sources.list
  - echo "deb-src http://mirrors.163.com/debian-security/ stretch/updates main non-free contrib" >> /etc/apt/sources.list
  - echo "Deploy to staging server ${aliyun_server_53_ip}"
  - apt-get --quiet update --yes
  - apt-get --quiet install --yes sshpass
  - echo "delete old version"
  - echo "sshpass -p ${aliyun_server_53_password} ssh -o StrictHostKeyChecking=no ${aliyun_server_53_username}@${aliyun_server_53_ip} 'rm -rf /alidata/gitlab-ci/tabao-mobile-web/*'"
  - sshpass -p ${aliyun_server_53_password} ssh -o StrictHostKeyChecking=no ${aliyun_server_53_username}@${aliyun_server_53_ip} 'rm -rf /alidata/gitlab-ci/tabao-mobile-web/*'
  - ls -l $CI_PROJECT_DIR/
  - ls -l $CI_PROJECT_DIR/dist
  - echo "upload $CI_PROJECT_DIR/dist"
  - echo "sshpass -p ${aliyun_server_53_password} scp -o StrictHostKeyChecking=no -r $CI_PROJECT_DIR/dist/index.html ${aliyun_server_53_username}@${aliyun_server_53_ip}:/alidata/gitlab-ci/tabao-mobile-web"
  - sshpass -p ${aliyun_server_53_password} scp -o StrictHostKeyChecking=no -r $CI_PROJECT_DIR/dist/index.html ${aliyun_server_53_username}@${aliyun_server_53_ip}:/alidata/gitlab-ci/tabao-mobile-web
  - sshpass -p ${aliyun_server_53_password} ssh -o StrictHostKeyChecking=no ${aliyun_server_53_username}@${aliyun_server_53_ip} "ls /alidata/gitlab-ci/tabao-mobile-web"
  - echo "run /alidata/gitlab-ci/shell/tabao-mobile-web.sh"
  - sshpass -p ${aliyun_server_53_password} ssh -o StrictHostKeyChecking=no ${aliyun_server_53_username}@${aliyun_server_53_ip} "/alidata/gitlab-ci/shell/tabao-mobile-web.sh"
  only:
  - release
  tags:
  - node

注意：这里和maven的有点区别，如果没有下面这段配置，执行deploy的时候，会删除一次dist，导致无法上传的BUG

  artifacts:
    expire_in: 1 week
    paths:
    - dist

猜测是maven的配置了cache的原因，没有深究

参考资料

https://docs.gitlab.com/runner/executors/docker.html

gitlab-ci踩坑

最后更新：2019-05-21

gitlab-runner的安装

tags

本地executor镜像

部署到远程服务器

maven的仓库配置

docker-executor的映射

spring boot项目的乱码

一个maven项目的`.gitlab-ci.yml`

一个前端项目的`.gitlab-ci.yml`

参考资料

gitlab-ci踩坑

最后更新：2019-05-21

gitlab-runner的安装

tags

本地executor镜像

部署到远程服务器

maven的仓库配置

docker-executor的映射

spring boot项目的乱码

一个maven项目的.gitlab-ci.yml

一个前端项目的.gitlab-ci.yml

参考资料

一个maven项目的`.gitlab-ci.yml`

一个前端项目的`.gitlab-ci.yml`