When building Npm for front-end projects, it often takes a long time to pull the front-end library. In addition, reuse between different jobs is also a problem. Whether it is artifacts or cache, we need to persist the reused files. Here we take cache as an example

Note: The Gitlab runner here is deployed to the k8s cluster using the helm chart method, and the runner deployment is ignored; the ceph s3 key pair needs to be prepared in advance to configure accesskey and secretkey

Create k8s secret

For Gitlab Runner to connect to ceph s3 later.

1
2
3
4
5
6
7
8
9

apiVersion: v1
data:
accesskey: N1NMT0hIRzYxddfsgxVzVssddfsdY=
secretkey: d25Uc0NDQVdsfsdUkssCQ1VsdEwxeUsdsNwb2R4TnRzZDliTG1DTUN6cQ==
kind: Secret
metadata:
name: gitlab-runner-s3
namespace: gitlab-managed-apps
type: Opaque

helm deploy gitlab runner

Specific Gitlab helm chart reference: https://gitlab.com/gitlab-org/charts/gitlab-runner.git, modify the values.yaml content in the helm chart directory as follows:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12

cache:
## General settings
cacheType: s3
cachePath: "devops" #Specify ceph s3 cache path, here we distinguish by department
cacheShared: true

## S3 settings
s3ServerAddress: "ops-rgw.test.cn"
s3BucketName: "runners-cache"
s3BucketLocation:
s3CacheInsecure: true
secretName: "gitlab-runner-s3"

Update helm configuration

1
2

cd gitlab-runner
helm upgrade runner-devops -f values.yaml -n gitlab-runner .

Test gitlab CI task

Configure gitlab Ci, modify .gitlab-ci.yaml, here we take the front-end project build as an example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19

stages:
- Build

build-and-deploy:
image: registry.test.cn/devops/node:latest
stage: Build
cache:
key: devops-vue
paths:
- node_modules/
-.yarn
tags:
-devopstest
script:
-yarn config set registry https://r.cnpmjs.org
-yarn config set @test:registry https://npm.test.cn/
-yarn --pure-lockfile --cache-folder .yarn --network-timeout 600000
-yarn build
when: always

After Gitlab CI triggers the build task, we observe the real-time status of the JOB build task. At this time, the cache file has been uploaded to ceph s3, and the efficiency of later build and compilation is greatly improved!

1
2
3
4
5
6
7
8

64 Creating cache devops-vue-4...
65 node_modules/: found 30710 matching files and directories
66 .yarn: found 34390 matching files and directories 
67 Uploading cache.zip to https://ops-rgw.test.cn/runners-cache/devops/project/4187/devops-vue-js-starter-4
68 Created cache
70Cleaning up file based variables
00:00
72 Job succeeded