[master] 100ccc2b2 Major reorganization of the CircleCI workflows
Dridi Boukelmoune
dridi.boukelmoune at gmail.com
Fri May 15 09:12:07 UTC 2020
commit 100ccc2b23ae096e8d25b094622fd2afdc6c60c1
Author: Martin Tzvetanov Grigorov <mgrigorov at apache.org>
Date: Thu Mar 26 09:48:02 2020 +0200
Major reorganization of the CircleCI workflows
In order to build aarch64 packages it relies on the machine executors
feature based on QEMU. Since the same scripts were needed to build
packages for different architectures they were extracted in standalone
shell scripts.
Alpine Linux packages are now built in addition to RPM and Deb packages
but considering how long emulation takes for aarch64 packages, packaging
was moved to a nightly workflow.
Some changes happened in the pkg-varnish-cache repository to accomodate
the CircleCI reorganization, mainly around dependency management.
More information in the CircleCI README file.
Closes #3313
Reviewed-by: Guillaume Quintard <guillaume at varnish-software.com>
Signed-off-by: Dridi Boukelmoune <dridi.boukelmoune at gmail.com>
diff --git a/.circleci/README.rst b/.circleci/README.rst
new file mode 100644
index 000000000..0c52c4235
--- /dev/null
+++ b/.circleci/README.rst
@@ -0,0 +1,93 @@
+Multiarch building, testing & packaging
+=======================================
+
+Varnish Cache uses CircleCI_ for building, testing and creating packages for
+several Linux distributions for both x86_64 and aarch64 architectures.
+
+Since CircleCI provides only x86_64 VMs the setup uses Docker and QEMU to be
+able to build, test and create packages for aarch64. This is accomplished by
+registering ``qemu-user-static`` for the CircleCI ``machine`` executor::
+
+ sudo docker run --rm --privileged multiarch/qemu-user-static --reset --credential yes --persistent yes
+
+Note 1: **--credential yes** is needed so that *setuid* flag is working.
+Without it ``sudo`` does not work in the Docker containers with architecture
+different than x86_64.
+
+Note 2: **--persistent yes** is needed so that there is no need to use
+``:register`` tag. This way one can run locally pure foreign arch Docker
+images, like the official ``arm64v8/***`` ones.
+
+With QEMU registered each build step can start a Docker image for any of the
+supported architectures to execute the ``configure``, ``make``, package steps.
+
+Workflows
+---------
+
+There are two CircleCI workflows:
+
+commit
+~~~~~~
+
+It is executed after each push to any branch, including Pull Requests
+
+The ``commit`` workflow runs two jobs:
+
+- ``dist`` - this job creates the source code distribution of Varnish Cache as
+ compressed archive (``varnish-${VERSION}.tar.gz``).
+
+- ``distcheck`` - untars the source code distribution from ``dist`` job and
+ builds (*configure*, *make*) on different Linux distributions
+
+nightly
+~~~~~~~
+
+It is executed once per day at 04:00 AM UTC time.
+
+This workflow also builds binary packages for different Linux distributions
+and CPU architectures (x86_64 & aarch64) and for this reason its run takes
+longer.
+
+It runs the following jobs:
+
+- The first two jobs that run in parallel are:
+
+ - ``tar_pkg_tools`` - this step checks out pkg-varnish-cache_ with the
+ packaging descriptions for Debian, RedHat and Alpine, and stores them in
+ the build workspace for the next steps in the pipeline.
+
+ - ``dist`` - this step creates the source code distribution of Varnish Cache
+ as compressed archive (``varnish-${VERSION}.tar.gz``). This archive is
+ also stored in the build workspace and used later by the packaging steps.
+
+
+- The next job in the workflow is ``package`` - a job that creates the
+ packages (e.g. .rpm, .deb) for each supported CPU architecture, Linux
+ distribution and its major version (e.g. *x64_centos_7*,
+ *aarch64_ubuntu_bionic*, *x64_alpine_3*, etc.). This step creates a
+ Dockerfile on the fly by using a base Docker image. This custom Docker image
+ executes a Shell script that has the recipe for creating the package for the
+ specific Linux flavor, e.g. *make-rpm-packages.sh*. The step stores the
+ packages in the build workspace.
+
+- Finally, if the previous jobs are successful, a final step is executed -
+ ``collect_packages``. This step creates an archive with all packages and
+ stores it as an artifact that can be uploaded to PackageCloud_.
+
+
+More
+----
+
+This setup can be easily extended for any CPU architectures supported by QEMU
+and for any Linux distributions which have Docker image. To do this one needs
+to add a new ``package`` job with the proper parameters for it.
+
+At the moment the setup uses *raw* Docker images and installs the required
+Linux distribution dependencies before running the tests/build/packaging code.
+This could be optimized to save some execution time by creating custom Docker
+images that extend the current ones and pre-installs the required
+dependencies.
+
+.. _CircleCI: https://app.circleci.com/pipelines/github/varnishcache/varnish-cache
+.. _pkg-varnish-cache: https://github.com/varnishcache/pkg-varnish-cache
+.. _PackageCloud: https://packagecloud.io/varnishcache/
diff --git a/.circleci/config.yml b/.circleci/config.yml
index 26716acb8..0ede16afa 100644
--- a/.circleci/config.yml
+++ b/.circleci/config.yml
@@ -1,5 +1,11 @@
version: 2.1
+aliases:
+ pkg_req: &pkg_req
+ requires:
+ - dist
+ - tar_pkg_tools
+
parameters:
vc-commit:
type: string
@@ -9,6 +15,7 @@ parameters:
default: "master"
jobs:
dist:
+ description: Builds varnish-x.y.z.tar.gz that is used later for the packaging jobs
docker:
- image: centos:7
steps:
@@ -45,7 +52,9 @@ jobs:
- .is_weekly
- varnish*.tar.gz
- tools/*.suppr
+ - .circleci
tar_pkg_tools:
+ description: Builds archives with the packaging tools from https://github.com/varnishcache/pkg-varnish-cache
docker:
- image: centos:7
steps:
@@ -62,7 +71,7 @@ jobs:
git clone https://github.com/varnishcache/pkg-varnish-cache.git .
git checkout << pipeline.parameters.pkg-commit >>
tar cvzf debian.tar.gz debian --dereference
- tar cvzf redhat.tar.gz redhat --dereference
+ tar cvzf redhat.tar.gz redhat --dereference
tar cvzf alpine.tar.gz alpine --dereference
- persist_to_workspace:
root: .
@@ -70,6 +79,53 @@ jobs:
- debian.tar.gz
- redhat.tar.gz
- alpine.tar.gz
+ package:
+ parameters:
+ dist:
+ description: the Linux distribution (debian|ubuntu|centos)
+ type: string
+ release:
+ description: the release name (stretch|buster|xenial|bionic|7|8)
+ type: string
+ ext:
+ description: the package extension (deb|rpm|apk)
+ type: string
+ arch:
+ description: the architecture (x64|aarch64)
+ type: string
+ image:
+ description: the base Docker image for Dockerfile
+ type: string
+ machine:
+ image: ubuntu-1604:201903-01
+ steps:
+ - attach_workspace:
+ at: ~/project
+ - run: ls -la ~/project
+ - run:
+ name: Activate QEMU
+ command: |
+ sudo docker run -it --rm --privileged multiarch/qemu-user-static --reset --credential yes --persistent yes
+ - run:
+ name: Create Dockerfile
+ command: |
+ echo "FROM << parameters.image >>" > Dockerfile
+ echo "ADD make-<< parameters.ext >>-packages.sh /usr/bin/" >> Dockerfile
+ echo 'CMD ["make-<< parameters.ext >>-packages.sh"]' >> Dockerfile
+ - run:
+ name: Build << parameters.dist >> << parameters.release >> << parameters.arch >> << parameters.ext >>
+ command: |
+ mkdir -p packages
+ cp .circleci/make-<< parameters.ext >>-packages.sh .
+ docker build -t varnish-<< parameters.ext >>-package-build:<< parameters.arch >> .
+ docker run --rm -it -e PARAM_DIST=<< parameters.dist >> -e PARAM_RELEASE=<< parameters.release >> -v$(pwd):/varnish-cache varnish-<< parameters.ext >>-package-build:<< parameters.arch >>
+ - run:
+ name: List created packages
+ command: find ./packages -name "*.<< parameters.ext >>"
+ - persist_to_workspace:
+ root: .
+ paths:
+ - "packages"
distcheck:
parameters:
dist:
@@ -87,7 +143,7 @@ jobs:
working_directory: /workspace
steps:
- run:
- name: Possibly activate centos:8 extra repos
+ name: Possible << parameters.dist >>:<< parameters.release >> extra repos
command: |
if [ << parameters.dist >> = centos ]; then
if [ << parameters.release >> = 8 ]; then
@@ -150,13 +206,14 @@ jobs:
- run:
name: Extract and distcheck
command: |
- tar xavf *.tar.gz --strip 1
+ tar xavf varnish-*.tar.gz --strip 1
if [ << parameters.dist >> = centos ]; then
adduser varnish
else
adduser --disabled-password --gecos "" varnish
fi
- chown -R varnish:varnish /workspace
+
+ chown -R varnish:varnish .
export ASAN_OPTIONS=abort_on_error=1,detect_odr_violation=1,detect_leaks=1,detect_stack_use_after_return=1,detect_invalid_pointer_pairs=1,handle_segv=0,handle_sigbus=0,use_sigaltstack=0,disable_coredump=0
export LSAN_OPTIONS=abort_on_error=1,use_sigaltstack=0,suppressions=$(pwd)/tools/lsan.suppr
@@ -174,285 +231,28 @@ jobs:
sudo -u varnish \
--preserve-env=ASAN_OPTIONS,LSAN_OPTIONS,TSAN_OPTIONS,UBSAN_OPTIONS \
make distcheck VERBOSE=1 -j 12 -k
- build_apks:
- parameters:
- dist:
- description: the Linux distribution (alpine)
- default: alpine
- type: string
- release:
- description: the release version
- default: latest
- type: string
- description: Build alpine apks
- docker:
- - image: alpine
- working_directory: /workspace
- steps:
- - run:
- name: Install certificates to mount the workspace, and tar
- command: |
- apk update
- apk add -q ca-certificates tar
- - attach_workspace:
- at: /workspace
- - run:
- name: Untar alpine
- command: |
- tar xavf alpine.tar.gz --strip 1
- - run:
- name: Install sdk, add user
- command: |
- apk add alpine-sdk
- adduser -D builder
- echo "builder ALL=(ALL) NOPASSWD: ALL" > /etc/sudoers
- addgroup builder abuild
- mkdir -p /var/cache/distfiles
- chmod a+w /var/cache/distfiles
- - run:
- name: Generate key
- command: |
- su builder -c "abuild-keygen -nai"
- - run:
- name: Fix APKBUILD's variables
- command: |
- tar xavf varnish-*.tar.gz
- VERSION=$(varnish-*/configure --version | awk 'NR == 1 {print $NF}')
- sed -i "s/@VERSION@/$VERSION/" APKBUILD
- rm -rf varnish-*/
- - run:
- name: Fix checksums, build
- command: |
- chown builder -R /workspace
- su builder -c "abuild checksum"
- su builder -c "abuild -r"
- - run:
- name: Fix the APKBUILD's version
- command: |
- mkdir apks
- cp /home/builder/packages/x86_64/*.apk apks
- - run:
- name: Import the packages into the workspace
- command: |
- mkdir -p packages/<< parameters.dist >>/<< parameters.release >>
- mv /home/builder/packages/x86_64/*.apk packages/<< parameters.dist >>/<< parameters.release >>
- - persist_to_workspace:
- root: .
- paths:
- - packages/<< parameters.dist >>/<< parameters.release >>/varnish*
- - store_artifacts:
- path: packages/<< parameters.dist >>/<< parameters.release >>
- destination: ./
- build_debs:
- parameters:
- dist:
- description: the Linux distribution (debian|ubuntu)
- type: string
- release:
- description: the release name (stretch|buster|xenial|bionic)
- type: string
- description: Build << parameters.release >> debs
- docker:
- - image: << parameters.dist >>:<< parameters.release >>
- steps:
- - run:
- name: Install packaging tools
- command: |
- apt-get update
- apt-get install -y dpkg-dev ca-certificates debhelper devscripts equivs
- - attach_workspace:
- at: ~/project
- - run:
- name: Untar debian
- command: tar xavf debian.tar.gz
- - run:
- name: Untar orig
- command: tar xavf varnish*.tar.gz --strip 1
- - run:
- name: Update changelog version
- command: |
- if [ -e .is_weekly ]; then
- WEEKLY='-weekly'
- else
- WEEKLY=
- fi
- VERSION=$(./configure --version | awk 'NR == 1 {print $NF}')$WEEKLY~<< parameters.release >>
- sed -i -e "s|@VERSION@|$VERSION-1|" "debian/changelog"
- - run:
- name: Install Build-Depends packages
- command: |
- export DEBIAN_FRONTEND=noninteractive
- export DEBCONF_NONINTERACTIVE_SEEN=true
- yes | mk-build-deps --install debian/control || true
- - run:
- name: Build the packages
- command: |
- dpkg-buildpackage -us -uc -j16
- - run:
- name: Import the packages into the workspace
- command: |
- mkdir -p packages/<< parameters.dist >>/<< parameters.release >>
- mv ../*.deb ../*.dsc packages/<< parameters.dist >>/<< parameters.release >>
- - persist_to_workspace:
- root: .
- paths:
- - packages/<< parameters.dist >>/<< parameters.release >>/varnish*
- - store_artifacts:
- path: packages/<< parameters.dist >>/<< parameters.release >>
- destination: ./
- build_rpms:
- parameters:
- dist:
- description: the Linux distribution (centos|amazonlinux)
- type: string
- release:
- description: the distribution version (7|8|2)
- type: string
- description: Build << parameters.dist >>:<< parameters.release >> rpms
- docker:
- - image: centos:<< parameters.release >>
- environment:
- DIST_DIR: build
- steps:
- - run:
- name: Install packaging tools
- command: |
- yum install -y rpm-build yum-utils epel-release
- # XXX: we should NOT have to do that here, they should be in the
- # spec as BuildRequires
- yum install -y make gcc
- - attach_workspace:
- at: ~/project
- - run:
- name: Create build dir
- command: mkdir $DIST_DIR
- - run:
- name: Untar redhat
- command: |
- tar xavf redhat.tar.gz -C build
- - run:
- name: Untar orig
- command: |
- tar xavf varnish*.tar.gz -C build --strip 1
- - run:
- name: Build Packages
- command: |
- set -e
- set -u
-
- if [ << parameters.release >> = 8 ]; then
- yum config-manager --set-enabled PowerTools
- fi
- # use python3
- sed -i '1 i\%global __python %{__python3}' "$DIST_DIR"/redhat/varnish.spec
- if [ -e .is_weekly ]; then
- WEEKLY='.weekly'
- else
- WEEKLY=
- fi
- VERSION=$("$DIST_DIR"/configure --version | awk 'NR == 1 {print $NF}')$WEEKLY
- cp -r -L "$DIST_DIR"/redhat/* "$DIST_DIR"/
- tar zcf "$DIST_DIR.tgz" --exclude "$DIST_DIR/redhat" "$DIST_DIR"/
-
- RPMVERSION="$VERSION"
-
- RESULT_DIR="rpms"
- CUR_DIR="$(pwd)"
-
- rpmbuild() {
- command rpmbuild \
- --define "_smp_mflags -j10" \
- --define "_sourcedir $CUR_DIR" \
- --define "_srcrpmdir $CUR_DIR/${RESULT_DIR}" \
- --define "_rpmdir $CUR_DIR/${RESULT_DIR}" \
- --define "versiontag ${RPMVERSION}" \
- --define "releasetag 0.0" \
- --define "srcname $DIST_DIR" \
- --define "nocheck 1" \
- "$@"
- }
- yum-builddep -y "$DIST_DIR"/redhat/varnish.spec
- rpmbuild -bs "$DIST_DIR"/redhat/varnish.spec
- rpmbuild --rebuild "$RESULT_DIR"/varnish-*.src.rpm
- - run:
- name: Prepare the packages for storage
- command: |
- mkdir -p packages/<< parameters.dist >>/<< parameters.release >>
- mv rpms/*/*.rpm packages/<< parameters.dist >>/<< parameters.release >>
- - persist_to_workspace:
- root: .
- paths:
- - packages/<< parameters.dist >>/<< parameters.release >>/*.rpm
- - store_artifacts:
- path: packages/<< parameters.dist >>/<< parameters.release >>
- destination: ./
collect_packages:
docker:
- image: centos:7
steps:
- attach_workspace:
at: ~/project
+ - run: ls -la ~/project/
- run:
name: Tar the packages
command: |
- tar cvzf packages.tar.gz packages
+ tar cvzf packages.tar.gz packages
- store_artifacts:
destination: packages.tar.gz
path: packages.tar.gz
-pkg_req: &pkg_req
- requires:
- - dist
- - tar_pkg_tools
+
workflows:
version: 2
- build:
+ commit:
jobs:
- dist
- - tar_pkg_tools
- - build_debs:
- name: build_debian_stretch
- dist: debian
- release: stretch
- <<: *pkg_req
- - build_debs:
- name: build_debian_buster
- dist: debian
- release: buster
- <<: *pkg_req
- - build_debs:
- name: build_ubuntu_xenial
- dist: ubuntu
- release: xenial
- <<: *pkg_req
- - build_debs:
- name: build_ubuntu_bionic
- dist: ubuntu
- release: bionic
- <<: *pkg_req
- - build_rpms:
- name: build_centos_7
- dist: centos
- release: "7"
- <<: *pkg_req
- - build_rpms:
- name: build_centos_8
- dist: centos
- release: "8"
- <<: *pkg_req
- - build_apks:
- name: build_alpine
- <<: *pkg_req
- - collect_packages:
- requires:
- - build_debian_stretch
- - build_debian_buster
- - build_ubuntu_xenial
- - build_ubuntu_bionic
- - build_centos_7
- - build_centos_8
- - build_alpine
- distcheck:
name: distcheck_centos_7
dist: centos
@@ -479,3 +279,161 @@ workflows:
#extra_conf: --without-jemalloc
requires:
- dist
+ nightly:
+ triggers:
+ - schedule:
+ cron: "0 4 * * *"
+ filters:
+ branches:
+ only:
+ - master
+ jobs:
+ - dist
+ - tar_pkg_tools
+ - package:
+ name: aarch64-ubuntu-bionic
+ dist: ubuntu
+ release: bionic
+ arch: aarch64
+ image: arm64v8/ubuntu:bionic
+ ext: deb
+ <<: *pkg_req
+ - package:
+ name: x64-ubuntu-bionic
+ dist: ubuntu
+ release: bionic
+ arch: x64
+ image: ubuntu:bionic
+ ext: deb
+ <<: *pkg_req
+ - package:
+ name: aarch64-ubuntu-xenial
+ dist: ubuntu
+ release: xenial
+ arch: aarch64
+ image: arm64v8/ubuntu:xenial
+ ext: deb
+ <<: *pkg_req
+ - package:
+ name: x64-ubuntu-xenial
+ dist: ubuntu
+ release: xenial
+ arch: x64
+ image: ubuntu:xenial
+ ext: deb
+ <<: *pkg_req
+ - package:
+ name: aarch64-debian-bullseye
+ dist: debian
+ release: bullseye
+ arch: aarch64
+ image: arm64v8/debian:bullseye-slim
+ ext: deb
+ <<: *pkg_req
+ - package:
+ name: x64-debian-bullseye
+ dist: debian
+ release: bullseye
+ arch: x64
+ image: debian:bullseye-slim
+ ext: deb
+ <<: *pkg_req
+ - package:
+ name: aarch64-debian-buster
+ dist: debian
+ release: buster
+ arch: aarch64
+ image: arm64v8/debian:buster-slim
+ ext: deb
+ <<: *pkg_req
+ - package:
+ name: x64-debian-buster
+ dist: debian
+ release: buster
+ arch: x64
+ image: debian:buster-slim
+ ext: deb
+ <<: *pkg_req
+ - package:
+ name: aarch64-debian-stretch
+ dist: debian
+ release: stretch
+ arch: aarch64
+ image: arm64v8/debian:stretch-slim
+ ext: deb
+ <<: *pkg_req
+ - package:
+ name: x64-debian-stretch
+ dist: debian
+ release: stretch
+ arch: x64
+ image: debian:stretch-slim
+ ext: deb
+ <<: *pkg_req
+ - package:
+ name: aarch64-centos-7
+ dist: centos
+ release: "7"
+ arch: aarch64
+ image: arm64v8/centos:7
+ ext: rpm
+ <<: *pkg_req
+ - package:
+ name: x64-centos-7
+ dist: centos
+ release: "7"
+ arch: x64
+ image: centos:7
+ ext: rpm
+ <<: *pkg_req
+ - package:
+ name: aarch64-centos-8
+ dist: centos
+ release: "8"
+ arch: aarch64
+ image: arm64v8/centos:8
+ ext: rpm
+ <<: *pkg_req
+ - package:
+ name: x64-centos-8
+ dist: centos
+ release: "8"
+ arch: x64
+ image: centos:8
+ ext: rpm
+ <<: *pkg_req
+ - package:
+ name: x64-alpine-3
+ dist: alpine
+ release: "3"
+ arch: x64
+ image: alpine:3
+ ext: apk
+ <<: *pkg_req
+ - package:
+ name: aarch64-alpine-3
+ dist: alpine
+ release: "3"
+ arch: aarch64
+ image: arm64v8/alpine:3
+ ext: apk
+ <<: *pkg_req
+ - collect_packages:
+ requires:
+ - x64-ubuntu-xenial
+ - x64-ubuntu-bionic
+ - aarch64-ubuntu-xenial
+ - aarch64-ubuntu-bionic
+ - x64-debian-bullseye
+ - aarch64-debian-bullseye
+ - x64-debian-buster
+ - aarch64-debian-buster
+ - x64-debian-stretch
+ - aarch64-debian-stretch
+ - x64-centos-7
+ - x64-centos-8
+ - aarch64-centos-7
+ - aarch64-centos-8
+ - x64-alpine-3
+ - aarch64-alpine-3
+
diff --git a/.circleci/make-apk-packages.sh b/.circleci/make-apk-packages.sh
new file mode 100755
index 000000000..26b672c2c
--- /dev/null
+++ b/.circleci/make-apk-packages.sh
@@ -0,0 +1,53 @@
+#!/usr/bin/env sh
+
+set -eux
+
+apk add -q --no-progress --update tar alpine-sdk
+
+echo "PARAM_RELEASE: $PARAM_RELEASE"
+echo "PARAM_DIST: $PARAM_DIST"
+
+if [ -z "$PARAM_RELEASE" ]; then
+ echo "Env variable PARAM_RELEASE is not set! For example PARAM_RELEASE=8, for CentOS 8"
+ exit 1
+elif [ -z "$PARAM_DIST" ]; then
+ echo "Env variable PARAM_DIST is not set! For example PARAM_DIST=centos"
+ exit 1
+fi
+
+cd /varnish-cache
+tar xazf alpine.tar.gz --strip 1
+
+adduser -D builder
+echo "builder ALL=(ALL) NOPASSWD: ALL" > /etc/sudoers
+addgroup builder abuild
+mkdir -p /var/cache/distfiles
+chmod -R a+w /var/cache/distfiles
+
+echo "Generate key"
+su builder -c "abuild-keygen -nai"
+
+echo "Fix APKBUILD's variables"
+tar xavf varnish-*.tar.gz
+VERSION=$(varnish-*/configure --version | awk 'NR == 1 {print $NF}')
+echo "Version: $VERSION"
+sed -i "s/@VERSION@/$VERSION/" APKBUILD
+rm -rf varnish-*/
+
+echo "Change the ownership so that abuild is able to write its logs"
+chown builder -R .
+echo "Fix checksums, build"
+su builder -c "abuild checksum"
+su builder -c "abuild -r"
+
+echo "Fix the APKBUILD's version"
+su builder -c "mkdir apks"
+ARCH=`uname -m`
+su builder -c "cp /home/builder/packages/$ARCH/*.apk apks"
+
+echo "Import the packages into the workspace"
+mkdir -p packages/$PARAM_DIST/$PARAM_RELEASE/$ARCH/
+mv /home/builder/packages/$ARCH/*.apk packages/$PARAM_DIST/$PARAM_RELEASE/$ARCH/
+
+echo "Allow to read the packages by 'circleci' user outside of Docker after 'chown builder -R .' above"
+chmod -R a+rwx .
diff --git a/.circleci/make-deb-packages.sh b/.circleci/make-deb-packages.sh
new file mode 100755
index 000000000..ba4126f93
--- /dev/null
+++ b/.circleci/make-deb-packages.sh
@@ -0,0 +1,48 @@
+#!/usr/bin/env bash
+
+set -eux
+
+export DEBIAN_FRONTEND=noninteractive
+export DEBCONF_NONINTERACTIVE_SEEN=true
+apt-get update
+apt-get install -y dpkg-dev debhelper devscripts equivs pkg-config apt-utils
+
+echo "PARAM_RELEASE: $PARAM_RELEASE"
+echo "PARAM_DIST: $PARAM_DIST"
+
+
+if [ -z "$PARAM_RELEASE" ]; then
+ echo "Env variable PARAM_RELEASE is not set! For example PARAM_RELEASE=8, for CentOS 8"
+ exit 1
+elif [ -z "$PARAM_DIST" ]; then
+ echo "Env variable PARAM_DIST is not set! For example PARAM_DIST=centos"
+ exit 1
+fi
+
+cd /varnish-cache
+ls -la
+
+echo "Untar debian..."
+tar xavf debian.tar.gz
+
+echo "Untar orig..."
+tar xavf varnish-*.tar.gz --strip 1
+
+echo "Update changelog version..."
+if [ -e .is_weekly ]; then
+ WEEKLY='-weekly'
+else
+ WEEKLY=
+fi
+VERSION=$(./configure --version | awk 'NR == 1 {print $NF}')$WEEKLY~$PARAM_RELEASE
+sed -i -e "s|@VERSION@|$VERSION-1|" "debian/changelog"
+
+echo "Install Build-Depends packages..."
+yes | mk-build-deps --install debian/control || true
+
+echo "Build the packages..."
+dpkg-buildpackage -us -uc -j16
+
+echo "Prepare the packages for storage..."
+mkdir -p packages/$PARAM_DIST/$PARAM_RELEASE/
+mv ../*.deb packages/$PARAM_DIST/$PARAM_RELEASE/
diff --git a/.circleci/make-rpm-packages.sh b/.circleci/make-rpm-packages.sh
new file mode 100755
index 000000000..b443c9139
--- /dev/null
+++ b/.circleci/make-rpm-packages.sh
@@ -0,0 +1,75 @@
+#!/usr/bin/env bash
+
+set -eux
+
+echo "PARAM_RELEASE: $PARAM_RELEASE"
+echo "PARAM_DIST: $PARAM_DIST"
+
+if [ -z "$PARAM_RELEASE" ]; then
+ echo "Env variable PARAM_RELEASE is not set! For example PARAM_RELEASE=8, for CentOS 8"
+ exit 1
+elif [ -z "$PARAM_DIST" ]; then
+ echo "Env variable PARAM_DIST is not set! For example PARAM_DIST=centos"
+ exit 1
+fi
+
+yum install -y epel-release
+
+if [ "$PARAM_DIST" = centos ]; then
+ if [ "$PARAM_RELEASE" = 8 ]; then
+ dnf install -y 'dnf-command(config-manager)'
+ yum config-manager --set-enabled PowerTools
+ fi
+fi
+
+yum install -y rpm-build yum-utils
+
+export DIST_DIR=build
+
+cd /varnish-cache
+rm -rf $DIST_DIR
+mkdir $DIST_DIR
+
+
+echo "Untar redhat..."
+tar xavf redhat.tar.gz -C $DIST_DIR
+
+echo "Untar orig..."
+tar xavf varnish-*.tar.gz -C $DIST_DIR --strip 1
+
+echo "Build Packages..."
+if [ -e .is_weekly ]; then
+ WEEKLY='.weekly'
+else
+ WEEKLY=
+fi
+VERSION=$("$DIST_DIR"/configure --version | awk 'NR == 1 {print $NF}')$WEEKLY
+
+cp -r -L "$DIST_DIR"/redhat/* "$DIST_DIR"/
+tar zcf "$DIST_DIR.tgz" --exclude "$DIST_DIR/redhat" "$DIST_DIR"/
+
+RPMVERSION="$VERSION"
+
+RESULT_DIR="rpms"
+CUR_DIR="$(pwd)"
+
+rpmbuild() {
+ command rpmbuild \
+ --define "_smp_mflags -j10" \
+ --define "_sourcedir $CUR_DIR" \
+ --define "_srcrpmdir $CUR_DIR/${RESULT_DIR}" \
+ --define "_rpmdir $CUR_DIR/${RESULT_DIR}" \
+ --define "versiontag ${RPMVERSION}" \
+ --define "releasetag 0.0" \
+ --define "srcname $DIST_DIR" \
+ --define "nocheck 1" \
+ "$@"
+}
+
+yum-builddep -y "$DIST_DIR"/redhat/varnish.spec
+rpmbuild -bs "$DIST_DIR"/redhat/varnish.spec
+rpmbuild --rebuild "$RESULT_DIR"/varnish-*.src.rpm
+
+echo "Prepare the packages for storage..."
+mkdir -p packages/$PARAM_DIST/$PARAM_RELEASE/
+mv rpms/*/*.rpm packages/$PARAM_DIST/$PARAM_RELEASE/
More information about the varnish-commit
mailing list