mirror of
https://ops.gitlab.net/gitlab-org/gitlab-build-images.git
synced 2025-12-09 10:02:56 +01:00
519554013c
Currently the `gsutil rsync` command does not support the `-z` or `-Z` options available in `gsutil cp` to compress files locally via gzip before uploading (https://github.com/GoogleCloudPlatform/gsutil/issues/579). As https://cloud.google.com/storage/docs/gsutil/commands/cp states: When you specify the -z option, the data from your files is compressed before it is uploaded, but your actual files are left uncompressed on the local disk. The uploaded objects retain the Content-Type and name of the original files, but have their Content-Encoding metadata set to gzip to indicate that the object data stored are compressed on the Cloud Storage servers and have their Cache-Control metadata set to no-transform. about.gitlab.com is currently serving uncompressed HTML files because `Cache-Control: max-age=0` is set (see https://gitlab.com/gitlab-com/www-gitlab-com/-/merge_requests/87045), and Fastly has a custom rule to skip HTML files and therefore won't cache them. This patches the `rsync.py` using https://github.com/GoogleCloudPlatform/gsutil/pull/1430 to support these command-line options so local gzip compression can be performed. Relates to https://gitlab.com/gitlab-com/gl-infra/infrastructure/-/issues/14852
57 lines
1.8 KiB
Bash
Executable file
57 lines
1.8 KiB
Bash
Executable file
#!/bin/bash
|
|
|
|
set -xeuo pipefail
|
|
IFS=$'\n\t'
|
|
|
|
export DEBIAN_FRONTEND=noninteractive
|
|
|
|
# echo "deb http://deb.debian.org/debian testing main" | tee -a /etc/apt/sources.list.d/testing.list
|
|
# Install packages
|
|
apt-get update
|
|
apt-get install -yq --no-install-recommends \
|
|
make gcc g++ locales \
|
|
rsync git-core \
|
|
ed file curl gnupg2 \
|
|
unzip \
|
|
python3 python3-pip python3-crcmod python-minimal \
|
|
patch
|
|
|
|
# Install Imagemagick for cropping the pictures on the team page
|
|
apt-get install -yq --no-install-recommends imagemagick
|
|
|
|
# Install node & yarn
|
|
NODE_INSTALL_VERSION=12.22.1
|
|
YARN_INSTALL_VERSION=1.22.10
|
|
/scripts/install-node $NODE_INSTALL_VERSION $YARN_INSTALL_VERSION && node --version && yarn --version
|
|
|
|
# Install yamllint
|
|
# We need version 1.25.0+: https://github.com/adrienverge/yamllint/blob/master/CHANGELOG.rst
|
|
YAMLLINT_VERSION=1.25.0
|
|
|
|
# Temporarily pin pyyaml and pathspec to reduce compatibility with packages from Debian bullseye (testing)
|
|
pip3 install yamllint==${YAMLLINT_VERSION} pyyaml==5.4.1 pathspec==0.8.1
|
|
|
|
# Install gitlab-runner
|
|
curl -O -J -L https://gitlab-ci-multi-runner-downloads.s3.amazonaws.com/latest/binaries/gitlab-ci-multi-runner-linux-amd64
|
|
mv gitlab-ci-multi-runner-linux-amd64 /usr/bin/gitlab-runner-helper
|
|
chmod +x /usr/bin/gitlab-runner-helper
|
|
|
|
# Patch gsutil to support gzip compression with rsync command:
|
|
# https://github.com/GoogleCloudPlatform/gsutil/pull/1430
|
|
if [[ -d "/patches/gsutil" ]]; then
|
|
for i in /patches/gsutil/*.patch; do
|
|
echo "$i..."
|
|
patch -d /usr/lib/google-cloud-sdk/platform/gsutil -p1 -i "$i"
|
|
done
|
|
fi
|
|
|
|
# Set UTF-8
|
|
echo "C.UTF-8 UTF-8" > /etc/locale.gen
|
|
locale-gen
|
|
update-locale LANG=C.UTF-8 LC_CTYPE=C.UTF-8 LC_ALL=C.UTF-8
|
|
locale -a
|
|
|
|
# Clean up
|
|
apt-get autoremove -yq
|
|
apt-get clean -yqq
|
|
rm -rf /var/lib/apt/lists/*
|