NcamGnrvngu/gitlab-build-images
1
0
Fork 0
mirror of https://ops.gitlab.net/gitlab-org/gitlab-build-images.git synced 2025-12-09 10:02:56 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions

Add custom www-gitlab-com image with patched gsutil rsync command

Browse source 
Currently the `gsutil rsync` command does not support the `-z` or `-Z`
options available in `gsutil cp` to compress files locally via gzip
before uploading
(https://github.com/GoogleCloudPlatform/gsutil/issues/579). As
https://cloud.google.com/storage/docs/gsutil/commands/cp states:

When you specify the -z option, the data from your files is
compressed before it is uploaded, but your actual files are left
uncompressed on the local disk. The uploaded objects retain the
Content-Type and name of the original files, but have their
Content-Encoding metadata set to gzip to indicate that the object data
stored are compressed on the Cloud Storage servers and have their
Cache-Control metadata set to no-transform.

about.gitlab.com is currently serving uncompressed HTML files because
`Cache-Control: max-age=0` is set (see
https://gitlab.com/gitlab-com/www-gitlab-com/-/merge_requests/87045),
and Fastly has a custom rule to skip HTML files and therefore won't
cache them.

This patches the `rsync.py` using
https://github.com/GoogleCloudPlatform/gsutil/pull/1430 to support these
command-line options so local gzip compression can be performed.

Relates to
https://gitlab.com/gitlab-com/gl-infra/infrastructure/-/issues/14852
This commit is contained in:
Stan Hu 2021-12-22 22:36:51 -08:00
parent 58a310f6ce
commit 519554013c
No known key found for this signature in database
GPG key ID: 8D3931AD39CC7A20
4 changed files with 131 additions and 1 deletions
Download patch file Download diff file Expand all files Collapse all files

12
scripts/install-www-gitlab-com
View file

@ -13,7 +13,8 @@ apt-get install -yq --no-install-recommends \
rsync git-core \
ed file curl gnupg2 \
unzip \
python3 python3-pip python3-crcmod python-minimal
python3 python3-pip python3-crcmod python-minimal \
patch
# Install Imagemagick for cropping the pictures on the team page
apt-get install -yq --no-install-recommends imagemagick
@ -35,6 +36,15 @@ curl -O -J -L https://gitlab-ci-multi-runner-downloads.s3.amazonaws.com/latest/b
mv gitlab-ci-multi-runner-linux-amd64 /usr/bin/gitlab-runner-helper
chmod +x /usr/bin/gitlab-runner-helper
# Patch gsutil to support gzip compression with rsync command:
# https://github.com/GoogleCloudPlatform/gsutil/pull/1430
if [[ -d "/patches/gsutil" ]]; then
for i in /patches/gsutil/*.patch; do
echo "$i..."
patch -d /usr/lib/google-cloud-sdk/platform/gsutil -p1 -i "$i"
done
fi
# Set UTF-8
echo "C.UTF-8 UTF-8" > /etc/locale.gen
locale-gen