Commit graph

2 commits

Author SHA1 Message Date
Stan Hu
b90f2f9d79
Fix gsutil rsync supported args
This fixes a problem with -x <args> no longer worked. Seen in
https://gitlab.com/gitlab-com/www-gitlab-com/-/jobs/1946099171.
2022-01-05 20:20:54 -08:00
Stan Hu
519554013c
Add custom www-gitlab-com image with patched gsutil rsync command
Currently the `gsutil rsync` command does not support the `-z` or `-Z`
options available in `gsutil cp` to compress files locally via gzip
before uploading
(https://github.com/GoogleCloudPlatform/gsutil/issues/579). As
https://cloud.google.com/storage/docs/gsutil/commands/cp states:

When you specify the -z option, the data from your files is
compressed before it is uploaded, but your actual files are left
uncompressed on the local disk. The uploaded objects retain the
Content-Type and name of the original files, but have their
Content-Encoding metadata set to gzip to indicate that the object data
stored are compressed on the Cloud Storage servers and have their
Cache-Control metadata set to no-transform.

about.gitlab.com is currently serving uncompressed HTML files because
`Cache-Control: max-age=0` is set (see
https://gitlab.com/gitlab-com/www-gitlab-com/-/merge_requests/87045),
and Fastly has a custom rule to skip HTML files and therefore won't
cache them.

This patches the `rsync.py` using
https://github.com/GoogleCloudPlatform/gsutil/pull/1430 to support these
command-line options so local gzip compression can be performed.

Relates to
https://gitlab.com/gitlab-com/gl-infra/infrastructure/-/issues/14852
2022-01-05 08:37:26 -08:00