I recently ran into a little issue trying to execute a startup script on
Google’s Container Optimized OS (COS). Running a startup script
startup-script worked fine. For some reason, no matter the contents, anything specified in
failed to execute; and it seemed to be only COS that is affected.
Trawling the log data
Thankfully, all the startup information is logged in the OS’ journal. Trawling through the logs (using the command
I came across the following few lines:
startup-script: INFO Starting startup scripts. startup-script: INFO Found startup-script-url in metadata. startup-script: WARNING gsutil is not installed, cannot download items from Google Storage. startup-script: INFO No startup scripts found in metadata. startup-script: INFO Finished running startup scripts.
Notice the line containing
WARNING. I had naively thought that the
gsutil binaries were already bundled
inside COS (as they are with other images). It turns out this isn’t the case.
Attempting to fix it
gsutil isn’t available, my first thought was to reference the GCS URL for the startup script: something
https://storage.cloud.google.com/test-bucket-name/startup.sh. However, it appears this still requires the use
gsutil to download the contents.
Making use of the public URL (such as
https://storage.googleapis.com/test-bucket-name/startup.sh) seems to circumvent
the need to use
gsutil. The log entries below show how GCE is able to fetch a publicly accessible startup script (Simply
pulling the latest version of the Alpine Docker image as a test):
startup-script: INFO Starting startup scripts. startup-script: INFO Found startup-script-url in metadata. startup-script: INFO Downloading url from https://storage.googleapis.com/test-bucket-name/startup.sh to /var/lib/google/startup-aXNBqI/tmpyUFrNE. startup-script: INFO startup-script-url: + docker pull alpine:3.6 startup-script: INFO startup-script-url: 3.6: Pulling from library/alpine startup-script: INFO startup-script-url: 88286f41530e: Pulling fs layer startup-script: INFO startup-script-url: 88286f41530e: Verifying Checksum startup-script: INFO startup-script-url: 88286f41530e: Download complete startup-script: INFO startup-script-url: 88286f41530e: Pull complete startup-script: INFO startup-script-url: Digest: sha256:f006ecbb824d87947d0b51ab8488634bf69fe4094959d935c0c103f4820a417d startup-script: INFO startup-script-url: Status: Downloaded newer image for alpine:3.6 startup-script: INFO startup-script-url: Return code 0. startup-script: INFO Finished running startup scripts.
If you have no sensitive information in your startup scripts and are happy for your startup scripts to be publicly accessible, then this is your solution.
If you don’t want to leave your startup scripts as publicly available, then continue reading for some ways of securing them.
Securing the startup script
There are a few ways of securing your startup scripts. Ideally, the best way would be to not include any sensitive data in your startup scripts (the metadata server is available, which could be used to keep sensitive data out of your scripts) - thus removing the requirement for them to be secured.
However, this is not always possible. So, here are three methods that I know of that possibly be used to secure your startup scripts:
1. Host the startup scripts on an external server
If you spin up a separate server, you could store the startup scripts on this server. Using something like NGiNX’s secure link module, you are able to ensure that a specific token is provided before accessing your startup scripts.
The downside to this is that you need a separate and standalone server to host these scripts.
2. Host the startup scripts on an internal server
Pretty much the same as the previous suggestion, with one difference. Ensure that the server hosting the startup scripts has no external IP address assigned (assuming this is a GCE instance in the same project), and reference the download URL using the internal project hostname.
Side note: I haven’t tested this method, but I don’t see why it wouldn’t work.
gsutil to generate a signed URL
gsutil command has the ability to generate a signed URL. This is basically a URL that is pubcliy valid for a defined
period of time, but only if you know the full URL.
The following example shows how a GCS object is signed, and publicly available via GET requests for the next two minutes:
gsutil signurl -d 2m -m GET path/to/serviceAccount.json gs://test-bucket-name/startup.sh
This generates a signed URL like the following:
Using this method, you’re able to ensure your startup script is publicly available only for those with the direct link and those who access it within the specified time period.
As you can see, this is a very manual process. However, there’s nothing stopping this from being automated.
I hope this helps in ensuring your COS images are able to make use of the
startup-script-url metadata keys in Google
Compute Engine. If you have any additional ideas on how to secure your startup script URLs, please drop a comment below.