Convert image to raw format on Google Cloud
Contributors
Suggest changes
PDFs
The image being used to deploy new instances, upgrades, or being used in existing images will be shared with the clients through the NetApp Support Site (NSS). The signed digest, and the certificates will be available to download through the NSS portal. Make sure you are downloading the digest and certificates for the right release corresponding to the image shared by NetApp Support. For instance, 9.13.0 images will have a 9.13.0 signed digest and certificates available on NSS.
Why is this step needed?
The images from Google Cloud cannot be downloaded directly. In order to verify the image against the signed digest and the certificates, you need to have a mechanism to compare the two files and download the image. To do so, you must export/convert the image into a disk.raw format and save the results in a storage bucket on Google Cloud. The disk.raw file is tarred and gzipped in the process.
The user/service-account will need privileges to perform the following:
-
Access to Google storage bucket
-
Write to Google Storage bucket
-
Create cloud build jobs (used during export process)
-
Access to the desired image
-
Create export image tasks
To verify the image, it must be converted to a disk.raw format and then downloaded.
Use Google Cloud command line to export Google Cloud image
The preferred way to export an image to Cloud Storage is to use the gcloud compute images export command. This command takes the provided image and converts it to a disk.raw file which gets tarred and gzipped. The generated file is saved at the destination URL and can then be downloaded for verification.
The user/account must have privileges to access and write to the desired bucket, export the image, and cloud builds (used by Google to export the image) to execute this operation.
Export Google Cloud image using gcloud
Click to display the script
$ gcloud compute images export \
--destination-uri DESTINATION_URI \
--image IMAGE_NAME
# For our example:
$ gcloud compute images export \
--destination-uri gs://vsa-dev-bucket1/example-user-exportimage-gcp-demo \
--image example-user-20230120115139
## DEMO ##
# Step 1 - Optional: Checking access and listing objects in the destination bucket
$ gsutil ls gs://example-user-export-image-bucket/
# Step 2 - Exporting the desired image to the bucket
$ gcloud compute images export --image example-user-export-image-demo --destination-uri gs://example-user-export-image-bucket/export-demo.tar.gz
Created [https://cloudbuild.googleapis.com/v1/projects/example-demo-project/locations/us-central1/builds/xxxxxxxxxxxx].
Logs are available at [https://console.cloud.google.com/cloud-build/builds;region=us-central1/xxxxxxxxxxxx?project=xxxxxxxxxxxx].
[image-export]: 2023-01-25T18:13:48Z Fetching image "example-user-export-image-demo" from project "example-demo-project".
[image-export]: 2023-01-25T18:13:49Z Validating workflow
[image-export]: 2023-01-25T18:13:49Z Validating step "setup-disks"
[image-export]: 2023-01-25T18:13:49Z Validating step "image-export-export-disk"
[image-export.image-export-export-disk]: 2023-01-25T18:13:49Z Validating step "setup-disks"
[image-export.image-export-export-disk]: 2023-01-25T18:13:49Z Validating step "run-image-export-export-disk"
[image-export.image-export-export-disk]: 2023-01-25T18:13:50Z Validating step "wait-for-inst-image-export-export-disk"
[image-export.image-export-export-disk]: 2023-01-25T18:13:50Z Validating step "copy-image-object"
[image-export.image-export-export-disk]: 2023-01-25T18:13:50Z Validating step "delete-inst"
[image-export]: 2023-01-25T18:13:51Z Validation Complete
[image-export]: 2023-01-25T18:13:51Z Workflow Project: example-demo-project
[image-export]: 2023-01-25T18:13:51Z Workflow Zone: us-central1-c
[image-export]: 2023-01-25T18:13:51Z Workflow GCSPath: gs://example-demo-project-example-bkt-us/
[image-export]: 2023-01-25T18:13:51Z Example scratch path: https://console.cloud.google.com/storage/browser/example-demo-project-example-bkt-us/example-image-export-20230125-18:13:49-r88px
[image-export]: 2023-01-25T18:13:51Z Uploading sources
[image-export]: 2023-01-25T18:13:51Z Running workflow
[image-export]: 2023-01-25T18:13:51Z Running step "setup-disks" (CreateDisks)
[image-export.setup-disks]: 2023-01-25T18:13:51Z CreateDisks: Creating disk "disk-image-export-image-export-r88px".
[image-export]: 2023-01-25T18:14:02Z Step "setup-disks" (CreateDisks) successfully finished.
[image-export]: 2023-01-25T18:14:02Z Running step "image-export-export-disk" (IncludeWorkflow)
[image-export.image-export-export-disk]: 2023-01-25T18:14:02Z Running step "setup-disks" (CreateDisks)
[image-export.image-export-export-disk.setup-disks]: 2023-01-25T18:14:02Z CreateDisks: Creating disk "disk-image-export-export-disk-image-export-image-export--r88px".
[image-export.image-export-export-disk]: 2023-01-25T18:14:02Z Step "setup-disks" (CreateDisks) successfully finished.
[image-export.image-export-export-disk]: 2023-01-25T18:14:02Z Running step "run-image-export-export-disk" (CreateInstances)
[image-export.image-export-export-disk.run-image-export-export-disk]: 2023-01-25T18:14:02Z CreateInstances: Creating instance "inst-image-export-export-disk-image-export-image-export--r88px".
[image-export.image-export-export-disk]: 2023-01-25T18:14:08Z Step "run-image-export-export-disk" (CreateInstances) successfully finished.
[image-export.image-export-export-disk.run-image-export-export-disk]: 2023-01-25T18:14:08Z CreateInstances: Streaming instance "inst-image-export-export-disk-image-export-image-export--r88px" serial port 1 output to https://storage.cloud.google.com/example-demo-project-example-bkt-us/example-image-export-20230125-18:13:49-r88px/logs/inst-image-export-export-disk-image-export-image-export--r88px-serial-port1.log
[image-export.image-export-export-disk]: 2023-01-25T18:14:08Z Running step "wait-for-inst-image-export-export-disk" (WaitForInstancesSignal)
[image-export.image-export-export-disk.wait-for-inst-image-export-export-disk]: 2023-01-25T18:14:08Z WaitForInstancesSignal: Instance "inst-image-export-export-disk-image-export-image-export--r88px": watching serial port 1, SuccessMatch: "ExportSuccess", FailureMatch: ["ExportFailed:"] (this is not an error), StatusMatch: "GCEExport:".
[image-export.image-export-export-disk.wait-for-inst-image-export-export-disk]: 2023-01-25T18:14:29Z WaitForInstancesSignal: Instance "inst-image-export-export-disk-image-export-image-export--r88px": StatusMatch found: "GCEExport: <serial-output key:'source-size-gb' value:'10'>"
[image-export.image-export-export-disk.wait-for-inst-image-export-export-disk]: 2023-01-25T18:14:29Z WaitForInstancesSignal: Instance "inst-image-export-export-disk-image-export-image-export--r88px": StatusMatch found: "GCEExport: Running export tool."
[image-export.image-export-export-disk.wait-for-inst-image-export-export-disk]: 2023-01-25T18:14:29Z WaitForInstancesSignal: Instance "inst-image-export-export-disk-image-export-image-export--r88px": StatusMatch found: "GCEExport: Disk /dev/sdb is 10 GiB, compressed size will most likely be much smaller."
[image-export.image-export-export-disk.wait-for-inst-image-export-export-disk]: 2023-01-25T18:14:29Z WaitForInstancesSignal: Instance "inst-image-export-export-disk-image-export-image-export--r88px": StatusMatch found: "GCEExport: Beginning export process..."
[image-export.image-export-export-disk.wait-for-inst-image-export-export-disk]: 2023-01-25T18:14:29Z WaitForInstancesSignal: Instance "inst-image-export-export-disk-image-export-image-export--r88px": StatusMatch found: "GCEExport: Copying \"/dev/sdb\" to gs://example-demo-project-example-bkt-us/example-image-export-20230125-18:13:49-r88px/outs/image-export-export-disk.tar.gz."
[image-export.image-export-export-disk.wait-for-inst-image-export-export-disk]: 2023-01-25T18:14:29Z WaitForInstancesSignal: Instance "inst-image-export-export-disk-image-export-image-export--r88px": StatusMatch found: "GCEExport: Using \"/root/upload\" as the buffer prefix, 1.0 GiB as the buffer size, and 4 as the number of workers."
[image-export.image-export-export-disk.wait-for-inst-image-export-export-disk]: 2023-01-25T18:14:29Z WaitForInstancesSignal: Instance "inst-image-export-export-disk-image-export-image-export--r88px": StatusMatch found: "GCEExport: Creating gzipped image of \"/dev/sdb\"."
[image-export.image-export-export-disk.wait-for-inst-image-export-export-disk]: 2023-01-25T18:14:29Z WaitForInstancesSignal: Instance "inst-image-export-export-disk-image-export-image-export--r88px": StatusMatch found: "GCEExport: Read 1.0 GiB of 10 GiB (212 MiB/sec), total written size: 992 MiB (198 MiB/sec)"
[image-export.image-export-export-disk.wait-for-inst-image-export-export-disk]: 2023-01-25T18:14:59Z WaitForInstancesSignal: Instance "inst-image-export-export-disk-image-export-image-export--r88px": StatusMatch found: "GCEExport: Read 8.0 GiB of 10 GiB (237 MiB/sec), total written size: 1.5 GiB (17 MiB/sec)"
[image-export.image-export-export-disk.wait-for-inst-image-export-export-disk]: 2023-01-25T18:15:19Z WaitForInstancesSignal: Instance "inst-image-export-export-disk-image-export-image-export--r88px": StatusMatch found: "GCEExport: Finished creating gzipped image of \"/dev/sdb\" in 48.956433327s [213 MiB/s] with a compression ratio of 6."
[image-export.image-export-export-disk.wait-for-inst-image-export-export-disk]: 2023-01-25T18:15:19Z WaitForInstancesSignal: Instance "inst-image-export-export-disk-image-export-image-export--r88px": StatusMatch found: "GCEExport: Finished export in 48.957347731s"
[image-export.image-export-export-disk.wait-for-inst-image-export-export-disk]: 2023-01-25T18:15:19Z WaitForInstancesSignal: Instance "inst-image-export-export-disk-image-export-image-export--r88px": StatusMatch found: "GCEExport: <serial-output key:'target-size-gb' value:'2'>"
[image-export.image-export-export-disk.wait-for-inst-image-export-export-disk]: 2023-01-25T18:15:19Z WaitForInstancesSignal: Instance "inst-image-export-export-disk-image-export-image-export--r88px": SuccessMatch found "ExportSuccess"
[image-export.image-export-export-disk]: 2023-01-25T18:15:19Z Step "wait-for-inst-image-export-export-disk" (WaitForInstancesSignal) successfully finished.
[image-export.image-export-export-disk]: 2023-01-25T18:15:19Z Running step "copy-image-object" (CopyGCSObjects)
[image-export.image-export-export-disk]: 2023-01-25T18:15:19Z Running step "delete-inst" (DeleteResources)
[image-export.image-export-export-disk.delete-inst]: 2023-01-25T18:15:19Z DeleteResources: Deleting instance "inst-image-export-export-disk".
[image-export.image-export-export-disk]: 2023-01-25T18:15:19Z Step "copy-image-object" (CopyGCSObjects) successfully finished.
[image-export.image-export-export-disk]: 2023-01-25T18:15:34Z Step "delete-inst" (DeleteResources) successfully finished.
[image-export]: 2023-01-25T18:15:34Z Step "image-export-export-disk" (IncludeWorkflow) successfully finished.
[image-export]: 2023-01-25T18:15:34Z Serial-output value -> source-size-gb:10
[image-export]: 2023-01-25T18:15:34Z Serial-output value -> target-size-gb:2
[image-export]: 2023-01-25T18:15:34Z Workflow "image-export" cleaning up (this may take up to 2 minutes).
[image-export]: 2023-01-25T18:15:35Z Workflow "image-export" finished cleanup.
# Step 3 - Validating the image was successfully exported
$ gsutil ls gs://example-user-export-image-bucket/
gs://example-user-export-image-bucket/export-demo.tar.gz
# Step 4 - Download the exported image
$ gcloud storage cp gs://BUCKET_NAME/OBJECT_NAME SAVE_TO_LOCATION
$ gcloud storage cp gs://example-user-export-image-bucket/export-demo.tar.gz CVO_GCP_Signed_Digest.tar.gz
Copying gs://example-user-export-image-bucket/export-demo.tar.gz to file://CVO_GCP_Signed_Digest.tar.gz
Completed files 1/1 | 1.5GiB/1.5GiB | 185.0MiB/s
Average throughput: 213.3MiB/s
$ ls -l
total 1565036
-rw-r--r-- 1 example-user example-user 1602589949 Jan 25 18:44 CVO_GCP_Signed_Digest.tar.gzExtract zipped files
# Extracting files from the digest $ tar -xf CVO_GCP_Signed_Digest.tar.gz
|
See Google Cloud doc on Exporting an image for more information on how to export an image through Google Cloud. |