E-mail comments

Download an `image`

NOTE:

At the current stage it is not possible to download images directly from the user interface. However, the instructions here allow you to fetch a copy of an image of yours off the HPC Cloud at your own pace.

If you find yourself struggling to succeed in following these instructions, you may ask the HPC Cloud operators to try to make an image available for you by sending an e-mail to our helpdesk@surfsara.nl.

You can download a copy of an image by attaching that image as if it were just another storage drive to a running VM. The following overview outlines the steps we will be following throughout this guide. We will be:

Making sure the image you want to download is not in use by any VM
Starting a VM without using the image you want to download (for example, create a new one from the AppMarket) Let’s call this Utility VM
Attaching the image you want to download to the Utility VM
Making a file off the image you want to download
Downloading the file you just created
Clearing your footprints

Each section now goes into more detail for each of the previous steps.

Preparing the image for download

You must make sure that the image you are willing to download is currently not in use by any running VM.

On the UI: click on the image you want to download to display the extended information about that image.
On the UI: Then click on the tab VMs of the extended information, so that you can see the list of all VMs that are using the image.
On the UI: For every VM that you see on the list of VMs that are using the image, click on that VM’s line to see extended information about that VM, and shut that VM down.
On the UI: Go back to the Info tab of the image’s extended information. Change now the Type to DATABLOCk.

Creating a utility VM

We will be using a VM as the central place of operations for the rest of the tasks. We will call it Utility VM.

We will be creating a file in the Utility VM containing all the contents of the image you want to download. That file is what you will download in the end. Therefore, the Utility VM needs to have enough space to host that file.

NOTE:

Because the image you want to download is likely larger than the average space available in an Apps appliance sole image, we will attach an extra Ceph Datablock to the Utility VM so that we can create the large file with the contents of the image in there. This means, in turn, that you must have enough quota available in the Ceph datastore. You can view your current quotas on the UI itself. From the User view, click on the Storage tab from the main menu on the left, and then Datastores under it. You should see there a list with all the datastores you have access to. If the quota you see there is not enough to fit a whole copy of the image you want to download, then please get in contact with us: helpdesk@surfsara.nl.

On the UI: Go to the Apps option in the Storage section and import an appliance you feel comfortable with (detailed instructions in our General Start article). We will be using CentOS 7 throughout this guide.
On the UI: Create a new empty datablock image in the Ceph datastore that is big enough to fit a copy of your image (detailed instructions in our Datablocks article). Let’s give this datablock image the name wagon.
On the UI: Make sure that you edit the template you have just imported from the appliance so that you:
- in the Storage tab: include a disk with the newly created empty datablock image (the wagon).
- in the Network tab: include a nic connected to the Internet
On the UI: Save the changes to the template you just edited
On the UI: Instantiate the template. Make sure you can log in to the Utility VM via SSH.

Attaching the image to be downloaded

We will now hot-attach the image you want to download to the Utility VM. But first we need to know which drives you currently see in the Utility VM, so that we can more easily identify each of them.

In your laptop: Connect via SSH to the Utility VM
In the Utility VM: become the root user. Run the command: fdisk -l. You can probably see plenty of output, and somewhere around there, two lines (not necessarily following one another) will read something like: Disk /dev/vda: ... and Disk /dev/vdb: .... These indicate that you have two drives; namely: the Operating System disk is one, and the empty wagon datablock is the other one. Probably, looking at their sizes, you can easily identify which is which. For the rest of this guide, let’s assume the following (your case may just as well be the opposite):
- /dev/vda is the Operating System image (the Utility VM’s Operating System, remember; it is not the one you may be willing to download)
- /dev/vdb is the wagon datablock
We will have to format and mount the wagon datablock. Like this (for more details see our Datablocks article):
- In the Utility VM: Become root, with sudo su -
- In the Utility VM: mkdir /wagon
- In the Utility VM: mkfs -t xfs /dev/vdb
- In the Utility VM: mount /dev/vdb /wagon
- In the Utility VM: mkdir /etc/rc.d
- In the Utility VM: touch /etc/rc.d/rc.local
- In the Utility VM: echo "echo 4096 > /sys/block/vdb/queue/read_ahead_kb" > /etc/rc.d/rc.local
- In the Utility VM: chmod 755 /etc/rc.d/rc.local
On the UI: Go to the VMs option of the Instances section, and click on the Utility VM’s line so that you can see its extended information.
On the UI: Click on the Storage tab to see the disks currently attached to the VM.
On the UI: Click on the Attach disk green button to bring up the Attach new disk dialogue.
On the UI: On the Attach new disk dialogue, look for the image you want to download. Click on the image to select it. Then finally click on the Attach green button at the bottom of the dialogue. The dialogue will disappear and y our Utility VM will come into the HOTPLUG status and it will remain so until the newly attached disk is ready for use. Then the Utility VM will come back to status RUNNING.
On the UI: Wait until the Utility VM is back in status RUNNING.
We want to find and see the image you want to download in the Utility VM. Therefore:
- In the Utility VM: Become root, with sudo su -
- In the Utility VM: Run again fdisk -l. You should see a new drive in the listing from that command, likely: Disk /dev/vdc: .... For the rest of the guide we will assume that the image you want to download is located in /dev/vdc.

Making a file off the image to be downloaded

We will be making a bit-by-bit copy of the image you want to download, and storing that copy into a plain file that you can then download. We will be using the command dd which does all of that for you (you can read more in Wikipedia.

In the Utility VM: become the root user.
In the Utility VM: run the command to create the copy of the image into a file: dd if=/dev/vdc of=/wagon/image_file.img
In the Utility VM: Once the previous command finishes, verify that the file is properly created. You can issue a command that should return empty (or nothing) when comparing two files: diff /dev/vdc /wagon/image_file.img

Downloading the file

In your laptop: You can now download the file via SSH. For example, you can use scp or sftp. Like this: scp admincentos@145.100.XXX.YYY:/wagon/image_file.img.

Hint:

If you are acquainted with different virtualisation technologies and know about things such us the qcow2 format and tools like qemu, then you may want to convert the raw image you have just created into any other format you may like. It may be worth looking into this because using a format with compression may save some bandwith and download time.

Clearing your footprints

Once you have downloaded the image_file.img file, you can destroy all the scaffolding you rigged up to be able to download it. That means that you can:

Delete the Utility VM
Delete the template for the Utility VM
Delete the image for the Utility VM
Delete the wagon datablock image.

Further, if you do not need the image you wanted to download in the UI, then you can delete it now as well. If you still want to use it, however, you may want to change its type back to OS.