Adventures in OpenStack – On-Disk Anatomy of an Instance

When an instance is set up and started on OpenStack, the base image will be copied over and stored the local compute node from Glance. Here is what it looks like on disk, from a test instance of mine:

# ls -al 
total 65628
drwxr-xr-x. 2 nova nova       69 Feb 12 23:30 .
drwxr-xr-x. 5 nova nova       93 Feb 12 23:30 ..
-rw-rw----. 1 root root    14796 Feb 13 00:17 console.log
-rw-r--r--. 1 root root 67174400 Feb 13 00:17 disk
-rw-r--r--. 1 nova nova       79 Feb 12 23:29
-rw-r--r--. 1 nova nova     2629 Feb 12 23:43 libvirt.xml

And checking the disk file, we see the following:

# qemu-img info disk
image: disk
file format: qcow2
virtual size: 5.0G (5368709120 bytes)
disk size: 64M
cluster_size: 65536
backing file: /var/lib/nova/instances/_base/acc45e1bf0ad2336505dcad48418ce2564b701c4
Format specific information:
    compat: 1.1
    lazy refcounts: false

Also, while the user instance disk is qcow2, the backing file is *not*:

image: /var/lib/nova/instances/_base/acc45e1bf0ad2336505dcad48418ce2564b701c4
file format: raw
virtual size: 2.0G (2147483648 bytes)
disk size: 1.1G

Remember that instance storage for Nova needs to be engineered properly: /var/lib/nova/instances could be local, but should be shared or replicated storage to ensure proper availability levels.