This morning I was setting up some test instances on my home OpenStack deployment and ran into some issues. The first instance launched okay, but the second failed, telling me that no suitable host was found.
I have not really oversubscribed, or so I thought. Turns out I was oversubscribed on both RAM and Disk.
OpenStack has defaults surrounding this:
* Disk is not oversubscribed by default. There is a 1-1 subscription per host. In a production setup, this should never be changed – an instance launching on a host that could run out of space and take down all the instances on that node is definitely a bad thing.
* Memory is oversubscribed 1.5 times.
* CPU is oversubscribed 16 times (cores).
Disk and memory allocation are not so much as critical on standalone setups. These will inevitably need changing as more instances are stood up as OpenStack still counts shut down instances towards the limit, even for RAM and CPU.
Changing the defaults
The defaults be changed: just edit your Nova configuration (ie:
/etc/nova/nova.conf) and look for:
disk_allocation_ratio for disk
ram_allocation_ratio for Memory
cpu_allocation_ratio for CPU.
Change these limits as you see fit and then just restart Nova (ie:
openstack-service restart nova).
When an instance is set up and started on OpenStack, the base image will be copied over and stored the local compute node from Glance. Here is what it looks like on disk, from a test instance of mine:
# ls -al
drwxr-xr-x. 2 nova nova 69 Feb 12 23:30 .
drwxr-xr-x. 5 nova nova 93 Feb 12 23:30 ..
-rw-rw----. 1 root root 14796 Feb 13 00:17 console.log
-rw-r--r--. 1 root root 67174400 Feb 13 00:17 disk
-rw-r--r--. 1 nova nova 79 Feb 12 23:29 disk.info
-rw-r--r--. 1 nova nova 2629 Feb 12 23:43 libvirt.xml
And checking the disk file, we see the following:
# qemu-img info disk
file format: qcow2
virtual size: 5.0G (5368709120 bytes)
disk size: 64M
backing file: /var/lib/nova/instances/_base/acc45e1bf0ad2336505dcad48418ce2564b701c4
Format specific information:
lazy refcounts: false
Also, while the user instance disk is qcow2, the backing file is *not*:
file format: raw
virtual size: 2.0G (2147483648 bytes)
disk size: 1.1G
Remember that instance storage for Nova needs to be engineered properly: /var/lib/nova/instances could be local, but should be shared or replicated storage to ensure proper availability levels.