In one of my previous posts, I showed how one can get started with Libvirt and KVM. This virtualization stack is meant not to be used as a desktop virtualization software, rather it is meant to run on servers offering greater flexibility, efficiency and stability, instead of user-friendliness. It is meant be automated to the n^th degree rather than relying on manual configuration. So let's see how you can connect to your libvirt daemon and automate basic VM management and monitoring using Python.

The Setup and Basic Assumptions

I am using a Libvirt KVM installation on a Debian server. The Python scripts I will be using are running in a Python 3.7.3 environment. This article is supposed to get your feet wet with Libvirt's Python bindings, when you are designing your application you should always refer to the official documentation which cover a wide range of use cases and are updated reasonably often.

Let's install all the dependencies required for libvirt first:

$ sudo apt install  pkg-config libvirt-dev
$ pip3 install libvirt-python

That's all the packages you need.

The following scripts and snippets are run locally on the Libvirt host, as root, rather than being run on a remote client. You can access the services remotely, however, that would require a long digression into securing the connection between the client and the server. Therefore, we will be connecting locally, for simplicity's sake.

Establishing Connection with the Libvirtd service

To get started, let's open a Python prompt, import the libvirt library and open a connection with the libvirt.open method.

root@deb:~# python3
Python 3.7.3 (default, Apr 15 2019, 01:55:37)
[GCC 6.3.0 20170516] on linux

Type “help”, “copyright”, “credits” or “license” for more information.

>>> import libvirt
>>> conn = libvirt.open('qemu:///system')

The variable conn can now be used to query your libvirt daemon and we will do that shortly. But first, a little digression.

Libvirt can be used to manage a number of different virtualization and containerization stack. KVM-QEMU, Xen and LXC are the most popular of these. So when you enter libvirt.open('qemu:///system') libvirt enables you to gather information about, and manage, QEMU guests. You can just as well, talk to LXD daemon or Xen hypervisor using lxc:///system or xen:///system respectively.

Similarly, the method libvirt.open() is not the only one at your disposal. open(name), openAuth(uri, auth, flags) and openReadOnly(name) are three different calls each of which returns a virConnect object and offers varying level of control over the host. You can read more about them here. For now, we have conn as an object of the virConnect class. This object is a gateway for doing almost anything from configuring the hypervisor itself to modifying the guests and their resource allocation.

Once you are done working with the object, make sure to close the connection by calling the close method on it.

>>> conn.close()

However, don't run the above command, just yet. Because we will play around with libvirt a bit more. Let's ask our hypervisor a few details about itself, like the hostname, and the number of vCPUs that it can offer to guest VMs in total.

>>> conn.getHostname()
'deb'
>>> conn.getMaxVcpus('qemu')
16

Now, we need to understand that with Libvirt metadata about objects like hypervisor stats, VMs, their networking and storage info, etc are all represented in XML format. XML is sorta like JSON only a bit clumsier (and a bit older). The data is stored and presented as a string literal and what that means is that if you query libvirt and the output of that query is XML you will get a really long single line output with '\n' present as a literal string rather than a new line. Python's built-in print function can clean it up for human readability

>>> print(conn.getSysinfo())


Dell Inc.
A14
…
 

Listing and Monitoring VMs

If you are maintaining a large array of VMs you need a method to create hundreds of VMs with uniform configuration which also scale properly from simple single threaded workloads to multi-core, multi-threaded processing. Libvirt calls the guest VMs (or containers if you are using LXC) Domains and you can list information about individual domains as well as configure them if your virConnect object has sufficient privileges.

To get information about the VMs and their resource utilization you can use the following calls:

>>> conn.listDomainsID()
[4, 5]

This returns an array of domain IDs which are just small integers for a simple libvirt setup. A more reliable way of labeling your VMs, without having two VMs (let's say on different nodes) with the same ID or name, is to use UUIDs. In libvirt everything can have a UUID, which is randomly generated 128 bit number. The chances of you creating two identical UUID are quite small indeed.

The network for your Virtual Machines, the VMs themselves and even the storage pools and volumes have their individual UUIDs. Make liberal use of them in your Python code, instead of relying on human assigned names. Unfortunately, the way to get the UUIDs of domains is a bit messy in the current implementation of this library, in my opinion. It does require you to supply the ID of the VM (the domain ID), here is how it looks.

domainIDs = conn.listDomainsID()
for domainID in domainIDs:
domain = conn.lookupByID()
uuid = domain.UUIDString()
print(uuid)

Now you can see the list of domain UUIDs. We have also stumbled across a new Python Object libvirt.virDomain, which, has its own set of methods associated with it much like the variable conn which was a libvirt.virConnect object and had methods like listDomainsID() and lookupByID() associated with it.

For both these methods you can use Python's built-in dir() methods so that the objects can list their internal variables and methods.

For example:

>>> dir(conn)
['_… gs', 'schedulerType', 'screenshot', 'securityLabel', 'securityLabelList',
'sendKey', 'sendProcessSignal', 'setAutostart', 'setBlkioParameters', 'setBlockIoTune',
'setGuestVcpus','setInterfaceParameters', 'setMaxMemory', 'setMemory', 'setMemoryFlags',
'setMemoryParameters','setMemoryStatsPeriod', 'setMetadata', 'setNumaParameters',
'setPerfEvents', 'setSchedulerParameters','setSchedulerParametersFlags', 'setTime',
'setUse'… ]

This can really help you recall quickly the exact name of a method and the object it ought to be used with. Now that we have a libvirt.virDomain object, let's use it to list various details about this running VM.

>>> domain.info()

This gives you the information regarding the state of the VM, maximum memory and cpu cores as shown here.

You can also find other information about the VM using different methods like OSType()

>>> domain.OSType()
'hvm'

There's a lot of flexibility when it comes to the API that libvirt exposes and you only have to worry about your use case and without worrying about the enormous complexity that libvirt handles.

Conclusion

In my voyages into the Libvirt technology, the absence of UUIDs as a first class citizen was probably the only pain point that I faced which seemed like a bad design choice. Other than that, libvirt is pretty nifty for what it accomplishes. Yes, there are a lot of other things that could have been done in a better way, but that's always the case with software. In hindsight, bad decisions are always obvious but the cost of rewriting a piece of software, as widespread as libvirt, is often tremendous.

A lot has been built on top of it, as the project as evolved slowly and steadily.

Instead of trying to learn the entire library at once, I would recommend come up with a small project or an idea and implement that  using Python and Libvirt. The documentation is pretty extensive with a lot of examples and it really forces you to think about proper software design and virtualization stack at the same time.