In my previous post of this small series, titled Security for your virtual machines: what is KMIP?, I talked about the new generation of the main hypervisors, VMware vSphere 6.5 and Microsoft Hyper-V 2016, and how they both introduced new encryption capabilities for virtual machines. I described the underlying technology used by VMware, KMIP; it’s not time to implement it in my lab and see how it interacts with data protection.

1. Install the KMIP server

Once your environment is updated to vSphere 6.5 for both vCenter and the ESXi servers, you can connect vCenter itself as a client of a KMIP server. As I explained in my previous post, the free pyKMIP solution IS NOT a safe technology to be used in a production environment as keys are only stored in memory, and they are deleted every time the service is restarted. However, it’s free and easy to be deployed, so it makes a lot of sense for a lab like mine.

William Lam created an excellent post (you can read it here), so I’m not going to repeat . If you need to install and configure docker in a CentOS 7 machine, here are the commands you need. First, install the docker package:

yum install docker

Then, check if the service is installed, and enable it:

systemctl list-unit-files | grep docker
 docker-storage-setup.service                  disabled
 docker.service                                disabled

systemctl enable docker.service
systemctl start docker.service

And you will finally have docker service running in the machine. You can check its status using:

systemctl status docker.service

and see a green line that says active (running). Then, pull down the pyKMIP package created by William and run it as a service.

docker pull lamw/vmwkmip
docker run -d -p 5696:5696 lamw/vmwkmip

I prefer to run the pyKMIP as a service and not in interactive mode, as i’m working remotely in my lab via ssh and issues can happen to my connection.

2. Configure vCenter 6.5 to use the KMIP server and cypher a test VM

Once the KMIP service is up and running, you need to configure vCenter to become a KMIP client, by connecting it to the pyKMIP service. To do so, go in the web client into VCSA > Configure > Key Management Server, and choose to add a KMS. Fill the needed information by creating a new cluster and input the network information of the pyKMIP service (default port is 5696, and you can see it running docker ps:

Once you have accepted the trust certificate, the KMIP server is registered and ready to be used:

Now, I choose a test VM, and by editing its Storage Policies, I apply the pre-configured “VM Encryption Policy” that makes the VM itself encrypted:

After the encryption is completed (it may take a while) the new status will be clearly listed in the VM summary page:

3. Backup of a cyphered VM

Things are going to change a little bit when you need to protect a cyphered VM. First, obviously you need to have a software that is compatible with vSphere 6.5. For Veeam Backup & Replication, you need to use version 9.5 Update 1 (build 9.5.0.823). But you also need to configure your backup infrastructure properly. If you want to do backups using network mode, that is by retrieveing data over the ESXi management network, nothing is going to change, backups would simply work:

Sometimes however network mode can be slow, especially if the ESXi servers only have 1GB connection, or if this connection is shared with other traffi types. For this reason, administrators may prefer to use different methods. First of all, sorry by Direct Storage doesn’t work, as data has to be read through the ESXi storage stack in order to be decyphered and cannot be retrieved directly from the storage array. This means that the only other available method is hotadd. But in order to use hotadd, there are some requirements that need to be satisfied: first, the VM that needs to be protected needs to have its disks connected via SCSI and not IDE, but also the virtual proxy needs to be encrypted using the same KMS server, otherwise you may hit this error:

If you cypher the virtual proxy, and try again to run the job, it will complete using this time hotadd instead of network mode:

One last note: the status of the VM disks as they leave the hypervisor is NON ENCRYPTED, as the blocks are read and unencrypted by ESXi and passed to the backup proxy. For this reason, if you want to keep the data of the virtual machine in an encrypted state, you should also enable the encryption options of your backup software. In Veeam, this is done on a job bases, by editing its configuration.