As stated in a forum post by Anton Gostev, Veeam Backup & Replication 5.0.2 has still some incompatibilities with the new vSphere 5:

• Incremental replication passes to ESXi 5 target fail (initial pass works normally). • Backup in direct SAN access mode will fail for VM disks on VMFS5 volumes. • SureBackup is not functional when ESXi 5 is used to host the virtual lab. • Multi-OS FLR is not functional when ESXi 5 is used to run the FLR helper virtual appliance. • vCenter Server 5.0 causes job to fails with the “Inactive datastore” error. Jobs on standalone ESXi 5 More >