E3-1268L GPU Passthrough not working, Redhat KVM and Supermicro Motherboard

[DNaug1]

If anyone has any suggestions for this, I would appreciate it. I'm trying to get GPU (PCI) passthrough working on a E3-1286L with a Supermicro X10SLQ motherboard. From what I can tell, the platform does not meet the RMRR requirements for passing through the on board graphics (to windows running in a VM). The KVM is running fine, and there a multiple machines running. The problem is with the GPU passthrough. Has anyone been down this road?

Supermicro Product Information (and basic BIOS information):

X10SLV-Q

Version: 2.0a

Build Date: 7/10/2014

Memory: 16384MB (DDR3)

VT-d [Enabled]

Intel Virtualization Technology [Enabled]

Problem Description:

GPU Passthrough does not work correctly in the linux KVM:

Error message:

vfio-pci 0000:00:02.0: Device is ineligible for IOMMU domain attach due to platform RMRR requirement. Contact your platform vendor.

I was using the following for documentation:

https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Virtualization_Deployment_and_Administration_Guide/chap-Guest_virtual_machine_device_configuration.html https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Virtualization_Deployment_and_Administration_Guide/chap-Guest_virtual_machine_device_configuration.html

Comments from Redhat Engineering on the problem:

The platform vendor has the ability to request that the VT-d IOMMU subsystem within the kernel retain specific mappings for devices using entries in the Advanced Configuration and Power Interface Direct Memory Access Remapping (ACPI DMAR) table known as Reserved Memory Region Reporting (RMRR) structures. However, QEMU-KVM and VFIO have no visibility to these mapping requirements and no API exists to disable any potential ongoing communication that may occur through these regions. Therefore, a device associated with an RMRR could continue to use DMA through this address space even after the device was assigned to a guest VM. This could cause a device to overwrite VM memory with DMA data intended for the memory described by the RMRR. To fix this bug, devices with associated RMRRs are excluded from participating in the kernel internal IOMMU API. Users can now identify such devices using dmesg logs and are also protected from assignment of devices making use of mappings which have the ability to cause instability within guest VMs. Users prevented from making use of PCI device assignment as a result of this change should contact their platform vendor for a BIOS update to release the I/O device from the imposed RMRR requirement. (BZ# https://bugzilla.redhat.com/1097907 1097907)

- RHSA-2014:0290 - Moderate: kernel bug fix and enhancement update - Red Hat Customer Portal

https://access.redhat.com/articles/1352803 https://access.redhat.com/articles/1352803"

Operating System Information:

Kernel : 3.10.0-229.el7.x86_64

KVM: libvirt-1.2.8-16.el7_1.3.x86_64

[Hans_B_Intel]

Perhaps somebody here can share their experiences, but I would suggest that you contact super micro directly? As per the Comments from Redhat Engineering that you posted, they suggest you contact the vendor for a BIOS update that can correct this issue.

[DNaug1]

Thanks Hansb. I have reached out to Supermicro as well.

My only thoughts were that the BIOS code that Supermicro used was probably a cut and paste from the BIOS they received from Intel for the reference design. So it could takes weeks working it's way through Supermicro, only to find someone that says "It was like that when I found it". (probably in Mandarin).

I guess if anyone at Intel could confirm that the feature works in the first place, that would be a place to start. Thanks though.