E810 OCP Interrupt Test : failed

[dev246]

Hello, I have 4 identical servers with Intel E810-XXV-2 for OCP 2.0 and E810-C-Q2 network cards.

On all 4 servers i have fresh install of windows Server 2022, i install latest drivers 28.1 with latest firmware on both cards - 4.32(4.20).

On intel ProSet Adapter configuration utility i run hardware tests, on E810-C-Q2  card all test passed, on E810-XXV-2 for OCP 2.0  i got Interrupt Status : failed. On all 4 servers.

What can cause that only on the ocp card there is a problem with interrupt test ?, or maybe it is a false result and there is nothing to worry about

[B_Y]

Hello @dev246,

Thank you for posting in Intel Ethernet Communities. 

We are sorry to hear that you are facing Interrupt Status: failed issue when perform hardware tests on your Ethernet. For us to further check the issue, please provide the following details.

1. System Support Utility(SSU) log of your system. This will allow us to check your Adapter details and configuration. This would also help us identify if you are using an OEM or retail version of Intel Ethernet Adapter. Kindly refer to the steps below.

a- Go to https://downloadcenter.intel.com/download/25293/Intel-System-Support-Utility-for-Windows-?product=91600 and download the software.

b- Open SSU.exe

c- Mark the box "Everything" and then click "Scan".

d- When finished scanning, click "Next".

e- Click on "Save" and attach the file to a post.

2. What is the brand and model of the motherboard?

3. Do you encounter performance issue on your Ethernet or how does the Interrupt Status: Failed affects the performance on your Ethernet?

4. Do you notice the same issue if you run the hardware tests more than once on E810-XXV-2 for OCP 2.0? Did you try restart the system and try the hardware test again?

5. Which version of Intel ProSet Adapter configuration utility that you used to run hardware tests.

Looking forward to your reply.

Thank you.

Best regards,

BY_Intel

Intel Customer Support

[dev246]

2. Dell R750 server

3. It is fresh installation, non production yest so i don't know if i have Ethernet interrupts or performance issues.

4. Yes this same issue after run test more than once, after restart server, and on all of my 4 servers, and and issue occurs only on OCP E810 card not on clasic Pci-e E810 card.

5. version form 28.1 driver package.

[B_Y]

Hello @dev246,

Thank you for all the information. Please help us on the following:

1) May I know the link you downloaded from for the driver and firmware.

2) If you temporary disable the E810-C-Q2 from 1 of the servers, and run the Intel ProSet Adapter configuration utility on E810-XXV-2 for OCP 3.0 while also test on its network connection as well.

3) Is the E810-XXV-2 for OCP 3.0 came/purchase together as package with the a Dell R750?

4) Please share the screenshot of failed rom the Interrupt Test.

Thank you.

Best regards,

BY_Intel

Intel Customer Support

[dev246]

1. https://www.intel.com/content/www/us/en/download/15084/intel-ethernet-adapter-complete-driver-pack.html

2. If I disable E810-C-Q2 the test on OCP card is passed

3. No this is OEM part purchase separately

4. I attach screenshot from Intel Proset

[dev246]

I noticed a strange behavior with the E810-C-Q2 card, as I run the hardware test several times in a row, sometimes the same problem appears that always occurs on the OCP card, i.e. failed Interrupt Status after hardware test

[B_Y]

Hello @dev246,

Thank you for all the testings. Since it is fresh Microsoft Windows 2022 installation, it's best to check with the manufacturer as the software drivers linked from Intel page are generic versions, and can be used for general purposes. To avoid any potential installation incompatibilities on OEM system, Intel recommends checking with the OEM and use the software provided via system manufacturer. 

It is always recommended to update to latest bios (which you did, 1.10.2), latest chipset, and latest driver & firmware/Non-Volatile Memory (NVM) for NIC.

As you mentioned E810-XXV-2 for OCP 3.0 is from OEM and E810-C-Q2 was with Dell server?

Best regards,

BY_Intel

Intel Customer Support

[dev246]

As you mentioned E810-XXV-2 for OCP 3.0 is from OEM and E810-C-Q2 was with Dell server?

No both card are purchase separately from the server. Both are genuine intel network card, not intel brand by dell/hp/lenovo or other vendor.

[B_Y]

Hello @dev246,

Please send photos of the network cards on both sides for us to validate both of the cards: E810-XXV-2 for OCP 3.0 and E810-C-Q2.

Thank you.

Best regards,

BY_Intel

Intel Customer Support

[dev246]

Pictures attached

[B_Y]

Hello @dev246,

Please give us 1-3 business days to look into the matter internally. 

While we are at it, may I suggest to perform some network testing accordingly. Thank you.

Best regards,

BY_Intel

Intel Customer Support

[dev246]

Do you have any recommended network testing tools? So that the tests bring as much to the topic as possible and have a chance to show any problems. Since the network generally works, Pink goes through, file transfer between servers works, the card supports RDMA traffic. But I'm not able to check if it's doing it efficiently etc because I don't have much comparison of what it should look like

[B_Y]

Hello @dev246,

These should sufficient and ultimately based on your requirement. You may use iPerf3 - tool for active measurements of the maximum achievable bandwidth on IP networks. Please give us 1-3 business days to look into the matter: Interrupt Test : failed internally. 

Thank you.

Best regards,

BY_Intel

Intel Customer Support

[B_Y]

Hello @dev246,

Thank you for your patience.

We received some update that this should have been fixed in the Intel® Ethernet Adapter Complete Driver Pack 28.1.1. Good news that 28.2 was just released with more fix, please go to the link to download and test it:

https://www.intel.com/content/www/us/en/download/15084/intel-ethernet-adapter-complete-driver-pack.html

If it still fails, we would like some logs to further troubleshoot, here is the instruction to obtain them.

How to obtain QV SDK logs from PROSet:

1.      Install PROSet

2.      Go to registry location HKEY_LOCAL_MACHINE\SOFTWARE\INTEL\Network Services\NCS2 and set “Trace” to a value of “2” - this will enable QV logging only

a.      If you want to get both QV and PROSet logs set "Trace" to 4.

3.      Do ONE of the following:

a.      Restart the system.

b.      Kill all WmiPrvSe (WMI Provider Host) processes in task manager

                  or use the following PowerShell command: 

                  (Get-Process -Name wmiprvse | Where-Object {$_.Modules.ModuleName -contains "ncs2provider.dll"}) | Stop-Process -force

4.      QV.txt is created in C:\Program Files\Intel\Wired Networking\NCS2\

5.      Perform PROSet reproduction steps to invoke QV and get logs.

I'm looking forward to hearing from you.

Best regards,

BY_Intel

Intel Customer Support

[B_Y]

Hello @dev246,

I hope this message finds you well. I am just sending a follow up if you have any questions from the previous post. If so, please let us know. Thank you.

Best regards,

BY_Intel

Intel Customer Support

[B_Y]

Hello @dev246,

Please be informed that we will now close this request since we haven't received any response from our previous follow ups. Just feel free to post a new question if you may have any other inquiry in the future as this thread will no longer be monitored.

Best regards,

BY_Intel

Intel Customer Support

[dev246]

sorry for not replying but i was on vacation, is there any way to reopen this ticket? . The problem persists after the update. The generated logs are added as an attachment.