- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
HI.
I have supermicro server with :
- Intel® 7500 (Boxboro-EX) chipset
- ICH10R
4 processors with 6 cores in each (Intel(R) Xeon(R) CPU X7542 @ 2.67GHz)
I installed 2 10G intel NICs(82599EB) and the latest ixgbe driver (v3.2.10).
Yesterday I made a test with pktgen and got some strange results(I generated traffic on server (A) and flooded it through supermicro(B)):
1) While I spread interrupts on B 10G nic among 12 cores (the 1-st and the 2-d processors) I recieved 100% utilization on all 12 cores at rate 5Gbit/s (~1 mln pps)
2) While I spread interrupts on B 10G nic among 8 cores (the 1-st (6cores) and the 2-d(2cores) processors) I recieved 100% utilization on all 8 cores at rate 7Gbit/s (~1 mln pps)
3) While I spread interrupts on B 10G nic among 6 cores (the 1-st (6cores) processors) I DID NOT recieved 100% utilization and I could forward 9.5Gbit/s (~1 mln pps)
Of course I have heard that spreading interrupts among different processors could make things worth, but not so dramatically.
I made test on 2.6.37-gentoo-r1.
So can you tell me, how to overcome this problem?
Link Copied
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
okay, it is good you're running a newish kernel (2.6.37)
Please make sure to just allow the driver to load with driver defaults (no module options)
rmmod ixgbe
killall irqbalance
modprobe ixgbe
ip ad ad dev ethM
...
ip link set ethM up
...
then run the (included in ixgbe-3.2.10.tar.gz) set_irq_affinity.sh ethM ethN ethO ethP script
run your test
this will spread all interrupts from each port across all cores. It is probably better to limit the cores in use to the ones directly connected to the I/O controller that is only one hop from your processors, but that is difficult to do currently.
If you want to try the second suggestion, reload the driver like so
rmmod ixgbe
modprobe ixgbe FdirMode=0,0,0,0 RSS=12,12,12,12
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
sorry could u please a basic and interface setting method??
im not and advanced user as im has been first time face this..
tq
- Subscribe to RSS Feed
- Mark Topic as New
- Mark Topic as Read
- Float this Topic for Current User
- Bookmark
- Subscribe
- Printer Friendly Page