Odd issue with OS6900, VMware, and Intel X710

[misterflan]

Hi guys, I got pointed here from a guy on reddit and was hoping that someone maybe able to shed some light on some issues I've been having, to copy paste from the post I made in /r/vmware and /r/networking,

Hello VMware users of reddit.
I was wondering if any of you wonderful people have got ESXi hosts running with an Intel X710 (in a R730 fwiw)?

Just we have, and we've been having all manner of issues which we seemingly can't put our finger on. Basically what happens is a vMotion happens, and the VM drops offline. We've checked all the usual things:

[*] Notify switches yes, promiscuous off etc on the DPG's
[*] MAC learning correctly on switches
[*] Firmware driver combos
[*] Switch config (this is a bit out of my control, I don't personally know Alcatel Lucent OS6900 switches all that well, I'm more Cisco..)
[*] Hosts being up to date

VMware haven't been all that helpful as the issue isn't always reproducible, and point the finger to outside of the host.. We generally fix it usually by vMotioning it back to whence it came and it works again, as customers scream and want their money back. Or recreating the vNIC as this has a good chance of resolving the issue.

This also happens in different environments, but with the same kit (Dell R730xd, X710-DA4, OS6900) - so it's not a build issue. The only vague commonality is about 80% of the VMs that experience the issue were migrated into us using Zerto.

It's getting to the point now where we are tempted to start ripping and replacing hardware, but to me, there has to be an answer as a chip giant such as Intel and a network vendor such as AL shouldn't be having these issues...

So, before we start arranging downtime and raising PO's, I'd like to know if anyone else has had any funky issues with this wretched card?

Cheers.

And then one of the replies pointed me in the direction of this post, https://eositis.wordpress.com/2013/11/0 ... r-vmotion/ - in which he mentions some commands for Force10 and Cisco, but I have no idea what the ALU equivilent would be... Not saying this is the fix, but it'd sure help to go to our network guys with something to check who keep saying everythings fine and it's all just layer-2 trunk ports and pointing the finger back at VMware/us.

So if anyone could be of any assistance I'd be very very grateful.

Thanks!

[sputniki]

I don't know what your problem really is and I can only guess what kind of setup you have - so please don't expect too much information if you don't share details

Just one comment for the Link where Elmar is talking about network problems in case of vMotion: If you use OS6900 switches in a Virtual Chassis setup you don't will have such problems. As far as I know Force10 doesn't support stacking or real virtual chassis like Alcatel does: There is always ONE arp table or all switches. So changing of MAC address is global within all virtual chassis - other than Force10 switches with Virtual Link Trunk that only duplicates traffic.

To your question: Is the MAC-address of the new VM after vMotion process visible on the OS6900 trunk ports? Is the vSwitch configuration really 100% the same as on the other hosts?

[skycaptain]

I know this is an old ticket but thought this might be helpful to others.
We have had issues with the X710 card in a Dell Server/VMWare/Juniper Switch environment. The TCP offload features on the card and the drivers that Intel supplies for the card's use with VMWare do not work very well in our experience. We traced the issue down using wireshark to duplicated and bounced back layer 2 broadcast packets. It wreaked havoc with virtual firewalls that use VRRP and anything that depended on Layer 2 multicast. We removed the X710 cards from the environment and all issues were resolved. Feel free to PM me for additional details.

[sputniki]

Thanks for sharing your solution, @skycaptain!

[pkremer]

I'm very interested in details but it looks like PMs are disabled.