VMware: Custom TCP/IP Stacks in VMware ESXi 6.0

So I stumbled upon this new and interesting feature in VMware ESXi 6.0 that has been around since 5.5 but wasn’t really used that much in common circles. But one of the hurdles faced when make a ESXi cluster’s networking aspect if that by default all TCP/IP traffic uses the default TCP/IP stack, which as you might not know uses one default gateway. This can be really problematic if you want to ‘truly’ segment traffic when using multiple vLAN’s in your network design.

Thankfully, in ESXi 5.5 and 6.0 you can create custom TCP/IP stack. This has many added benefits and perks that allow a vSphere administrator to customize network traffic flows when they leave the ESXi hosts and connect to physical networking fabric that, if used with physical vLAN’s and switch segmentation, makes the network design much more complete from layer 2/3 all the way up to layer 7. This equates into better performance,  stability, and allows for better design characteristics.

 

Benefits of a Separate TCP/IP Stack

    • Separate Memory Heap
    • Personalized ARP Table
    • Personalized Routing Table
    • Personalized Subnet
    • Personalized Default Gateway
    • Personalized Security through network isolation

The command for enabling this custom stack can only be done from the command line, which is access from your favorite telnet program. I like using PuTTy, you need to make sure SSH is enable on the ESXi host and once you get to the DCUI to you use the follow command:

 

esxcli network ip netstack add -N “NameOfYourStack”

 

Me personally in my home-lab and production I make a TCP/IP stack for pretty much all of my vLAN’s for thing like vStorage vMotion, iSCSI, Servers, Printers, Desktops, VOIP Phones, Veeam Backups, Guest Internet with me using it for getting really isolated like with the Sales Department and Accounting.This obviously is at a much higher level of network understanding, but I’m merely stating what this means from a network design standpoint as to what it can do. 🙂

It’s worth noting in ESXi 6.0 vMotion has its own default TCP/IP stack that operates at layer 3 which allows it to work globally over a WAN which is pretty kool for WAN-based ESXi deployment in a multi-site deployment.

I hope this posting is helpful for you and gives you some insight into things to think about in your travels with VMware ESXi 6.0.