Enabling IPFIX with Open vSwitch to gain visibility into virtual switches

Many customers running the Riverbed Technology NPM toolset including SteelCentral NetProfiler and Flow Gateway are interested in gaining network visibility at the hypervisor layer from their vSwitches. For those running Open vSwitch as their virtual switch, we can export IPFIX (sometimes thought of as NetFlow v10) to the Flow Gateway to gain NPM insights. IPFIX enables the exporting of IPv6.

 

Some considerations to cover  prior to implementation:

  • The current sizing of your Flow Gateway and NetProfiler. Adding additional flow sources will add additional flows add to the flow count as you will now be collecting flows for all traffic entering a virtual switch. There are options for increasing the license limitation on most of our NetProfiler variants.
  • The current flow log retention, adding flows will will reduce flow log retention.
  • Additional load on the hypervisor generating IPFIX flows. Whilst usually insignificant, it may be a consideration if your hypervisors are under significant load.
  • Additional load on the network. Flow data usually accounts for 3-5% of traffic, however in some cases I have seen it reach up to 10%. This can be mitigated using regional Flow gateways to de-duplicate flows prior to sending them across the WAN. Alternatively, using Fixed Target rules and packet mode optimisation on SteelHeads can significantly reduce the IPFIX overhead.
  • Sampling flows. Flow sampling is the default option on the Open vSwitch, whilst this reduces the load on the hyperviser, it does not give accurate accounts of traffic, and entire communications can be missed. I always recommend disabling sampling if at all possible.
  • Flows should be exported every 60 seconds.
  • IPFIX does not provide visibility into Netowrk RTT, Server Delay, Retransmissions. It does give visibility into Which devices are talking to each other using which protocol/port, and how much they are talking.

 

You can check the current NetProfiler flow counts from System menu and selecting the Information menu item. This will allow you to see flow counts historically and determine if your NetProfiler and Flow Gateway can support the additional flows.

 

 

You can check the current Flow Log retention by selecting the Configuration menu, and selecting the Flow Log menu item.

 

 

My environment contains a NetExpres, and a Proxmox PVE as the hypervisor with Open vSwitch as the virtual switch.

 

NetExpress/Flow Gateway

IP Address: 10.0.0.203

Netflow/IPFix Port: UDP/2055

 

Proxmox PVE

IP Address: 192.168.20.45

Open vSwitch bridge: vmbr1

 

To configure Open vSwitch we can SSH onto the hypervisor, and as the root user run the ovs-ovsctl command to list bridges that are currently under Open vSwitch control.

 

root@pve-2:~# ovs-vsctl list bridge

_uuid               : d9cb0d05-b2b3-4abb-a11a-73ad2031df97

auto_attach         : []

controller          : []

datapath_id         : "0000060dcbd9bb4a"

datapath_type       : ""

datapath_version    : "<unknown>"

external_ids        : {}

fail_mode           : []

flood_vlans         : []

flow_tables         : {}

ipfix               : []

mcast_snooping_enable: false

mirrors             : []

name                : "vmbr1"

netflow             : []

other_config        : {}

ports               : [05366820-676a-4cbe-9074-e1acfd3a8b28, 0977f759-b1f3-4146-ab95-117aff034619, 09fefd5a-afb3-44c1-a666-6d8994cf5f32, 204bdf9a-da0a-4858-a3a8-70fc2af9dc5c, 3e9d15a7-f8bc-4b8f-8da6-249ae8198041, 457cb24e-367d-4ac5-ae5c-ca3dc55066fb, 6b8fc75c-3783-4713-af8e-213b850e8659, 70ee0675-45b4-4c2c-bf83-c1c16067b00e, 931f8f70-443a-46e3-9fdf-4f807ee21a4c, 96ae9553-63e5-4478-9a89-cd0c821d4d1c, a5504446-72f8-468e-b2de-e8383cc1e9c5, b9b6d690-c7a9-4dba-9f14-53809ca13a00, bc049074-c041-4f30-86ae-b73c27c4546d, bd801913-628b-4b8e-a152-9782462f152a, bee6761f-b92b-4e0c-908f-a6f4b08fd765, bf56df3e-f856-43d0-ada9-8a1321dc5365, ce77fa20-b834-4218-8167-4de63815c849, f4364d0c-d63b-4d74-bab9-b2a280b9eec2, ff940a12-3f32-4216-bb08-691b7c40b9f5]

protocols           : []

rstp_enable         : false

rstp_status         : {}

sflow               : []

status              : {}

stp_enable          : false

 

To enable IPFIX exports to the Flow Gateway (or NetExpress) we can issue the the following command.

 

ovs-vsctl set Bridge <<bridge>> ipfix=@i -- --id=@i create IPFIX targets=\"<<Flow Gateway IP>>:<<Flow Gateway port>>\" obs_domain_id=123 obs_point_id=456 cache_active_timeout=60 other_config:enable-input-sampling=false  other_config:enable-tunnel-sampling=false

 

root@pve-2:~# ovs-vsctl set Bridge vmbr1 ipfix=@i -- --id=@i create IPFIX targets=\"10.0.0.203:2055\" obs_domain_id=123 obs_point_id=456 cache_active_timeout=60 other_config:enable-input-sampling=false  other_config:enable-tunnel-sampling=false

 

We can now see that IPFIX has been enabled.

root@pve-2:~# ovs-vsctl list bridge

_uuid               : d9cb0d05-b2b3-4abb-a11a-73ad2031df97

auto_attach         : []

controller          : []

datapath_id         : "0000060dcbd9bb4a"

datapath_type       : ""

datapath_version    : "<unknown>"

external_ids        : {}

fail_mode           : []

flood_vlans         : []

flow_tables         : {}

ipfix               : b88511f5-7c0a-437c-b2a3-c96c1971cb70

mcast_snooping_enable: false

mirrors             : []

name                : "vmbr1"

netflow             : []

other_config        : {}

ports               : [05366820-676a-4cbe-9074-e1acfd3a8b28, 0977f759-b1f3-4146-ab95-117aff034619, 09fefd5a-afb3-44c1-a666-6d8994cf5f32, 204bdf9a-da0a-4858-a3a8-70fc2af9dc5c, 3e9d15a7-f8bc-4b8f-8da6-249ae8198041, 457cb24e-367d-4ac5-ae5c-ca3dc55066fb, 6b8fc75c-3783-4713-af8e-213b850e8659, 70ee0675-45b4-4c2c-bf83-c1c16067b00e, 931f8f70-443a-46e3-9fdf-4f807ee21a4c, 96ae9553-63e5-4478-9a89-cd0c821d4d1c, a5504446-72f8-468e-b2de-e8383cc1e9c5, b9b6d690-c7a9-4dba-9f14-53809ca13a00, bc049074-c041-4f30-86ae-b73c27c4546d, bd801913-628b-4b8e-a152-9782462f152a, bee6761f-b92b-4e0c-908f-a6f4b08fd765, bf56df3e-f856-43d0-ada9-8a1321dc5365, ce77fa20-b834-4218-8167-4de63815c849, f4364d0c-d63b-4d74-bab9-b2a280b9eec2, ff940a12-3f32-4216-bb08-691b7c40b9f5]

protocols           : []

rstp_enable         : false

rstp_status         : {}

sflow               : []

status              : {}

stp_enable          : false

 

We can now see the hypervisor on the NetProfier by selecting the Configuration menu and then selecting the Devices/Interfaces menu item. The node should be green within 5-10 minutes.

 

 

We can traffic from this host by clicking on it, which will generate a 6 minute report for this host.

 

 

 

In the event that you want to disable IPFIX you can disable it using the ovs-ovsctl clear command.

root@pve-2:~# ovs-vsctl clear bridge vmbr1 ipfix

 

More information can be found via the following links: