> For the complete documentation index, see [llms.txt](https://docs.vectra.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.vectra.ai/deployment/traffic-engineering-and-validation/encapsulation-endpoints-gre-erspan-geneve-vxlan.md). # Encapsulation Endpoints (GRE, ERSPAN, GENEVE, VXLAN) ## Overview The Encapsulation Endpoints feature enables Vectra Sensors to decapsulate ERSPAN traffic, add an IP address to capture interfaces, and serve as a destination for tunneled encapsulations such as GRE, ERSPAN, GENEVE, and VXLAN. Vectra can still decapsulate these encapsulations when the tunnel destination is not the capture interface as long as the capture ports are passively observing the traffic through normal out-of-band mechanisms such as SPAN/COPY/MIRROR, TAPs, 3rd party packet brokers, etc. Support for decapsulating passively observed GRE, GENEVE, and VXLAN existed prior to the encapsulation endpoints feature becoming available. The new feature introduced ERSPAN decapsulation support along with the ability to add an IP address to capture interfaces. An additional benefit of this new feature is that ICMP based health checks can be directed at capture interfaces that have a configured IP address. The capture interfaces will only respond to ICMP and ARP and do not originate any other traffic. Capture interfaces configured with an encapsulation endpoint (IP address) still continue to process passively observed traffic as they did before the IP was added. {% hint style="info" %} **Please Note for v9.13** * All virtual appliances are now supported as of the v9.13 release. * Azure does NOT support GRE encapsulation and since ERSPAN requires GRE, ERSPAN is NOT supported in Azure either. * All physical appliances have already been supported since the initial release of this feature. * Mixed-mode deployment is not supported in v9.13 but support is planned for a future update. * A Sensor being enabled as an encapsulation endpoint must only be in Sensor mode in v9.13. * Up to 16 IPs can be assigned per capture interface. * Care must be taken to spread load across worker threads on each Sensor to achieve the full performance possible from a Sensor. * Please see [Spreading Load Across Worker Threads](#spreading-load-across-worker-threads) for details. {% endhint %} {% hint style="info" %} **Please Note for v9.12** This feature is now GA (Generally Available) as of the v9.12 release. Updates are still being made to the feature. Please continue reading for details. * Virtual appliances are now supported as of the v9.12 release but not all virtual appliance models are supported. * 2 and 4 core vSensors are supported in v9.12 for both IaaS and traditional hypervisor deployments. * 8 core and higher vSensors are NOT supported in v9.12 but support is planned for a future update. * Azure does NOT support GRE encapsulation and since ERSPAN requires GRE, ERSPAN is NOT supported in Azure either. * Mixed-mode deployment is not supported in v9.12 but support is planned for a future update. * A Sensor being enabled as an encapsulation endpoint must only be in Sensor mode in v9.12. * A single IP address is supported per capture interface. * Vectra plans to support up to 16 IPs per capture interface in a future update. * For ERSPAN, Type II and Type III are supported. * Type I (generally considered deprecated for most uses) is NOT supported. * For ERSPAN decapsulation, an encapsulation endpoint must be configured for ERSPAN decapsulation to function when the ERSPAN traffic is being passively observed and a Sensor capture interface is NOT serving as the ERSPAN tunnel endpoint. * It is ok to enable an encapsulation endpoint and not assign an IP to it if you do not need it to serve as an ERSPAN tunnel endpoint and just need the Sensor to passively observe and decapsulate ERSPAN traffic. * ERSPAN decapsulation is enabled on a per Sensor basis. Enabling an encapsulation endpoint, with or without an IP and gateway assigned, will allow the Sensor to observe ERSPAN traffic passively, on any interface, and decapsulate it. You do not need to enable an encapsulation endpoint on every interface if you will be observing the ERSPAN traffic passively. * For any capture interface that will serve as an ERSPAN tunnel endpoint, an IP address and gateway MUST be configured. {% endhint %} ## Supported Encapsulation Types The table below lists the encapsulations supported by Vectra Sensors. Please see [Multiple Encapsulation Layer Support](#multiple-encapsulation-layer-support) for details about supported and unsupported combinations of these encapsulations.
EncapsulationSupport notes
IEEE 802.1Q / IEEE 802.1adVLAN and QinQ traffic are supported when the traffic arrives with no more than two VLAN tags.
GREGRE-encapsulated traffic is supported.
ERSPANERSPAN traffic (Type II and Type III) is supported as of v9.10.
VXLANVXLAN-encapsulated traffic is supported.
GeneveGeneve-encapsulated traffic is supported.
IPsec Authentication HeaderIPsec AH traffic is supported.
## Configuration Navigate to *Configuration* :arrow-right: COVERAGE → *Data Sources* in the left hand panel. From there, select *Network* :arrow-right: *Encapsulation Endpoints.* {% stepper %} {% step %} #### Add an encapsulation endpoint To enable ERSPAN decapsulation and/or configure an IP address to use as an encapsulation endpoint, expand the desired Sensor and hover over the desired interface until the pencil icon appears, and then click it.
Enable the encapsulation endpoint for the interface you are modifying and click **Save**. {% hint style="info" %} **Please Note:** Enabling any interface on a Sensor as an encapsulation endpoint will enable ERSPAN decapsulation on all capture interfaces. ERSPAN decapsulation will only work on interfaces without an IP when the ERSPAN traffic is being observed passively. {% endhint %}
{% endstep %} {% step %} #### Add IP Address/Subnet and Default Gateway When adding encapsulation endpoints, it is very important to [spread the load across worker threads](#spreading-load-across-worker-threads) on the Sensor when the Sensor supports more than one worker thread. Enter the **IP Address/Subnet** in CIDR notation the selected capture interface will use when listening for traffic in supported encapsulation types. This IP can also be used for ICMP based health checks. Up to 16 encapsulation endpoints can be configured on a single interface. An IP address and gateway are not required to be entered if you are just enabling ERSPAN decapsulation for passively observed traffic and you do NOT need an IP address for tunnel termination. Enter the **Default Gateway** for this IP address and **Save** your configuration.
{% hint style="info" %} **Please Note:** Configured encapsulation endpoints, that have an IP address and gateway configured, will not initiate any communication, they will only respond to ICMP and ARP. {% endhint %} {% endstep %} {% step %} #### Repeat as necessary Repeat steps 1 and 2 for any other Sensor capture interfaces you wish to configure as an encapsulation endpoint. {% endstep %} {% endstepper %} ## Spreading Load Across Worker Threads The various Sensor models offered by Vectra each have a specific amount of bandwidth that can be processed. This will vary by configuration and mode. For example, a VMware vSensor can be deployed with varying CPU cores, memory, and disk space and each configuration supports different throughput when used in Sensor mode, or in Sensor/Match mode where the Sensor performs both normal NDR Sensor functions along with [Match](/deployment/match/deployment/introduction-and-requirements.md). The [Appliance specifications](/deployment/getting-started/appliance-specifications.md) page details the specifications for each Sensor type. {% hint style="info" %} **Please Note:** Vectra testing has shown a neglible performance impact for decapsulation operations on a Sensor, but because customer traffic mixes vary, it is recommended to allow for up to a 5% reduction in performance. For example, an S127 is rated for 58 Gbps when Match is not enabled. This means that when decapsulating all traffic coming to the Sensor it is recommended to tunnel a maximum of 55.1 Gbps to the Sensor. {% endhint %} When tunneled traffic is sent to a Sensor, it typically contains traffic for many different hosts but the outer encapsulation source is a single IP. Traffic that isn't tunneled typically has many different source IPs. When tunneled traffic is received on a configured Sensor encapsulation endpoint, the underlying hardware (virtual or physical) performs a hashing function based on the IP addresses of the source and the destination (assigned encapsulation endpoint). This will determine which RX queue and worker thread on the Sensor will process the traffic. Because tunneled traffic is received from a single IP, individual worker threads on the Sensors can become saturated while traffic from many source IPs is spread naturally amongst available worker threads. The number of worker threads available per Sensor varies by Sensor model. Many Sensor models can achieve full throughput with a single worker thread and the number of encapsulation endpoints and the specific IPs in use will be more a matter of customer architecture needs than a requirement for spreading load amongst worker queues. Please see [Worker Threads Supported Per Sensor](#worker-threads-supported-per-sensor) for details. Using the CLI of the Sensor, the `show traffic rss-queue` command will predict the RX queue and worker thread that will be used for a particular source and destination IP pair. On Sensors that have multiple worker threads, it is ok if an RX queue is re-used but care should be taken to spread load evenly across the worker threads. ### Tuning Workflow The following workflow should be used for Sensors that have more than one worker thread. For Sensors with a single worker thread, take care to not exceed the performance limitations of the Sensor by sending more traffic over the tunnel than the Sensor can process. If there is other traffic being observed by the Sensor, but not over a tunnel being terminated by an encapsulation endpoint, this will reduce the volume of traffic that can be tunneled to the encapsulation endpoint. [Worker Threads Supported Per Sensor](#worker-threads-supported-per-sensor) provides guidance for each Vectra appliance that can perform Sensor duties. {% stepper %} {% step %} #### Estimate each tunnel's expected volume You will need to work within the overall performance guidelines for the specific Sensor model and mode you are configuring. {% hint style="info" %} **Please Note:** Vectra testing has shown a negligible performance impact for decapsulation operations on a Sensor, but because customer traffic mixes vary, it is recommended to allow for up to a 5% reduction in performance. **Example** An S127 is rated for 58 Gbps when Match is not enabled. This means that it is recommended to tunnel a maximum of 55.1 Gbps to the Sensor. An S127 Sensor also has 4 worker threads and based on the above, no more than 55.1 Gbps of tunneled traffic should be directed to it. To achieve this performance level, up to 13.775 Gbps of traffic can be mapped to each worker thread. Load can be spread across up to 16 tunnels, but no tunnel should exceed the maximum of 13.775 Gbps for a single worker thread on an S127. {% endhint %} {% hint style="warning" %} **For Sensors with 40 or 100 Gbps QSFP NICs:** At least 4 tunnels are required to achieve full utilization. For example, an S101 has 2 worker threads but can use a 40 or 100 Gbps NIC. 4 tunnels will be required to achieve full utilization. It is ok if the same worker thread is mapped for mulitiple tunnels but you should spread the load evenly with at least 2 tunnels per worker thread. Please ensure that each tunnel does not exceed the maximum allowed per worker thread as explained above. {% endhint %} {% hint style="warning" %} **For X29 (all variants):** The X29 has only one worker thread and at least two tunnels are required to achieve full performance. It is ok that the same worker thread is mapped for these minimum of two tunnels. Please ensure the load is spread evenly between the tunnels. {% endhint %} When planning your traffic steering strategy, it is important to not direct more traffic to a single worker thread than can be processed by it, or data loss may result. It is ok for multiple tunnels to be mapped to the same RX queue or worker thread as long as you don't exceed the maximum bandwidth per worker thread as calculated in the example above. You will also need to work within the overall capability of each individual capture interface when deciding how much traffic to map to each tunnel. For example, a 10 Gbps SFP+ interface can't process more than 10 Gbps of traffic. Work with your networking team to estimate how much traffic each tunnel you want the Sensor to terminate will pass. {% endstep %} {% step %} #### Collect tunnel IP information You will need a source IP and destination IP for each tunnel that you plan to terminate at the Sensor. **Source IP** - Tunnel origination IP **Destination IP** - IP to assign as an encapsulation endpoint on each desired capture interface on the Sensor. {% endstep %} {% step %} #### Use CLI command to predict worker thread mapping Use the `show traffic rss-queue` command at the CLI of your Sensor to predict which worker threads a particular source and destination IP pair will be mapped to. If you are unfamiliar with logging in to the CLI of a Sensor, see: [SSH login process for CLI](/deployment/appliance-operations/ssh-login-process-for-cli.md) **Example:** ``` scli > show traffic rss-queue --interface eth0 --src-ip 192.168.148.1 --dst-ip 192.168.147.3 interface: eth0 driver: mlx5_pci src ip: 192.168.148.1 dst ip: 192.168.147.3 mapped rx queue: 19 configured rx queues: 20 mapped worker: 3 configured workers: 4 vscli > show traffic rss-queue --interface eth0 --src-ip 192.168.148.1 --dst-ip 192.168.147.5 interface: eth0 driver: mlx5_pci src ip: 192.168.148.1 dst ip: 192.168.147.5 mapped rx queue: 13 configured rx queues: 20 mapped worker: 2 configured workers: 4 vscli > show traffic rss-queue --interface eth1 --src-ip 192.168.148.1 --dst-ip 192.168.148.2 interface: eth1 driver: mlx5_pci src ip: 192.168.148.1 dst ip: 192.168.148.2 mapped rx queue: 0 configured rx queues: 20 mapped worker: 0 configured workers: 4 vscli > show traffic rss-queue --interface eth1 --src-ip 192.168.148.1 --dst-ip 192.168.148.15 interface: eth1 driver: mlx5_pci src ip: 192.168.148.1 dst ip: 192.168.148.15 mapped rx queue: 5 configured rx queues: 20 mapped worker: 1 configured workers: 4 ``` {% endstep %} {% step %} #### Choose address pairs that distribute load In the above example an S127 Sensor was used, and all selected source and destination IP pairs are ok to use because they result in a different `mapped worker`. Up to 13.775 Gbps of traffic could be directed at each of these destination IPs assuming the SFP in eth0 or eth1 was capable of that bandwidth. It is safe to use these IPs for the encapsulation endpoints you want to assign to this Sensor. If any mapped worker is repeated, ensure the total bandwidth you will be mapping to that worker will not exceed the maximum per worker you calculated in [Step 1](#estimate-each-tunnels-expected-volume). {% endstep %} {% endstepper %} ### Worker Threads Supported Per Sensor Please expand/collapse the sections below for tables showing the supported worker threads per Sensor type.
Physical Appliances | Model | Worker Threads | | :-----------: | :------------: | | S1 / S1v2 | 1 | | S2 (EOL) | 1 | | S11 | 1 | | S17 | 1 | | S101 / S101v2 | 2 | | S127 | 4 | | X3 | 1 | | X5 | 1 | | X29 / X29v2 | 1 | | X47 | 1 | | X80 (EOL) | 2 |
Virtual / Cloud Appliances | Platform | Cores or Instance Type | Worker Threads | | :------: | :------------------------------: | :------------: | | VMware | 2 | 1 | | VMware | 4 | 1 | | VMware | 8 | 1 | | VMware | 16 | 1 | | VMware | 32 | 2 | | Hyper-V | 2 | 1 | | Hyper-V | 4 | 1 | | Hyper-V | 8 | 1 | | Hyper-V | 16 | 1 | | KVM | 2 | 1 | | KVM | 4 | 1 | | KVM | 8 | 1 | | KVM | 16 | 1 | | Nutanix | 2 | 1 | | Nutanix | 4 | 1 | | Nutanix | 8 | 1 | | Nutanix | 16 | 1 | | AWS |

r5.large
r5x.xlarge

| 1 | | AWS |

r5.xlarge
r5n.xlarge

| 1 | | AWS |

r5.2xlarge
r5n.2xlarge

| 1 | | AWS |

r5.4xlarge
r5n.rxlarge

| 1 | | AWS | c5n.18xlarge | 2 | | Azure | Standard\_DS3\_v2 | 1 | | Azure | Standard\_DS11\_v2 | 1 | | GCP | e2-standard-2 | 1 | | GCP | e2-standard-4 | 1 | | GCP | e2-standard-16 | 1 | | GCP | e2-standard-32 | 1 |
## Validation Once saved, the **Encapsulation Endpoint Status** will update within a few minutes of the interface is receiving encapsulated traffic. This area only reports on encapsulated traffic.
To see a detailed view of what kind of traffic is being received on each interface, click the **Check Per Sensor Traffic Validation** button near the top of the Encapsulation Endpoints title. This will navigate you to *Network Stats* :arrow-right: TRAFFIC VALIDATION :arrow-right: *Per Sensor Traffic.* Near the bottom of the page there is a new table dedicated to Encapsulation Endpoints as seen below.
## Multiple Encapsulation Layer Support In some environments, mirrored traffic may arrive at the Sensor with more than one encapsulation layer. Vectra Sensors support several encapsulation types, but support for an individual encapsulation type does not mean that every nested combination of those encapsulations is supported. For multi-layer traffic, the Sensor must be able to decapsulate the packet in the order that the headers are encountered, starting with the outermost encapsulation layer. If the encapsulation layers are nested in an unsupported order, the traffic must be modified before it reaches the Sensor capture interface. In these cases, configure a switch, packet broker, cloud traffic mirroring service, or other upstream device to remove unsupported outer encapsulation layers before forwarding the traffic to the Sensor. ### VLAN and QinQ Support Vectra Sensors support VLAN-tagged traffic, including QinQ traffic, when the traffic arrives at the Sensor with no more than two VLAN tags. If traffic arrives with more than two VLAN tags, the additional tags must be removed before the traffic reaches the Sensor capture interface. In these cases, configure the switch, packet broker, or upstream device to strip the extra VLAN tags before forwarding the traffic to the Sensor. ### Supported ERSPAN Multi-layer Combinations The following ERSPAN multi-layer combinations are supported. **Single-encapsulated ERSPAN traffic - Supported** ERSPAN traffic is carried inside a GRE tunnel.
**ERSPAN carrying VXLAN traffic - Supported** ERSPAN traffic is carried inside a GRE tunnel, with VXLAN-encapsulated traffic as the mirrored payload.
**ERSPAN carrying Geneve traffic - Supported** ERSPAN traffic is carried inside a GRE tunnel, with Geneve-encapsulated traffic as the mirrored payload.
### Unsupported Nested Encapsulation Combinations Other nested encapsulation combinations are not supported, even when the individual encapsulation types are listed as supported. For example, VXLAN and Geneve are supported encapsulation types, but traffic with Geneve nested inside VXLAN is NOT supported. This can occur in AWS environments where traffic is first encapsulated by Gateway Load Balancer using Geneve, and then mirrored traffic is encapsulated again by AWS Traffic Mirroring using VXLAN. **VXLAN carrying Geneve traffic - NOT Supported** Traffic encapsulated by AWS Gateway Load Balancer using Geneve is then carried inside a VXLAN tunnel created by AWS Traffic Mirroring.
This traffic format is NOT supported unless the unsupported outer encapsulation layer is removed before the traffic reaches the Sensor capture interface. To use this traffic with a Sensor, configure a packet broker, cloud traffic mirroring service, or another upstream device to remove the unsupported outer encapsulation layer before forwarding the traffic to the Sensor. ### Summary of Multi-layer Support | Traffic format | Support | | ----------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------- | | ERSPAN carried inside GRE | Supported | | ERSPAN carried inside GRE, with VXLAN-encapsulated traffic as the mirrored payload | Supported | | ERSPAN carried inside GRE, with Geneve-encapsulated traffic as the mirrored payload | Supported | | VLAN-tagged traffic with up to two VLAN tags | Supported | | Traffic with more than two VLAN tags | Not supported unless extra tags are stripped before reaching the Sensor | | VXLAN carrying Geneve, such as AWS Traffic Mirroring over GWLB traffic | Not supported unless the unsupported outer layer is stripped before reaching the Sensor | | Other undocumented nested encapsulation combinations | Not supported unless confirmed by Vectra Support or product documentation. | --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.vectra.ai/deployment/traffic-engineering-and-validation/encapsulation-endpoints-gre-erspan-geneve-vxlan.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.