TCP reset does not stop modern attacks

Introduction

Hybrid attacks typically involve a number of accounts, hosts and other resources and require a holistic approach to response / containment. TCP reset (RST) is touted by some vendors as an effective response mechanism to stop and prevent further spread of ongoing attacks. This article dives into the technical reality around trying to use TCP RST. Because of the limitations and potential for harm of legitimate traffic and applications, Vectra does not offer TCP RST as a response mechanism. Finally, the article discusses Vectra response options.

This article is divided into 3 parts

  • Abstract – Concise summary of key points

    • This will contain some brief high-level descriptions of content in the other two sections. Read the more detailed sections for full details on each topic.

  • TCP Reset Limitations and Problems – Deep dive into the reality of using TCP RST as response option.

  • Vectra’s Response Options – Brief overview of response options offered by Vectra.

Abstract

TCP RST is a mechanism within the TCP protocol that is used to forcibly close a connection that is behaving abnormally. TCP FIN and TCP RST both can terminate a connection, but FIN is graceful termination and RST is a forceful, abrupt termination. Different types of security products have offered TCP RST as a response option, but many have moved on from using or recommending RST due to it being ineffective or problematic. A sampling of reasons is below:

  • It is difficult to implement in an effective manner.

    • RST packets require in window sequence numbers, this is not always easy, especially at high rates.

    • Anti spoofing features such as uRPF and the TCP ACK challenge from RFC 5961arrow-up-right can make implementation difficult and if turned off can make you vulnerable to RST attacks.

  • It can be easily bypassed by attackers.

    • Attackers can simply ignore RST packets.

    • Out of order, or delayed RST handling can make RST ineffective due to race conditions.

    • UPD, ICMP, and DNS traffic can be used and is immune to TCP RST.

  • It is ineffective against encrypted or obfuscated traffic.

    • Common malware frameworks and red team tools such as Cobalt Strike, Sliver, Metasploit, Havoc have options to resist or ignore TCP RST.

    • Encryption can prevent injecting forged TCP RST packets.

    • Encapsulated sessions are immune to TCP RST because only the outer transport layer is affected.

    • VPNs and other tunnels will reestablish connections when interrupted.

  • It can cause collateral damage and service disruptions.

    • Higher load can impact networks when TCP RST must send 2 packets for every session it hopes to disrupt.

    • Some applications can be corrupted or impacted by TCP RST.

    • Environments with BGP must be particularly cautious when using TCP RST.

  • It is stateless, lacks persistence, and is prone to race conditions.

    • Attackers can immediately reconnect using new ports, IPs, or other obfuscation techniques.

  • It does not address the root cause or kill chain progression.

    • Only active sessions can be disrupted. Attackers may already have footholds and you are essentially playing Whack-a-Mole and will miss or hit things you don’t intend to.

TCP RST is, at best, a short-lived reactionary tactic that modern adversaries can easily evade. Relying on advice from vendors that promote the idea that TCP RST is a preferred, effective, and problem free response option can give a false sense of security and can lead to service disruption for mission critical applications.

TCP RST is not a clean or secure response mechanism and customers would be better served to implement blocking controls in EDR, NAC, firewalls, and proxies. Large SOC teams may choose to automate a more complete response playbook through SOAR tools that leverage their existing investments in their chosen tools.

Vectra offers a variety of native response options that operate on a host or account level and can integrate with third party tools to further automate lockdown using technologies such as Vectra Automated Responsearrow-up-right (VAR). Additionally, Vectra has an open API and integrates with popular SOAR tools for customers who wish to automate response using their preferred tools.

TCP Reset Limitations and Problems

It is difficult to implement in an effective manner

TCP RST packet sequence number must fall within valid window

The tool sending a TCP RST must craft the reset packet with a correct sequence number within the valid window for the RST to be successful. In high-speed networks, this becomes much harder because the sequence numbers advance very rapidly. Tools typically send multiple different RST packets with increasing sequence numbers as they hope to successfully reset the connection.

uRPF on switch will drop packets

Unicast Reverse Path Forwarding or uRPF is a feature of most modern switches and routers that can be configured in both strict and loose modes. Essentially the feature checks whenever your router receives an IP packet if it has a matching entry in the routing table for the source IP address. If it doesn’t match, the packet will be discarded. This can make RST packets get dropped.

For example, while legitimate security tools may use TCP RST to block suspicious connections, attackers can also exploit this by spoofing the source IP of one side of a session and sending a TCP RST packet to disrupt communication. uRPF is proposed on most switches and routers to prevent IP address spoofing by ensuring incoming packets arrive on the correct interface based on the router's routing table. Therefore, if a security tool sends a crafted TCP RST packet to block a session, the TCP RST packet will likely come from an unexpected interface. uRPF (if in strict mode) will then drop the packet because the source IP doesn’t match the expected forwarding path.

If this has been enabled as a security feature, teams may be reluctant to turn it off and differences in firmware versions and vendors can make this even more difficult to manage, especially in larger environments.

TCP ACK challenge can make it harder

RFC 5961arrow-up-right introduced an ACK challenge mechanism to help protect against spoofed RST packets (RST packets that don’t originate from participants in the original TCP connection). The ACK challenge is enabled by default in many Linux-based OS’s. The ACK challenge makes an endpoint that receives a RST packet that is in-window, but does not match the exact sequence number, not immediately tear down the connection. Instead of closing the connection, the receiver sends an ACK challenge back to the sender, requiring confirmation.

The ACK challenge delays, and potentially negates the effectiveness of RSTs because resets, even with in-window sequence numbers are no longer immediately honored. This gives attackers even more time to complete the communications that they were attempting.

It can be easily bypassed by attackers

There are a surprising number of ways that attackers can bypass TCP RST or greatly reduce its effectiveness:

They can simply ignore them

  • Attackers can use simple firewall rules at a client level to just drop the RST packets so they are never processed by the client. When you control both hosts, this is easy to implement.

    • This was tested in the paper Ignoring the Great Firewall of Chinaarrow-up-right - Section 5 of the paper discusses how deliberately ignoring resets allowed hosts to communicate through the Great Firewall of China. All it took was a one line configuration using iptables in Linux to direct that TCP packets with the RST flag set should be discarded. They did this on both ends of the connection and successfully transferred content that would have otherwise been blocked over the http connection.

    • Linux iptables example:

      • iptables -A INPUT -p tcp --tcp-flags RST RST -j DROP

    • Another interesting observation about the Great Firewall of China was made in a 2008 article from The Atlantic called The Connection Has Been Resetarrow-up-right.

      • They said, “What the government cares about is making the quest for information just enough of a nuisance that people generally won’t bother”.

      • This speaks to the fact that the tactics used to stop communication were ultimately more of a nuisance than an impenetrable wall. Motivated attackers won’t be easily stopped by techniques that are simply a nuisance.

  • Attackers can use custom TCP stacks using raw sockets that ignore RST packets.

  • Common malware frameworks and red team tools such as Cobalt Strike, Sliver, Metasploit, Havoc have options to resist or ignore RST. Sometimes this is a built-in capability, or an option to use custom TCP handlers that ignore RST.

  • Custom backdoors can employ SOCKS proxies, VPN-like tunnels, or encrypted channels that do not respect standard TCP behavior and would not respect a RST packet.

  • Some common evasion methods include retries and persistent connections, alternate protocols, and layered tunneling.

Out-of-Order or delayed RST handling

  • Race Condition Exploitation: If an attacker sends a large amount of data before a reset arrives, some systems may process the data before closing the connection.

  • TCP Window Manipulation: Attackers can tune TCP window sizes to ensure a long enough delay before a RST takes effect, allowing data exfiltration to complete.

  • HTTP sessions are especially resilient against TCP RST because many connections are used, and all typically won’t be blocked. Short sessions can be initiated and ended before a RST has been generated and tries to terminate the session.

Encrypting or obfuscating traffic

  • TLS/SSH tunneling: If traffic is encrypted (e.g. VPN, TOR, or SSH tunnels), security tools cannot inject a valid RST since the traffic is protected at a higher layer.

    • Use of encryption prevents injecting forged TCP RST packets.

    • Encapsulated sessions are immune to TCP RST because only the outer transport layer is affected.

    • VPNs and other tunnels will reestablish connections when interrupted.

    • Protocols like SSH and TLS establish a stateful, authenticate session. If a TCP RST packet is receive, the protocol verifies the session integrity before terminating the connection, making blind RST injection difficult.

  • Packet fragmentation: Some attackers use TCP segmentation to split data into smaller packets, potentially making RST use ineffective.

Using multipath or redundant channels

  • Multiple connections (session hopping): Attackers can maintain multiple simultaneous connections, so even if one is reset, others remain active.

  • QUIC Instead of TCP: Since QUIC (used in HTTP/3) does not rely on TCP, RST injections do not affect it.

  • Proxy & relay networks: Attackers route traffic through proxies or relays so that a reset affects only an intermediary, not the origin system.

UDP, ICMP, and DNS can be used

  • TCP RST does not work with UDP or ICMP protocols so tunneling using those protocols can’t be stopped with TCP RST.

  • DNS traffic is normally UDP based and immune to TCP RST so tunneling via DNS can’t be stopped when using UDP.

    • It should be noted that DNS can use TCP, although typically only for zone transfers, large responses, and DNS over TLS (DoT) or DNS over HTTPS (DoH).

    • If DoT or DoH were used, since the traffic was encrypted, it would be hard to determine which connections to attempt to block. Even if decrypted, retries and connection reestablishment along with TCP Fast Open (TFO) and TLS session resumption would allow these sessions to quickly recover from any disruption that might be caused by a TCP RST against DNS traffic that does use TCP.

ACK challenge handling

  • As per the previous section, ACK challenges can make using TCP RST effectively harder.

  • Attackers can also:

    • Attempt to respond with correct acknowledgments to avoid connection termination.

    • Use spoofed source IPs to flood the system with fake resets, exhausting challenge ACK resources.

TCP reset flooding (defensive evasion)

  • DDoS via RST Flooding: Attackers may send large volumes of fake RST packets with spoofed IPs to overwhelm network security tools, making their legitimate connections harder to detect.

  • Reset Suppression Attacks: Some tools rely on heuristics for resets; attackers can overwhelm them with traffic to make resets ineffective.

Application-level persistence

  • Session Resume Features: If a security tool resets a connection but the application supports session resumption (e.g., TLS, QUIC, WebSockets), the attacker can reconnect instantly.

  • Retry Logic: Malware and attack tools often implement auto-reconnect mechanisms that detect resets and immediately re-establish connections.

It can cause collateral damage and service disruption

Using TCP RST as a response option with other security tools has had disastrous impacts for some customers that Vectra has spoken with. The risk is much higher when using tools that are especially prone to high rates of false positives.

Anecdotally, we can share that we heard of one situation where an organization that used TCP RST reported that it caused an outage due to resultant corruption of their customer self-service commerce application. The impact caused them to have a reduced commerce facility for several days where they could not offer self-service facilities, and required their customers to use human operators to purchase services.

Most concerning is when TCP resets are used within enterprise environments which use Border Gateway Protocol (BGP). A TCP reset on a BGP session will completely reset it and generate a significant impact on all services attached to the session. We know of an organization that utilized competitor technologies and found their data center was brought down due to the TCP RST function being used on active sessions within the DC being flagged as malicious when in fact they were essential to the operation of the core network.

It has also been seen that when technologies use the TCP RST facility it can cause strain on the infrastructure because of the requirement to send two TCP RST packets for each TCP connection actioned (one for the originating source and one for the destination). This could mean that full implementations using TCP RST may require additional hardware and ongoing management of it.

It is stateless, lacks persistence, and is prone to race conditions

Unlike more robust response options, TCP RST, is stateless and ephemeral. It does not prevent session reestablishment. Attackers can immediately reconnect using new ports, IPs, or other obfuscation techniques.

If an attacker’s packets are already in transit before the RST reaches the target, the attack payload may still execute. Attackers can also block or throttle RST packets to maintain persistence.

It does not address the root cause or kill chain progression

TCP RST can only be used against an active session. Attackers may have already been successful with initial access, recon, lateral movement, establishing persistence, exfiltrating data, etc. TCP RST use can feel like a game of Whack-A-Mole where you often miss the target, and the attack continues regardless of how many moles you may have whacked along the way.

Vectra Response Options

As mentioned in the introduction, Vectra offers a variety of native response options that operate on a host or account level and can integrate with 3^(rd) party tools to further automate lockdown. Additionally, Vectra has an open API and integrates with popular SOAR tools for customers who wish to automate response using their preferred tools. Please see the below links for more details:

  • See more details about Vectra 360 Response on our public website including demo videos:

    EDR Host Lockdown

  • Account Lockdown

    • This feature allows automated or manual lockdown of AD accounts.

  • Entra ID (Azure AD) Account Lockdown

    • This feature allows automated or manual lockdown of Entra ID (Azure AD) accounts.

  • Traffic Lockdown

    • Network-level containment feature that enables Vectra to automatically add compromised host IP addresses to an external blocklist that your firewalls can subscribe to. When a host is added to Traffic Lockdown (manually or automatically), its IP address is published to a plain-text threat feed that integrated firewall devices poll and use to block traffic from that host

  • Vectra Automated Response - https://github.com/vectranetworks/vectraautomatedresponsearrow-up-right

    • This framework allows for easy integration of additional 3^(rd) party security vendors and enables blocking by host, account, or detection.

    • It works with additional EDR tools that aren’t supported natively, firewalls, NACs, system calls to external programs/scripts, windows shutdown, VMware vSphere/ESXi, and static destination IP blocking.

    • This works in both Respond UX (RUX) and Quadrant UX (QUX) deployments.

    • While this is free to use, it requires a host that is external to the Vectra system to run it today and can be setup by the customer or by Vectra professional services on customer supplied resources.

  • SOAR integration

    • A variety of SOAR integrations already exist.

    • If an integration doesn’t exist already, Vectra’s APIs can be used to build a new integration in both RUX and QUX deployments.

Last updated

Was this helpful?