ts-2024-005
Vulnerability from tailscale

Description: Insufficient inbound packet filtering in subnet routers and exit nodes

What happened?

In Tailscale versions earlier than 1.66.0, exit nodes, subnet routers, and app connectors, could allow inbound connections to other tailnet nodes from their local area network (LAN). This vulnerability only affects Linux exit nodes, subnet routers, and app connectors in tailnets where ACLs allow "src": "*", such as with default ACLs.

Tailscale version 1.66.0 fixes the vulnerability. Additionally, a server-side update changes the interpretation of "src": "*" to mitigate the issue specifically for exit nodes.

Special thanks to Hakan Ergan for reporting a similar concern that led us to discover this vulnerability.

Who was affected?

This affected the following nodes using Tailscale version 1.65 or earlier:

  • Exit nodes on Linux
  • Subnet routers on Linux
  • App connectors on Linux
  • Regular nodes on all platforms connecting to the above nodes

Tailnets with custom ACLs that do not use "src": "*" or any other value that includes external IPs were not affected.

We are not aware of any active exploitation of this vulnerability.

What was the impact?

Devices outside of the tailnet, but on the same LAN as an exit node, subnet router, or app connector could connect to ports on tailnet nodes that are allowed by ACLs.

What do I need to do?

Upgrade the following nodes to 1.66.0 or later:

We recommend enabling auto-updates and updating all nodes to the latest version, but it is not required to mitigate this vulnerability.

A server-side change mitigated this vulnerability for other types of affected nodes.

Technical details

Below, we refer to exit nodes, subnet routers, and app connectors as packet-forwarding nodes, because the details apply to all of them. Specific types of packet-forwarding nodes are mentioned explicitly where their behavior is different.

Before 1.66.0, packets between regular nodes and destination hosts behind packet-forwarding nodes were filtered based on source/destination IP as specified in tailnet ACLs. Specifying "src": "*" in ACLs is equivalent to "src": ["0.0.0.0/0", "::/0"], meaning any IP address. This allowed source IPs outside of the tailnet to send packets to tailnet nodes via a packet-forwarding node. This could be abused by malicious LAN hosts to connect into the tailnet using a known tailnet node IP.

The attack only works on a LAN because:

  • it relies on next-hop routing, which only works in a LAN
  • destination IPs are in the subnet router's approved range, or in the CGNAT range 100.64.0.0/10, which are not routable over the Internet.
Attacks

Here are several attack scenarios.

Packet-forwarding node on an untrusted LAN.

diagram showing a LAN machine connecting to a victim node

A malicious host 10.0.0.1 on the same LAN as the packet-forwarding node 10.0.0.2 could craft packets with destination IP of a tailnet node 100.64.0.1 (using a command like ip route add 100.64.0.1/32 via 10.0.0.2 dev eth0) and send them to the packet-forwarding node. The packet-forwarding node would accept them and forward them to the victim node. The victim node would see a packet from 10.0.0.1 and accept it if the tailnet ACLs allow this source IP.

This scenario is very similar to a legitimate use-case of site-to-site networking, where two subnets are bridged using Tailscale subnet routers and the flag --snat-subnet-routes=false.

Malicious shared exit node

diagram showing a malicious exit node connecting a the victim node

A malicious exit node 100.64.0.4 from tailnet A could craft packets with destination IP of tailnet B node 100.64.0.3 and any source IP other than 100.64.0.4. Due to the built-in quarantining of shared nodes, packets from 100.64.0.4 are rejected.

Mitigations

We implemented 3 separate mitigations for these attacks.

Redefine "src": "*" in ACLs

While * is a convenient shorthand in ACLs, Tailscale users almost never need to allow connections from any IP. In most cases users intend * as "all other nodes in my tailnet". As a mitigation for this vulnerability, we redefined * in src section of ACLs to include:

  • all tailnet nodes
  • all IPs from approved subnet routes

The inclusion of IPs from approved subnet routes is needed for the site-to-site networking setup.

For users that need the old semantics of * we added a new autogroup:danger-all, which matches the old definition of *.

Stateful packet filtering on packet-forwarding nodes

On Linux packet-forwarding nodes we added stateful packet filtering. This means that these nodes keep track of forwarded connections and only allow return packets for existing outbound connections. Inbound packets that don't belong to an existing connection are dropped.

Because routing is implemented differently on non-Linux platforms, this mitigation is only necessary on Linux.

Stateful filtering is enabled by default, except for existing subnet routers that set --snat-subnet-routes=false. You can disable stateful filtering using tailscale up --stateful-filtering=false.

Client-side quarantining of shared nodes

Quarantining of shared nodes was implemented by a packet filter sent from the Tailscale control plane. This packet filter instructs nodes to drop any inbound connections from the source IP of the shared node. To prevent malicious shared exit nodes from crafting packets with different source IPs, additional client-side quarantining logic was added. The 1.66.0 and later clients reject all inbound connections from quarantined nodes, regardless of their source IP. This is similar to how the "shields up" mode works within the tailnet.

Show details on source website


{
   guidislink: false,
   id: "https://tailscale.com/security-bulletins/#ts-2024-005",
   link: "https://tailscale.com/security-bulletins/#ts-2024-005",
   links: [
      {
         href: "https://tailscale.com/security-bulletins/#ts-2024-005",
         rel: "alternate",
         type: "text/html",
      },
   ],
   published: "Wed, 08 May 2024 00:00:00 GMT",
   summary: "<p><strong><em>Description</em></strong>: Insufficient inbound packet filtering in subnet routers and exit nodes</p>\n<h5>What happened?</h5>\n<p>In Tailscale versions earlier than 1.66.0, <a href=\"https://tailscale.com/kb/1103/exit-nodes\">exit nodes</a>, <a href=\"https://tailscale.com/kb/1019/subnets\">subnet\nrouters</a>, and <a href=\"https://tailscale.com/kb/1281/app-connectors\">app connectors</a>, could\nallow inbound connections to other tailnet nodes from their local area network\n(LAN). This vulnerability only affects Linux exit nodes, subnet routers, and\napp connectors in tailnets where <a href=\"https://tailscale.com/kb/1018/acls\">ACLs</a> allow <code>\"src\": \"*\"</code>, such as\nwith <a href=\"https://tailscale.com/kb/1192/acl-samples#allow-all-default-acl\">default ACLs</a>.</p>\n<p>Tailscale version 1.66.0 fixes the vulnerability. Additionally, a server-side\nupdate changes the interpretation of <code>\"src\": \"*\"</code> to mitigate the issue\nspecifically for exit nodes.</p>\n<p>Special thanks to <a href=\"https://www.linkedin.com/in/hakan-ergan/\">Hakan Ergan</a> for reporting a similar\nconcern that led us to discover this vulnerability.</p>\n<h5>Who was affected?</h5>\n<p>This affected the following nodes using Tailscale version 1.65 or earlier:</p>\n<ul>\n<li>Exit nodes on Linux</li>\n<li>Subnet routers on Linux</li>\n<li>App connectors on Linux</li>\n<li>Regular nodes on all platforms connecting to the above nodes</li>\n</ul>\n<p>Tailnets with custom ACLs that do not use <code>\"src\": \"*\"</code> or any other value that\nincludes external IPs were not affected.</p>\n<p>We are not aware of any active exploitation of this vulnerability.</p>\n<h5>What was the impact?</h5>\n<p>Devices outside of the tailnet, but on the same LAN as an exit node, subnet\nrouter, or app connector could connect to ports on tailnet nodes that are\nallowed by ACLs.</p>\n<h5>What do I need to do?</h5>\n<p>Upgrade the following nodes to 1.66.0 or later:</p>\n<ul>\n<li>Subnet routers on Linux</li>\n<li>App connectors on Linux</li>\n<li>Regular nodes on all platforms that use exit nodes <a href=\"https://tailscale.com/kb/1084/sharing#sharing--exit-nodes\">shared from other\ntailnets</a> or <a href=\"https://tailscale.com/kb/1258/mullvad-exit-nodes\">Mullvad exit nodes</a></li>\n</ul>\n<p>We recommend enabling auto-updates and updating all nodes to the latest\nversion, but it is not required to mitigate this vulnerability.</p>\n<p>A server-side change mitigated this vulnerability for other types of affected\nnodes.</p>\n<h5>Technical details</h5>\n<p>Below, we refer to exit nodes, subnet routers, and app connectors as\n<em>packet-forwarding nodes</em>, because the details apply to all of them. Specific\ntypes of packet-forwarding nodes are mentioned explicitly where their behavior\nis different.</p>\n<p>Before 1.66.0, packets between regular nodes and destination hosts behind\npacket-forwarding nodes were filtered based on source/destination IP as\nspecified in tailnet ACLs. Specifying <code>\"src\": \"*\"</code> in ACLs is equivalent to\n<code>\"src\": [\"0.0.0.0/0\", \"::/0\"]</code>, meaning any IP address. This allowed source IPs\noutside of the tailnet to send packets to tailnet nodes via a packet-forwarding\nnode. This could be abused by malicious LAN hosts to connect into the tailnet\nusing a known tailnet node IP.</p>\n<p>The attack only works on a LAN because:</p>\n<ul>\n<li>it relies on next-hop routing, which only works in a LAN</li>\n<li>destination IPs are in the subnet router's approved range, or in the <a href=\"https://tailscale.com/kb/1015/100.x-addresses\">CGNAT\nrange</a> <code>100.64.0.0/10</code>, which are not routable over the Internet.</li>\n</ul>\n<h6>Attacks</h6>\n<p>Here are several attack scenarios.</p>\n<p><strong>Packet-forwarding node on an untrusted LAN.</strong></p>\n<p><img alt=\"diagram showing a LAN machine connecting to a victim node\" src=\"https://tailscale.com/files/images/security-bulletins/2024-05-08-01.svg\" /></p>\n<p>A malicious host <code>10.0.0.1</code> on the same LAN as the packet-forwarding node\n<code>10.0.0.2</code> could craft packets with destination IP of a tailnet node\n<code>100.64.0.1</code> (using a command like <code>ip route add 100.64.0.1/32 via 10.0.0.2 dev eth0</code>) and send them to the packet-forwarding node. The packet-forwarding node\nwould accept them and forward them to the victim node. The victim node would\nsee a packet from <code>10.0.0.1</code> and accept it if the tailnet ACLs allow this\nsource IP.</p>\n<p>This scenario is very similar to a legitimate use-case of\n<a href=\"https://tailscale.com/kb/1214/site-to-site\">site-to-site</a> networking, where two subnets are bridged using\nTailscale subnet routers and the flag <code>--snat-subnet-routes=false</code>.</p>\n<p><strong>Malicious shared exit node</strong></p>\n<p><img alt=\"diagram showing a malicious exit node connecting a the victim node\" src=\"https://tailscale.com/files/images/security-bulletins/2024-05-08-02.svg\" /></p>\n<p>A malicious exit node <code>100.64.0.4</code> from tailnet A could craft packets with\ndestination IP of tailnet B node <code>100.64.0.3</code> and any source IP other than\n<code>100.64.0.4</code>. Due to the built-in <a href=\"https://tailscale.com/kb/1084/sharing#quarantine\">quarantining</a> of shared\nnodes, packets from <code>100.64.0.4</code> are rejected.</p>\n<h6>Mitigations</h6>\n<p>We implemented 3 separate mitigations for these attacks.</p>\n<p><strong>Redefine <code>\"src\": \"*\"</code> in ACLs</strong></p>\n<p>While <code>*</code> is a convenient shorthand in ACLs, Tailscale users almost never need\nto allow connections from any IP. In most cases users intend <code>*</code> as \"all other\nnodes in my tailnet\". As a mitigation for this vulnerability, we redefined <code>*</code>\nin <code>src</code> section of ACLs to include:</p>\n<ul>\n<li>all tailnet nodes</li>\n<li>all IPs from approved subnet routes</li>\n</ul>\n<p>The inclusion of IPs from approved subnet routes is needed for the site-to-site\nnetworking setup.</p>\n<p>For users that need the old semantics of <code>*</code> we added a new\n<a href=\"https://tailscale.com/kb/1337/acl-syntax#src\"><code>autogroup:danger-all</code></a>, which matches the old definition of <code>*</code>.</p>\n<p><strong>Stateful packet filtering on packet-forwarding nodes</strong></p>\n<p>On Linux packet-forwarding nodes we added stateful packet filtering. This means\nthat these nodes keep track of forwarded connections and only allow return\npackets for existing outbound connections. Inbound packets that don't belong to\nan existing connection are dropped.</p>\n<p>Because routing is implemented differently on non-Linux platforms, this\nmitigation is only necessary on Linux.</p>\n<p>Stateful filtering is enabled by default, except for existing subnet routers\nthat set <code>--snat-subnet-routes=false</code>. You can disable stateful filtering using\n<code>tailscale up --stateful-filtering=false</code>.</p>\n<p><strong>Client-side quarantining of shared nodes</strong></p>\n<p><a href=\"https://tailscale.com/kb/1084/sharing#quarantine\">Quarantining</a> of shared nodes was implemented by a packet\nfilter sent from the Tailscale control plane. This packet filter instructs\nnodes to drop any inbound connections from the source IP of the shared node. To\nprevent malicious shared exit nodes from crafting packets with different source\nIPs, additional client-side quarantining logic was added. The 1.66.0 and later\nclients reject all inbound connections from quarantined nodes, regardless of\ntheir source IP. This is similar to how the <a href=\"https://tailscale.com/kb/1072/client-preferences#allow-incoming-connections\">\"shields up\"</a> mode\nworks within the tailnet.</p>",
   summary_detail: {
      base: "https://tailscale.com/security-bulletins/index.xml",
      language: null,
      type: "text/html",
      value: "<p><strong><em>Description</em></strong>: Insufficient inbound packet filtering in subnet routers and exit nodes</p>\n<h5>What happened?</h5>\n<p>In Tailscale versions earlier than 1.66.0, <a href=\"https://tailscale.com/kb/1103/exit-nodes\">exit nodes</a>, <a href=\"https://tailscale.com/kb/1019/subnets\">subnet\nrouters</a>, and <a href=\"https://tailscale.com/kb/1281/app-connectors\">app connectors</a>, could\nallow inbound connections to other tailnet nodes from their local area network\n(LAN). This vulnerability only affects Linux exit nodes, subnet routers, and\napp connectors in tailnets where <a href=\"https://tailscale.com/kb/1018/acls\">ACLs</a> allow <code>\"src\": \"*\"</code>, such as\nwith <a href=\"https://tailscale.com/kb/1192/acl-samples#allow-all-default-acl\">default ACLs</a>.</p>\n<p>Tailscale version 1.66.0 fixes the vulnerability. Additionally, a server-side\nupdate changes the interpretation of <code>\"src\": \"*\"</code> to mitigate the issue\nspecifically for exit nodes.</p>\n<p>Special thanks to <a href=\"https://www.linkedin.com/in/hakan-ergan/\">Hakan Ergan</a> for reporting a similar\nconcern that led us to discover this vulnerability.</p>\n<h5>Who was affected?</h5>\n<p>This affected the following nodes using Tailscale version 1.65 or earlier:</p>\n<ul>\n<li>Exit nodes on Linux</li>\n<li>Subnet routers on Linux</li>\n<li>App connectors on Linux</li>\n<li>Regular nodes on all platforms connecting to the above nodes</li>\n</ul>\n<p>Tailnets with custom ACLs that do not use <code>\"src\": \"*\"</code> or any other value that\nincludes external IPs were not affected.</p>\n<p>We are not aware of any active exploitation of this vulnerability.</p>\n<h5>What was the impact?</h5>\n<p>Devices outside of the tailnet, but on the same LAN as an exit node, subnet\nrouter, or app connector could connect to ports on tailnet nodes that are\nallowed by ACLs.</p>\n<h5>What do I need to do?</h5>\n<p>Upgrade the following nodes to 1.66.0 or later:</p>\n<ul>\n<li>Subnet routers on Linux</li>\n<li>App connectors on Linux</li>\n<li>Regular nodes on all platforms that use exit nodes <a href=\"https://tailscale.com/kb/1084/sharing#sharing--exit-nodes\">shared from other\ntailnets</a> or <a href=\"https://tailscale.com/kb/1258/mullvad-exit-nodes\">Mullvad exit nodes</a></li>\n</ul>\n<p>We recommend enabling auto-updates and updating all nodes to the latest\nversion, but it is not required to mitigate this vulnerability.</p>\n<p>A server-side change mitigated this vulnerability for other types of affected\nnodes.</p>\n<h5>Technical details</h5>\n<p>Below, we refer to exit nodes, subnet routers, and app connectors as\n<em>packet-forwarding nodes</em>, because the details apply to all of them. Specific\ntypes of packet-forwarding nodes are mentioned explicitly where their behavior\nis different.</p>\n<p>Before 1.66.0, packets between regular nodes and destination hosts behind\npacket-forwarding nodes were filtered based on source/destination IP as\nspecified in tailnet ACLs. Specifying <code>\"src\": \"*\"</code> in ACLs is equivalent to\n<code>\"src\": [\"0.0.0.0/0\", \"::/0\"]</code>, meaning any IP address. This allowed source IPs\noutside of the tailnet to send packets to tailnet nodes via a packet-forwarding\nnode. This could be abused by malicious LAN hosts to connect into the tailnet\nusing a known tailnet node IP.</p>\n<p>The attack only works on a LAN because:</p>\n<ul>\n<li>it relies on next-hop routing, which only works in a LAN</li>\n<li>destination IPs are in the subnet router's approved range, or in the <a href=\"https://tailscale.com/kb/1015/100.x-addresses\">CGNAT\nrange</a> <code>100.64.0.0/10</code>, which are not routable over the Internet.</li>\n</ul>\n<h6>Attacks</h6>\n<p>Here are several attack scenarios.</p>\n<p><strong>Packet-forwarding node on an untrusted LAN.</strong></p>\n<p><img alt=\"diagram showing a LAN machine connecting to a victim node\" src=\"https://tailscale.com/files/images/security-bulletins/2024-05-08-01.svg\" /></p>\n<p>A malicious host <code>10.0.0.1</code> on the same LAN as the packet-forwarding node\n<code>10.0.0.2</code> could craft packets with destination IP of a tailnet node\n<code>100.64.0.1</code> (using a command like <code>ip route add 100.64.0.1/32 via 10.0.0.2 dev eth0</code>) and send them to the packet-forwarding node. The packet-forwarding node\nwould accept them and forward them to the victim node. The victim node would\nsee a packet from <code>10.0.0.1</code> and accept it if the tailnet ACLs allow this\nsource IP.</p>\n<p>This scenario is very similar to a legitimate use-case of\n<a href=\"https://tailscale.com/kb/1214/site-to-site\">site-to-site</a> networking, where two subnets are bridged using\nTailscale subnet routers and the flag <code>--snat-subnet-routes=false</code>.</p>\n<p><strong>Malicious shared exit node</strong></p>\n<p><img alt=\"diagram showing a malicious exit node connecting a the victim node\" src=\"https://tailscale.com/files/images/security-bulletins/2024-05-08-02.svg\" /></p>\n<p>A malicious exit node <code>100.64.0.4</code> from tailnet A could craft packets with\ndestination IP of tailnet B node <code>100.64.0.3</code> and any source IP other than\n<code>100.64.0.4</code>. Due to the built-in <a href=\"https://tailscale.com/kb/1084/sharing#quarantine\">quarantining</a> of shared\nnodes, packets from <code>100.64.0.4</code> are rejected.</p>\n<h6>Mitigations</h6>\n<p>We implemented 3 separate mitigations for these attacks.</p>\n<p><strong>Redefine <code>\"src\": \"*\"</code> in ACLs</strong></p>\n<p>While <code>*</code> is a convenient shorthand in ACLs, Tailscale users almost never need\nto allow connections from any IP. In most cases users intend <code>*</code> as \"all other\nnodes in my tailnet\". As a mitigation for this vulnerability, we redefined <code>*</code>\nin <code>src</code> section of ACLs to include:</p>\n<ul>\n<li>all tailnet nodes</li>\n<li>all IPs from approved subnet routes</li>\n</ul>\n<p>The inclusion of IPs from approved subnet routes is needed for the site-to-site\nnetworking setup.</p>\n<p>For users that need the old semantics of <code>*</code> we added a new\n<a href=\"https://tailscale.com/kb/1337/acl-syntax#src\"><code>autogroup:danger-all</code></a>, which matches the old definition of <code>*</code>.</p>\n<p><strong>Stateful packet filtering on packet-forwarding nodes</strong></p>\n<p>On Linux packet-forwarding nodes we added stateful packet filtering. This means\nthat these nodes keep track of forwarded connections and only allow return\npackets for existing outbound connections. Inbound packets that don't belong to\nan existing connection are dropped.</p>\n<p>Because routing is implemented differently on non-Linux platforms, this\nmitigation is only necessary on Linux.</p>\n<p>Stateful filtering is enabled by default, except for existing subnet routers\nthat set <code>--snat-subnet-routes=false</code>. You can disable stateful filtering using\n<code>tailscale up --stateful-filtering=false</code>.</p>\n<p><strong>Client-side quarantining of shared nodes</strong></p>\n<p><a href=\"https://tailscale.com/kb/1084/sharing#quarantine\">Quarantining</a> of shared nodes was implemented by a packet\nfilter sent from the Tailscale control plane. This packet filter instructs\nnodes to drop any inbound connections from the source IP of the shared node. To\nprevent malicious shared exit nodes from crafting packets with different source\nIPs, additional client-side quarantining logic was added. The 1.66.0 and later\nclients reject all inbound connections from quarantined nodes, regardless of\ntheir source IP. This is similar to how the <a href=\"https://tailscale.com/kb/1072/client-preferences#allow-incoming-connections\">\"shields up\"</a> mode\nworks within the tailnet.</p>",
   },
   title: "TS-2024-005",
   title_detail: {
      base: "https://tailscale.com/security-bulletins/index.xml",
      language: null,
      type: "text/plain",
      value: "TS-2024-005",
   },
}


Log in or create an account to share your comment.

Security Advisory comment format.

This schema specifies the format of a comment related to a security advisory.

UUIDv4 of the comment
UUIDv4 of the Vulnerability-Lookup instance
When the comment was created originally
When the comment was last updated
Title of the comment
Description of the comment
The identifier of the vulnerability (CVE ID, GHSA-ID, PYSEC ID, etc.).



Tags
Taxonomy of the tags.


Loading…

Loading…

Loading…

Sightings

Author Source Type Date

Nomenclature

  • Seen: The vulnerability was mentioned, discussed, or seen somewhere by the user.
  • Confirmed: The vulnerability is confirmed from an analyst perspective.
  • Exploited: This vulnerability was exploited and seen by the user reporting the sighting.
  • Patched: This vulnerability was successfully patched by the user reporting the sighting.
  • Not exploited: This vulnerability was not exploited or seen by the user reporting the sighting.
  • Not confirmed: The user expresses doubt about the veracity of the vulnerability.
  • Not patched: This vulnerability was not successfully patched by the user reporting the sighting.