Re: Blocking packets from suspicious ports

[Christian Huitema <[e-mail]>]

On 5/3/2022 11:53 PM, Willy Tarreau wrote:
> On Tue, May 03, 2022 at 11:00:11PM -0700, Christian Huitema wrote:
>> QUIC includes a feature to facilitate moving a connection from an anycast
>> address to a unicast address. The client finds the anycast address through
>> the DNS, contacts the server, learns a "preferred server address" during the
>> handshake, and tries validating that preferred address before using it for
>> the rest of the connection duration, without risk losing the connection to a
>> routing event that redirects anycast to a different server. Which is nice,
>> but of course a villainous server could feed a bogus preferred address.
>> That's the client bounce for you. All the server packets before the bounce
>> are sent with the expected 5 tuple. And of course the "preferred address" is
>> sent using encrypted packets.
> Ah, I wasn't aware, seen like this, that's fun. Well, I guess the client
> will try to connect to the new address with some info allowing to identify
> the origin server (e.g. SNI), thus allowing a QUIC decoder/server at that
> address to spot where the client found the address, no ?

Everything is encrypted, except for a few bytes in the packet header 
that carry a connection identifier, i.e., an octet string chosen by the 
server. The server provides the client with a list of those; if needed, 
that list is known by the load balancer at the server side, so the LB 
can forward the packet to the right server in the farm. The encryption 
key is specific to the connection, so only the right server can respond.

>>> There *will* inevitably be some problems at the beginning, but one good
>>> thing is that solutions are reasonably simple, can instantly be deployed
>>> in field using port filtering, and will quickly land in the code itself
>>> because the various stack authors have full control of their code.
>> Port filtering at the edge will not catch the "preferred address attack" in
>> which the server proposes a non-anycast address to the client, so I guess
>> QUIC stacks will have to protect that. As in, if a server says "please move
>> the connection to 10.0.0.1:53", the client should just decline and stick to
>> the anycast address. And yes, using a configurable list makes sense.
> Yes, that's a good point, I agree. I think quite a bit of the burden will
> be on the clients' shoulder. It reminds me the good joke we had in the 90s,
> creating a "warez.<domain>" DNS entry pointing to 127.0.0.1. I've personally
> seen someone trapped and saying "what, they stole'my data!" after ftping
> there :-)
>
>>>> Yes. Should that really be "per source port" or "per source IP + port"?
>>> Per port should be sufficient. There's no reason from the server side that
>>> multiple clients collectively decide to use the same source port to send
>>> legit traffic. And using the port alone will allow to quicky detect a new
>>> internet-wide attack coming from a randomly vulnerable service (IoT
>>> gateways, ToIP etc) regardless of their address. I don't know how this
>>> would be distributed, but I guess that if a source port is responsible
>>> for 100 times more traffic than all other one combined, it will usually
>>> be a good indicator that you don't want to deal with that traffic anymore.
>> I would expect quite a bit of legit server-to-server traffic on ports 443 or
>> 853,
> I think that's exactly the thing we must avoid. Encouraging (or even
> allowing) this is exactly what will make it impossible for infrastructure
> to defend itself against massive attacks. There's no value in using the
> same port on both sides. The savings of sockets and source ports are
> already enormous, going from N to 1 from TCP to QUIC, if an agent needs
> to make outgoing connections, it certainly can afford a second socket,
> which is nothing compared to the hundreds of thousands it would have used
> in TCP. Not being willing to do that is putting the internet at risk. For
> example among the protections you usually find at the edge is the blocking
> of a TCP SYN packet from a source port below 1024. A few services in the
> past (rlogin, rsh) used to bind to ports 512-1023 to "prove" that the
> user ID was authenticated by the local OS, but that's never seen over the
> net. One service has caused trouble to this, active FTP with source port
> 20, and rules which were a bit too lax used to allow it to connect to any
> service (e.g. SMTP) thereby allowing to use FTP servers to send spams by
> retrieving files that contained SMTP protocol contents. This has
> contributed to making active FTP unpopular, and nowadays it has become
> safe to block SYN from sources < 1024 at the edge. UDP doesn't have such
> a thing as a SYN flag and it's critical that traffic cannot be made
> symmetrical, or there's no more infrastructure filtering and only
> application level filtering.

Too bad that nobody discussed that during the review of 
https://datatracker.ietf.org/doc/draft-ietf-quic-manageability/, that 
could have been a nice place for such considerations.

It seems that we really need to write all these recommendations in a 
draft. Services that are peer-to-peer in nature would tend to use a 
single port.Same for QUIC stacks. It is a little bit easier to use a 
single socket than to force outgoing connections on a single socket. If 
we want them to use two separate ports, we have to explain convincingly.

On the other hand, I think we are past the "port below 1024" rules. For 
example, memcached is on port 11211, MDNS on port 5353, and there are 
many more like that.

>> but yes, keeping a dynamic list of most abused ports makes sense.
>> Although that's mostly an issue for volumetric attacks, and host based
>> defenses are not too efficient there, but still it feels better than just
>> receiving the attack and waiting until it stops.
> Definitely.
>
>>>>> I think we'll discover new fun things over time and will learn new
>>>>> attacks and workarounds as deployments grow.
>>>> I have not heard of actual attacks "in the wild", but who knows...
>>> It's too early, stacks are still evolving and the level of deployment
>>> is not high yet. That doesn't mean nobody is trying to elaborate new
>>> attacks. For example yesterday on my home servers which do not advertise
>>> QUIC I received one short packet from port 443 to port 443 from a host
>>> named "scanners.labs.something", and a 1238-byte one from a random port
>>> to 443 from a host named "scan-19a.somethingelse". Thus it shows that
>>> studies are already in progress. We just need to be patient :-)
>>
>> We probably will not have to wait too long...
> I hope so. It's important to get real-world feedback on protocols early
> enough to allow them to improve. It's especially important right now that
> any QUIC site can fall back to H1/H2 over TCP so it's easy to just block
> it in case of massive failure. If big problems are reported only 10 years
> after deployment it will be much harder to disable it in emergency.

Yes, but out of our control...

-- Christian Huitema