Re: [Int-area] Adam Roach's Discuss on draft-ietf-intarea-provisioning-domains-10: (with DISCUSS and COMMENT)

[Adam Roach <[e-mail]>]

Thanks for the explanation and the further proposed mitigation.

Allowing the RA to specify an arbitrarily small "Delay" parameter seems 
to still allow for a pretty big burst of traffic. If I read the proposed 
interpretation of the "Delay" bits correctly (2**(Delay * 2)), the 
current behavior is specified to allow a delay upper bound selected from 
one of the following (approximate) values:

  * 1 ms
  * 4 ms
  * 16 ms
  * 64 ms
  * 256 ms
  * 1 second
  * 4 seconds
  * 16 seconds
  * 1 minute
  * 4 minutes
  * 17 minutes
  * 70 minutes
  * 4 hours, 40 minutes
  * 18 hours 38 minutes
  * 3 days, 3 hours
  * 1 week, 5 days


That's a pretty breathtaking scope, and it's hard to imagine that the 
first six or so are strictly needed, while all six are in a range that 
might overload a DDoS target. The final several seem a bit questionable 
as well, given normal operational timelines for network attachment. If 
the formula were revised to, e.g., "2**(Delay + 12)" instead of the 
current formula, you would have an enforced lower bound of roughly four 
seconds (which should be enough to blunt most DDoS attacks), and an 
upper bound of roughly 37 hours (which still seems excessive, although 
not quite as much as the previous upper bound).

Assuming the additional mitigation you propose below (10 maximum 
failures per attachment) as well as some means of achieving a 
lower-bound for "Delay" on the order of multiple seconds, I think I'm 
good clearing when a new version comes out.

Thanks for your work in thinking through practical solutions to this issue.

/a

On 1/22/20 16:22, Tommy Pauly wrote:
> Hi Adam,
>
> Thanks for taking a look! I'd like to avoid adding extra checks, such 
> as looking for particular DNS records, to avoid deployment complexity 
> and more opportunities for incomplete configuration. As such, I'd like 
> to dig into this a bit further.
>
> If the attacker in this case is a rogue actor on a local network 
> sending out RAs on their local link, any given attacking host would be 
> restricted in their scope to the devices they can reach. Of course, 
> there could be coordination across many different local networks 
> simultaneously, but that also requires more work on the side of the 
> attacker.
>
> The reason for the delay was to limit the impact on a relatively 
> low-powered and unsophisticated local HTTPS server for serving PvD 
> information, which may itself be on the router. I imagine that any 
> large web server deployment would not have any issue with the load 
> generated from a particular local network. Specifically, if we are 
> limiting any given host to requesting only a few times within a 10 
> second window on the network at all, and the number of hosts on the 
> network is bounded, the number of opportunities for the attacker to 
> cause load on the servers is limited.
>
> Another option, to avoid remaining concern about hitting wildcarded 
> hosts, is to simply say that if the host keeps receiving PvD IDs with 
> bogus (failing) servers, it disables all fetching of additional 
> information for the duration of the network attachment. Networks that 
> do implement better control over RAs (RA-guard, etc) presumably won't 
> have this issue, and since the additional info is optional, it 
> shouldn't cause any major connectivity issues.
>
> If we require such a limit (you only get to fail to fetch 10 times 
> total per attachment, say), does that mitigate things?
>
> Thanks,
> Tommy
>
>> On Jan 22, 2020, at 1:51 PM, Adam Roach <[e-mail] 
>> <mailto:[e-mail]>> wrote:
>>
>> Thanks! The new text is good, but I don't think it's sufficient. I 
>> have two remaining concerns in particular:
>>
>>   * The mitigation for wildcarded web hosts appears inadequate,
>>     especially given:
>>   * The mechanism clearly anticipates a scale where it can generate
>>     *single* short torrential burst sufficient to knock an average
>>     server over (hence the random delay mechanism for fetching data
>>     over HTTP). Given that fact, simple rate-limiting will never be
>>     enough if a single tight burst of traffic can be orchestrated.
>>
>> The more I think about it, the more I believe the TXT-based opt-in 
>> solution I proposed in my earlier email is a reasonable approach to 
>> protect general-purpose web servers from PvD-client-based attacks.
>>
>> One further comment inline below.
>>
>> /a
>>
>> On 1/22/20 15:17, Tommy Pauly wrote:
>>> Hi Adam,
>>>
>>> Thanks again for bringing this up! I've updated our text to include 
>>> mitigations for this attack. It can be found here 
>>> (https://github.com/IPv6-mPvD/mpvd-ietf-drafts/pull/25), but here's 
>>> an overview of the proposed text:
>>>
>>> In Section 4.1, I've added two new paragraphs. The first describes 
>>> time limits on fetching PvD info:
>>>
>>>     In addition to adding a random delay when fetching Additional
>>>     Information, hosts
>>>     MUST enforce a minimum time between requesting Additional
>>>     Information
>>>     for a given PvD on the same network. This minimum time is
>>>     RECOMMENDED
>>>     to be 10 seconds, in order to avoid hosts causing a
>>>     denial-of-service on the
>>>     PvD server. Hosts also MUST limit the number of requests that
>>>     are made to
>>>     different PvD Additional Information servers on the same network
>>>     within a short
>>>     period of time. A RECOMMENDED value is to issue no more than
>>>     five PvD
>>>     Additional Information requests in total on a given network
>>>     within 10 seconds.
>>>     For more discussion, see {{security}}.
>>>
>>> The second also makes clear the behavior to take in case of failure, 
>>> which will be the case for non-PvD web servers:
>>>
>>>
>>>     If the request for PvD Additional Information fails due to a TLS
>>>     error,
>>>     an HTTP error, or because the retrieved file does not contain
>>>     valid PvD JSON,
>>>     hosts MUST close any connection used to fetch the PvD Additional
>>>     Information,
>>>     and MUST NOT request the information for that PvD ID again for
>>>     the duration
>>>     of the local network attachment. For more discussion, see
>>>     {{security}}.
>>>
>>> In addition, I added text to the Security Considerations:
>>>
>>>     An attacker generating RAs on a local network can use the H-flag
>>>     and the PvD ID
>>>     to cause hosts on the network to make requests for PvD
>>>     Additional Information
>>>     from servers. This can become a denial-of-service attack if not
>>>     mitigated.
>>>
>>
>> This doesn't really convey the amplification involved, which I think 
>> is highly relevant.
>>
>>
>>>     To mitigate
>>>     this attack, hosts MUST limit the rate at which they fetch a
>>>     particular PvD's
>>>     Additional Information, limit the rate at which they fetch any
>>>     PvD Additional
>>>     Information on a given local network, and stop making requests
>>>     to any PvD ID
>>>     that does not respond with valid JSON. Details are provided in
>>>     {{retr}}. This attack
>>>     can be targeted at generic web servers, in which case the host
>>>     behavior of stopping
>>>     requesting for any server that doesn't behave like a PvD
>>>     Additional Information server
>>>     is critical. For cases in which an attacker is pointing hosts at
>>>     a valid PvD Additional
>>>     Information server (but one that is not actually associated with
>>>     the local network),
>>>     the server SHOULD reject any requests that do not originate from
>>>     the expected IPv6
>>>     prefix as described in {{serverop}}.
>>>
>>> The existing text referenced here about server behavior is:
>>>
>>>     The server providing the JSON files SHOULD also check whether the
>>>     client address is contained by the prefixes listed in the additional
>>>     information, and SHOULD return a 403 response code if there is no
>>>     match.
>>>
>>> Let me know if this addresses your concerns!
>>>
>>> Best,
>>> Tommy
>>>
>>>> On Jan 21, 2020, at 9:26 PM, Adam Roach via Datatracker 
>>>> <[e-mail] <mailto:[e-mail]>> wrote:
>>>>
>>>> ----------------------------------------------------------------------
>>>> DISCUSS:
>>>> ----------------------------------------------------------------------
>>>>
>>>> Thanks to the authors and working group for their work on this 
>>>> document.  I
>>>> have one major concern about the ability for this mechanism to be 
>>>> abused to
>>>> form DDoS attacks, described below. Unfortunately, while I have 
>>>> identified the
>>>> attack, I don't have an easy solution to propose that mitigates it 
>>>> satisfactorily.
>>>>
>>>> I also have a handful of mostly editorial comments on the document.
>>>>
>>>> ---------------------------------------------------------------------------
>>>>
>>>> §6:
>>>>
>>>> I was expecting to see a discussion of the DDoS attack that may 
>>>> result from a
>>>> large network (or a rogue host on such a network) sending out a PvD ID
>>>> containing the hostname of a victim machine, and setting the "H" flag.
>>>>
>>>> Since the messages used to trigger these HTTP connections are extremely
>>>> lightweight, unauthenticated UDP messages, and the resulting HTTP 
>>>> connections
>>>> require the exchange of a significant number of packets in addition 
>>>> to a
>>>> number of cryptographic operations, this is a very high ratio 
>>>> amplification
>>>> attack, both in terms of network and CPU resources.
>>>>
>>>> Given that the delay setting comes from the network instead of being
>>>> independently computed by the host, such an attack could be honed to be
>>>> particularly devastating.  Although it isn't a complete mitigation, one
>>>> approach to consider would be moving computation of the delay upper 
>>>> bound to
>>>> the host, or specifying a minimum upper bound of several minutes 
>>>> (where a
>>>> smaller value will cause the host to use this minimum upper bound).
>>>>
>>>> Regardless of how this is ultimately handled, I think this is a 
>>>> pretty severe
>>>> risk that needs addressing in the document prior to publication.
>>>>
>>>
>>
>