Re: [TLS] Record-level ACKs in DTLS 1.3

[Hanno Becker <[e-mail]>]

Hi Ekr,

thanks for your prompt reply.

> Thanks for your note. I don't think your proposal will be an
> improvement. It destroys information which could otherwise be used for
> improve round-trip and loss estimation (cf. the difference between
> QUIC and TCP ACKs).

What information is destroyed that was there before? You can still
use handshake metadata in place of record sequence numbers and
do the same analysis as before.
Also, with QUIC and TCP, we're mainly talking about throughput optimization
at the application stage, where DTLS doesn't ACK anyway.

> Second, it prevents the receiver from saying
> some non-sensical things like acknowledging part of a received packet.
> It's of course true that the current design allows you to ACK
>non-received packets, but that's much more straightforward to detect.

I don't think this is actually a complication: In both cases you
have a list of messages you've sent which are tagged somehow -- either
by a record sequence number or handshake metadata which at this point
can be treated opaque -- and if you receive an ACK for a non-existent
tag, you ignore it or fail, whatever the policy.

Beyond that, I consider the more fine-grained ACKing at the handshake
level a benefit, because it allows to acknowledge parts of records
containing multiple handshake messages only some of which could be
processed. Example:

    +---------------------------------+
    | Seq No 1                        |---------------> Received
    | Fragment 0..a                   |
    +---------------------------------+

    +---------------------------------+     lost
    | Seq No 1                        |----------x
    | Fragment a+1..b                 |
    +---------------------------------+

    +-----------------+----------+
    | Seq No 1        | Seq no 2 |-------------------> Received
    | Fragment b+1..N | complete |
    +-----------------+----------+

​If this happens with an implementation which only supports contiguous
reassembly (I think GnuTLS does it this way, or at least used to), it
will drop the second fragment, but the second message might still be
buffered and hence ACK'ed. With record-level ACKs, such fine-grained
ACKing isn't possible, and HS Seq No 2 will need to be resent even
though it has been received.

>  I don't think your proposal will be an improvement.

I uphold this: ACKing at the handshake level does not lead to complications
while (a) easing memory efficient implementations for IoT devices and
(b) increasing conceptual clarity by avoiding the current semantic ambiguity
of record level ACK with dependency on implementation-specific handshake
level details like buffering and reassembly strategy.

> Yes, I agree that you need to only ACK data which you have processed.
> I would be happy to take a PR to clarify this point.

It's actually a MUST only, not a need only, which is an important point
because it means that ACKing can _not_ be implemented as an automatic
mechanism at the record layer.
I'll file a PR once the discussion has concluded.

Best,
Hanno

________________________________
From: Eric Rescorla <[e-mail]>
Sent: Friday, February 28, 2020 2:44 PM
To: Hanno Becker <[e-mail]>
Cc: [e-mail] <[e-mail]>
Subject: Re: [TLS] Record-level ACKs in DTLS 1.3

Hanno,

Thanks for your note. I don't think your proposal will be an
improvement. It destroys information which could otherwise be used for
improve round-trip and loss estimation (cf. the difference between
QUIC and TCP ACKs). Second, it prevents the receiver from saying
some non-sensical things like acknowledging part of a received packet.
It's of course true that the current design allows you to ACK
non-received packets, but that's much more straightforward to
detect.

I agree that the current design requires keeping somewhat more
state during the period when no ACKs have been received. However,
your proposal actually requires retaining a similar data structure
once they have been received.

Some detailed comments below.





> Hi,
>
> TL;DR
> This is all about various aspects of how ACKs work in DTLS 1.3:
> - The DTLS 1.3 specification requires clarification regarding
>   when ACKs should be sent.
> - Record-level ACKs make efficient implementations for IoT
>   devices harder. I argue that handshake-level ACKs reduce
>   implementation complexity and allow for optimized
>   implementations.
>
> Details:
> To illustrate, consider the following flight exchange, where the
> second and third message in the second flight get reordered:
>
>       Client                                  Server
>    +----------+
>    | Seq No 1 |-------------------------->   Received
>    | Rec No a |
>    +----------+
>                                          +---------------+
>     Received    <------------------------|   Seq No 1    |
>                                          |   Rec No b    |
>                                          +---------------+
>                                          +---------------+
>                            +-------------|   Seq No 2    |
>                            |             |   Rec No c    |
>                            |             +---------------+
>                            |
>                            |             +---------------+
>     Received    <------------------------|   Seq No 3    |
>                            |             |   Rec No d    |
>                            |             +---------------+
>   +-----------+            |
>   |    ACK    |------------------------->
>   | RecNo ??? |            |
>   +-----------+            |
>                            |
>                            |
>                            |
>     Received    <----------+
>
> The specification recommends that the client SHOULD send an ACK
> when it receives out-of-order handshake message - here message (3,d)
> while awaiting (2,c). However:
>
> Question:
> Which records does the client acknowledge in the ACK?
>
> This is in fact implementation-dependent:
> The client must only acknowledge record (3,d) if it has buffered it.
> Implementations which don't implement out of order buffering of handshake
> messages (to save RAM and ROM) must not acknowledge messages that weren't
> buffered. The reason is that otherwise the server is mislead in which messages
> need resending and which don't.
>
> Conclusion 1:
> If ACKs must only be sent for records which contain handshake messages
> which were actually fed into the flight buffering / reassembly module
> of the implementation, the specification should clearly say so.
> I consider this point to be prone to misinterpretation, potentially
> leading to incompatible implementations, because ACKs acknowledge
> receipt at record granularity, and yet they must be sent only under
> some assumptions on how their content was processed.

Yes, I agree that you need to only ACK data which you have processed.
I would be happy to take a PR to clarify this point.


> As I understand, the simplest implementation of the retransmission state
> machine using record-level ACKs works by buffering copies of records sent
> in a flight and retransmits those that don't get acknowledged. The record
> sequence numbers of the original and retransmitted records are internally
> associated with the opaque record content, so that ACKs for both the original
> and the retransmitted records can be recognized.
>
> In this approach, the record contents are opaque to the sender's retransmission
> state machine, which simplifies the implementation. However, it has major drawbacks:
>
> (1) It requires buffering the entire flight, which incurs high memory
>     overhead that can be significant on constrained devices.
>
>     Implementations should be allowed to not buffer handshake
>     messages but re-generate them on the fly through a callback whenever
>     retransmission is needed.

Note that many handshake messages cannot be re-generated without extensive
caching (for instance, ServerHello). However, as long as the messages
are generated consistently (a requirement in any case), then you in
fact need-not buffer them in memory; you merely keep the record -> offset
mappings as an edit list.


>     Two examples:
>     - Consider the Certificate message: The raw certificate
>       (chain) must reside in RAM/ROM already, and the Certificate message
>       could easily be re-regenerated from that. Buffering handshake messages
>       instead creates significant and unnecessary overhead.
>
>     - This is of increasing importance with the advent of post-quantum
>       cryptography, which comes with significantly larger key material.
>
> (2) It doesn't allow switching the MTU for retransmission.
>
>     Once the MTU may change for retransmission, the sender cannot keep
>     track anymore of a single set of record sequence numbers per message such
>     that an ACK for any of them confirms receipt of the message.
>
> Of course, no implementation is strictly _forced_ to follow the above approach,
> but the current record-level ACKs design significantly hardens any other approach:



> For example, retransmission through callbacks or support for MTU switching
> requires maintaining the tuples (record seq nr, lists of handshake fragments)
> for all transmissions of the last flight. (Note that, in particular, just
> remembering the mapping for the last transmission isn't enough, since an
> acknowledgement for an earlier transmission might arrive late.)

Well, sort of. Nothing forbids duplicate transmission, so just remembering
the last transmission is mostly just inefficient.


> Conclusion 2:
> Handshake-level ACKs simplify the retransmission state machine
> for non-buffering implementations on constrained devices, while
> not hardening buffering implementations using record-level ACKs
> so far. I'm aware of the lateness of this proposal, but would be
> happy if the group would discuss and consider handshake-level ACKs
> instead of record-level ACKs.

As noted above, I do not believe we should make this change.

-Ekr


IMPORTANT NOTICE: The contents of this email and any attachments are confidential and may also be privileged. If you are not the intended recipient, please notify the sender immediately and do not disclose the contents to any other person, use it for any purpose, or store or copy the information in any medium. Thank you.