Re: [tcpm] AccECN field order

[Bob Briscoe <[e-mail]>]

Michael,

On 17/11/2020 14:25, Scharf, Michael wrote:
>
> Inline some comments [ms].
>
> Given that I don’t ask for any text chances, I’ll not follow-up 
> forever in this discussion. I don’t own running code and a lot of 
> encoding details mainly affect an implementation.
>
> IMHO the rest of the WG has to think about the option design more than 
> me. So, this is mostly about triggering a discussion…
>
> *From:* Bob Briscoe <[e-mail]>
> *Sent:* Tuesday, November 17, 2020 1:20 PM
> *To:* Scharf, Michael <[e-mail]>; Yoshifumi 
> Nishida <[e-mail]>
> *Cc:* Michael Tuexen <[e-mail]>; [e-mail]; 
> Mirja Kuehlewind <[e-mail]>; Scheffenegger, Richard 
> <[e-mail]>; tcpm IETF list <[e-mail]>
> *Subject:* Re: AccECN field order
>
> Michael,
>
> On 17/11/2020 08:10, Scharf, Michael wrote:
>
>     Bob,
>
>     I am fine with the two option kinds proposed in -13, but I don’t
>     buy your arguments why this encoding is better than others.
>
>     Most importantly, I don’t think that your “forward compatibility”
>     argument is a very compelling reason for two codepoints. All
>     proposals I am aware of have their pros and cons. And I am not
>     aware of a really comprehensive discussion. At least neither the
>     e-mail below nor today’s meeting discuss all tradeoffs I would
>     investigate.
>
>
> [BB] You're right - the discussion below is useful and necessary.
>
>
>     For instance, the benefit of the length encoding variant would be
>     to consume only one TCP option codepoint now. If we decided later
>     to that a flag byte is needed (say, at the end), a follow-up
>     proposed standard could specify another option format with that
>     flag byte, using a second codepoint.
>
>
> [BB]
> * The new codepoint would require years to deploy (including in 
> proxies) before it works e2e.
>
> [ms] That may depend **a lot** on the environment. In environments 
> where ECN can get deployed today, say in datacenters, rolling out a 
> new kernel version may perhaps not be the biggest operational 
> challenge. Also, proxies may not be a significant problem in some of 
> those cases. Deployment roadmaps are very hard to predict.
>
>
> * Whereas the 'forward compatibility' approach tries to ensure that 
> AccECN implementations that are being built today will be ready to 
> interwork with a future extension.
> * In contrast, if your hypothetical new AccECN codepoint did need to 
> be deployed later, it would need to include a fall-back to the current 
> AccECN option when one peer doesn't understand the new codepoint.
>
> [ms] This is a reasonable point, albeit not a particularly strong one, 
> given that the option is not really required for basic 
> interoperability. Also, a hypothetical follow-up standard could 
> include a way to explicitly request the option…
>
>
> * Fall-back would need to avoid a second round of handshake (given 
> this is for low latency), which would only work reliably with an 
> option on the SYN (because the 3rd ACK is unreliable)
>
> [ms] I can’t follow. If an endpoint is unhappy with getting AccECN 
> option “A” from a peer, I could send a “please send me the better 
> AccECN option B” after the 3-WHS. It could be an upgrade, not a 
> fallback. Also, at this point we don’t know what option B would be and 
> in which context it would be needed. Maybe other protocol features 
> would require negotiation, too. IMHO it is a solvable problem.
>

[BB] Maybe. I prefer to work out full details of a protocol proposal 
before I trust it will be a viable alternative to another proposal.
Through Ilpo's criticism of the handshake reflection stuff we did, I 
learned a lot about how any option cleverness after the SYN/ACK is 
fraught with difficulties, given it cannot be known whether one end or 
the other can be guaranteed to send some data, to be able to piggy-back 
reliable delivery of an option. And you have to cater for all the 
possibilities for reordering the messages once you are past the 3WHS. 
Probably do-able, but complexity could be prohibitive, even for a simple 
upgrade.

>
> * AccECN has so far avoided using any SYN option space, which is 
> important given that is the scarcest resource.
>
> [ms] Yep, but as long as no proposal needs SYN options, this is 
> nothing that would make one solution better than another.
>
> These days, extending TCP is like playing chess - we have to think 
> multiple moves ahead!
>
> [ms] There is one fundamental difference between chess and TCP: In 
> chess you precisely know the current situation and can predict all 
> potential moves in advance (albeit there are a lot). However, we don’t 
> know the future evolution of TCP, ECN and the Internet as a whole, nor 
> future requirements. If we try to engineer a TCP extension for some 
> potential future, we may figure out later that reality is different 
> from what we designed for. As much as I appreciate your attempts to 
> predict the future, it would be a big surprise to me if you could 
> correctly predict what TCP will be in 10 years from now. So, instead 
> of trying to write specifications that take into account future 
> requirements, which are in fact just unknown, it is more important for 
> me to write effective, efficient and secure specifications for what is 
> currently known for sure.
>
> [ms] I **don’t** know for sure whether the current AccECN option will 
> be needed, but I see quite some evidence that support of the AccECN 
> option(s) is not a top priority these days. That is also something to 
> think about when designing the details.
>
> I need to learn to articulate onto the list all my reasons for 
> rejecting certain parts of the design space. Sorry about that.
>
> Yes, I admit, the current proposal is at the cost of burning two TCP 
> option codepoints. However, you said yourself when you proposed this 
> approach, that they are not (currently) a scarce resource. Certainly 
> not as scarce as option space. I just checked: roughly 15% of the 
> option codepoint allocation space has been used so far. So that's 
> where I would suggest we compromise first.
>
> [ms] I made a similar calculation some years ago and came to a similar 
> conclusion. Investing two codepoints for AccECN is something we can 
> probably afford. But nonetheless we have to reason carefully why it is 
> really worth the effort.
>
>     Regarding the resulting codepoint consumption (two option kinds)
>     and the addition of a flag byte, this approach would basically end
>     up with the same result as the current proposal in -13 plus some
>     hypothetical future addition leveraging the length field beyond
>     the currently known use.
>
>     And, yes, the length encoding variant may be less flexible and/or
>     consume some more bits in some cases, but on the plus side it only
>     needs one codepoint now – and given that it is entirely unclear
>     whether any AccECN option will indeed be widely used in future,
>     this is a big plus. The fact that implementers are quite silent on
>     the option design is not a good sign.
>
>
> [BB] Here's another possibility then.
>
> a) [K|L| EE1B | ECEB | EE0B ]
> b) [K|L| EE1B | ECEB | EE0B |F]    if F = 0bXXXXXXX0
> c) [K|L| EE0B | ECEB | EE1B |F]    if F = 0bXXXXXXX1
>
> We'd have to decide which order alternative (a) takes. I've picked 
> EE1B first only 'cos you're right that there is not currently as much 
> pressure (on the public Internet) for AccECN with ECT(0).
>
> ==Legend==
> K:    Kind (1B)
> L:    Length (1B)
> ExxB: Echo xx byte counter
> F:    Flags (1B)
>
> Alternatives a) and b) are equivalent, at least while the only 
> allocated flag is for field order.
> However, alternative b) is available in case other flags are allocated 
> in future
> The flags byte can only be omitted if (L % 3 == 2).
> The flags byte is considered present if (L % 3) == 0.
>
> Forward compatibility: Options with (L % 3 == 1) MUST be assumed to 
> include a flags byte, and current implementations ignore the last byte.
>
> The flags byte is optional to implement (even if the AccECN option is 
> implemented)
> If the server includes a flags byte on the SYN/ACK but the client does 
> not include one on the 3rd ACK or the first data packet, the server 
> assumes the client does not implement the flags byte and uses only 
> alternative a) for the remainder of the connection.
>
>
> [ms] I am not sure if we need that, but I take this as existence proof 
> that other encodings are possible. Personally, I think that other 
> solutions would be doable if you took one bit from one or multiple 
> counters for format encoding. At least for some of the byte counters I 
> don’t think taking one bit for other purposes would result in an 
> Internet congestion collapse… But, I have already said that the -13 
> proposal works for me, so I’ll not try to come up with another variant 
> myself.
>
>     To me, one potential difference between two proposals would be
>     incremental deployment. The proposal in -13 only has an advantage
>     if middleboxes such as firewalls will indeed pass TCP options with
>     a format that contains content beyond the (first) Accurate ECN
>     standard (i.e., currently unused length values). IMHO it is too
>     early to know whether firewalls would indeed allow this in future.
>
>     From a security perspective, it is not clear to me whether
>     allowing arbitrary unspecified bytes in a TCP option is a good
>     idea **at all**. It will be interesting to hear the opinion from
>     SEC area on that. Personally, I am not convinced that this really
>     makes sense, but I my concerns are not strong enough to formally
>     push back. I’ll leave it to others to think about whether this is
>     a bug or a feature.
>
>
> [BB] Well, let's first try to deal with security ourselves:
> * Octets that are explicitly declared as part of an option cannot be 
> used for a buffer overflow attack. I don't really need to, but I could 
> add the following text to the forward compatibility wording:
>     A receiver considers octets beyond those it uses, but within the 
> specified length, as if they are padding.
> * And such octets cannot be any different from the current ability of 
> a sender to add padding. So there's no new attack possible here.
> * And there's no need to specify a max length for any AccECN Option, 
> which would just unnecessarily limit flexibility.
>
> [ms] I have more thought about the covert channel that some 
> unspecified bytes in a standards track TCP option could offer… In 
> particular if the AccECN option gets widely deployed and used, would 
> some malicious users find those bytes useful? For what? For instance, 
> what in the AccECN protocol design prevents use of these “padding” 
> bytes as super-cookie? To me, it is much more important to think about 
> abuse of whatever we standardize **now**, instead of engineering for 
> some entirely unknown future. I am not a security expert, but I 
> suggest a careful analysis of any security implications of the AccECN 
> option. We seldomly specify new TCP options and malicious users will 
> certainly look for “opportunities” in the spec. So far, I don’t know 
> whether that whole idea of forward compatibility is a feature or a bug.
>

[BB] If that were a problem, it could already be done, because (by my 
reading of RFC793) it is valid to set the data offset to stretch out the 
padding to any length within the max option space limits.


Bob

> Michael
>
>     Maybe one lesson learnt is that the document could have a
>     non-normative appendix that explains the rationale for the finally
>     picked TCP option encoding. That may also help if there are
>     further questions whether two codepoints are really required, e.g.
>     by the IESG (if two codepoints are still the design after WGLC).
>     At least for past TCP option codepoint allocations I recall some
>     discussions late in the IETF process. In those past cases, good
>     arguments in an appendix and running code has helped a lot.
>
>
> [BB] I can do that. Appx B is already similar, giving Design Rationale 
> for the ACE field.
>
>
> Bob
>
>
>     Michael
>
>     *From:*Bob Briscoe <[e-mail]> <mailto:[e-mail]>
>     *Sent:* Tuesday, November 17, 2020 6:10 AM
>     *To:* Scharf, Michael <[e-mail]>
>     <mailto:[e-mail]>; Yoshifumi Nishida
>     <[e-mail]> <mailto:[e-mail]>
>     *Cc:* Michael Tuexen <[e-mail]>
>     <mailto:[e-mail]>; [e-mail]
>     <mailto:[e-mail]>; Mirja Kuehlewind
>     <[e-mail]> <mailto:[e-mail]>; Scheffenegger,
>     Richard <[e-mail]>
>     <mailto:[e-mail]>; tcpm IETF list
>     <[e-mail]> <mailto:[e-mail]>
>     *Subject:* Re: AccECN field order
>
>     Michael,
>
>     On 16/11/2020 17:36, Scharf, Michael wrote:
>
>         Bob,
>
>         One proposal using the length field with **one option
>         codepoint only** is detailed in:
>         https://mailarchive.ietf.org/arch/msg/tcpm/zo-1OR0nRfhHocX8yvTvpC4BNMo/
>
>         It is the third option mentioned in this e-mail. One example
>         would be to use option length values 5/8/11 for one encoding
>         type and option length values 6/9/12 for the other encoding
>         type (i.e., order of fields). Or one could use some other
>         combination of length values – the only requirement is that a
>         certain value for the option length is only used by one of the
>         option formats. In that approach, the value of the length
>         field would thus directly describe the encoding of the option.
>         Unless I miss something, this would work and it would just
>         require one option codepoint.
>
>         Thus, alternatives to two option codepoints exist and I have
>         explained them on the list in March 2020.
>
>
>     OK, sorry, yes, I remember this now.
>
>     As I will explain in the AccECN status update talk today in
>     virtual Bangkok, the draft has made provision for different length
>     values than 5/8/11. It says existing implementations MUST accept
>     length values other than those currently defined. But then read in
>     as many whole 3-byte fields as they can.
>
>     This can be used to add a flags byte on the end in future, for
>     extensibility. Or any other form of extensibility the WG might
>     decide in the future.
>
>     I know a flags byte at the end seems odd compared to at the
>     beginning. But (if decided it's needed in future) it's reasonably
>     easy to implement by reading the whole option, then processing the
>     last byte, before reading the rest of the option.
>
>     I believe you will agree that this is a better way to utilize
>     different lengths.
>
>     And thank you for repeatedly emphasizing that you're happy with
>     the 2-kind scheme, or other alternatives.
>
>
>     Bob
>
>
>
>         Anyway, I don’t really care how the options are encoded as
>         long as the receiver doesn’t need per-connection state for
>         decoding a TCP option. So, personally, I would be fine with
>         using e.g. the length field as described in my old e-mail. Or
>         an additional flag byte. And one could come up with further
>         encodings, e.g., by using one or a few bits as a short “type”
>         field for each counter. This is all about protocol
>         engineering. And all these variants have their pros and cons.
>
>         I am also fine with using two option codepoints as specified
>         in -13; this is probably the approach that consumes the least
>         number of bits.
>
>         Michael (w/o any hat)
>
>         *From:* Bob Briscoe <[e-mail]>
>         <mailto:[e-mail]>
>         *Sent:* Monday, November 16, 2020 5:52 PM
>         *To:* Yoshifumi Nishida <[e-mail]>
>         <mailto:[e-mail]>
>         *Cc:* Scharf, Michael <[e-mail]>
>         <mailto:[e-mail]>; Michael Tuexen
>         <[e-mail]> <mailto:[e-mail]>;
>         [e-mail] <mailto:[e-mail]>; Mirja
>         Kuehlewind <[e-mail]> <mailto:[e-mail]>;
>         Scheffenegger, Richard <[e-mail]>
>         <mailto:[e-mail]>; tcpm IETF list
>         <[e-mail]> <mailto:[e-mail]>
>         *Subject:* AccECN field order
>
>         Yoshi, (adding the tcpm list in cc)
>
>         On 05/11/2020 06:58, Yoshifumi Nishida wrote:
>
>             Hi Bob,
>
>             On Wed, Nov 4, 2020 at 3:29 PM Bob Briscoe
>             <[e-mail] <mailto:[e-mail]>> wrote:
>
>                 Yoshi,
>
>                 On 04/11/2020 06:51, Yoshifumi Nishida wrote:
>
>                     Hi, folks,
>
>
>
>
>
>                     In my understanding, I'm not sure if we settled
>                     down on using two option kinds or encoding schemes
>                     for 24bits fields in acc ecn draft.
>
>                     So, I think there're still something to be
>                     clarified and hope things will be settled at the
>                     meeting.
>
>
>                 [BB] I know a WG can change it's mind at any time. But
>                 I'd rather we just clarified what a previous decision
>                 was, to avoid the need to keep re-opening discussion
>                 on a question that have been decided then changed
>                 three different ways already.
>
>                 My memory is not so good these days. I trusted that
>                 Michael S remembered the decision correctly, and I
>                 seem to remember that decision being made.
>                 I've just checked the minutes of the last interim:
>                 https://datatracker.ietf.org/doc/minutes-interim-2020-tcpm-01-202004291600/
>                 and they mention Michael's proposal to use two kinds,
>                 but don't record any decision.
>                 The jabber log gives no clues about any decision.
>
>                 I can't find an audio or video recording. Can you
>                 point me at one?
>
>             I thought that it's because there was no clear decision at
>             the meeting.
>
>             But, you can check https://www.youtube.com/watch?v=KsB0_Nb8-kA
>
>             Please let us know if you have any questions or opinions
>             with regard to this.
>
>
>         [BB] I checked the Youtube link you sent below.
>
>         First I think we're agreed no-one was fighting for us to keep
>         the previous way we did this (using the initial value of the
>         field to set the order for the connection).
>         In my presentation I said there was strong resistance from
>         Michael to do it a different way.
>         (also, offlist, the co-authors including me also didn't like
>         this so much. And Ilpo said it made the implementation complex.)
>
>         Then came the question of what we do instead. There were three
>         alternative proposals:
>         a) use 2 option kinds
>         b) add a flags byte
>         c) somehow use the length field maybe
>
>         Michael raised (c) in the meeting as a possibility, but no-one
>         could think how to distinguish two options of the same length
>         but a different field order using the length field. Michael
>         said he'd post any ideas to the list if he could think of any,
>         but that didn't happen.
>
>         So we're left choosing between (a) and (b).
>         I said in the meeting (and on the list when discussing with
>         Ilpo) that I'd be happy to go with (b), but only if there was
>         another use for a flag. Because it would consume 1B more
>         options space in many packets, which is a scarce resource.
>
>         Ilpo had a proposed use for another flag (to help synch
>         counters after a loss), but I think the discussion about it
>         ended that it wouldn't be helpful, 'cos the way it worked
>         depended on itself (circular logic).
>
>         In conclusion, I don't think there was an explicit decision to
>         go with 2 option kinds, but it ended up as the 'last person
>         standing'.
>         I like it. It's simple. And apparently option kinds are not
>         such a scarce resource.
>
>         Perhaps we can ratify this in the WG tomorrow.
>
>
>         Bob
>
>
>
>
>             Thanks,
>
>             --
>
>             Yoshi
>
>
>
>
>
>         -- 
>
>         ________________________________________________________________
>
>         Bob Briscoehttp://bobbriscoe.net/
>
>
>
>
>     -- 
>
>     ________________________________________________________________
>
>     Bob Briscoehttp://bobbriscoe.net/
>
>
>
> -- 
> ________________________________________________________________
> Bob Briscoehttp://bobbriscoe.net/

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/