Re: [tcpm] AccECN field order

[Mirja Kuehlewind <[e-mail]>]

Hi all,

I think I also support two options as this is the most simple solution. 

We probably need to look at section 3.2.3.2. (Path Traversal of the AccECN Option) again, as I think this section assumes mostly one option. Or I guess we could say that only one option should be used for a connection.

Regarding use of the option length for extensibility. I know new options are hard to deploy but that’s the extensibility mechanism we have with TCP. Not sure we want to invent a new now. If we think we need more extensibility for AccECN for any reason, we should specify valid option length values that include a flags field now.

Mirja



> On 18. Nov 2020, at 01:03, Neal Cardwell <[e-mail]> wrote:
> 
> As someone who contributes to a TCP implementation, I would put in a  vote for the "Two Option Kinds" proposal outlined in slide 6 here:
> 
>  https://datatracker.ietf.org/meeting/109/materials/slides-109-tcpm-more-accurate-ecn-feedback-in-tcp-ecn-adding-explicit-congestion-notification-01#page=6
> 
> This approach has the virtue of simplicity, which hopefully may help avoid middlebox bugs, speed standardization and development, and ease the process of maintaining fast and stable implementations.
> 
> neal
> 
> 
> On Tue, Nov 17, 2020 at 9:25 AM Scharf, Michael <[e-mail]> wrote:
> Inline some comments [ms].
> 
>  
> 
> Given that I don’t ask for any text chances, I’ll not follow-up forever in this discussion. I don’t own running code and a lot of encoding details mainly affect an implementation.
> 
>  
> 
> IMHO the rest of the WG has to think about the option design more than me. So, this is mostly about triggering a discussion…
> 
>  
> 
> From: Bob Briscoe <[e-mail]> 
> Sent: Tuesday, November 17, 2020 1:20 PM
> To: Scharf, Michael <[e-mail]>; Yoshifumi Nishida <[e-mail]>
> Cc: Michael Tuexen <[e-mail]>; [e-mail]; Mirja Kuehlewind <[e-mail]>; Scheffenegger, Richard <[e-mail]>; tcpm IETF list <[e-mail]>
> Subject: Re: AccECN field order
> 
>  
> 
> Michael,
> 
> On 17/11/2020 08:10, Scharf, Michael wrote:
> 
> Bob,
> 
>  
> 
> I am fine with the two option kinds proposed in -13, but I don’t buy your arguments why this encoding is better than others.
> 
>  
> 
> Most importantly, I don’t think that your “forward compatibility” argument is a very compelling reason for two codepoints. All proposals I am aware of have their pros and cons. And I am not aware of a really comprehensive discussion. At least neither the e-mail below nor today’s meeting discuss all tradeoffs I would investigate.
> 
> 
> [BB] You're right - the discussion below is useful and necessary.
> 
> 
> 
>  
> 
> For instance, the benefit of the length encoding variant would be to consume only one TCP option codepoint now. If we decided later to that a flag byte is needed (say, at the end), a follow-up proposed standard could specify another option format with that flag byte, using a second codepoint.
> 
> 
> [BB] 
> * The new codepoint would require years to deploy (including in proxies) before it works e2e.
> 
>  
> 
> [ms] That may depend *a lot* on the environment. In environments where ECN can get deployed today, say in datacenters, rolling out a new kernel version may perhaps not be the biggest operational challenge. Also, proxies may not be a significant problem in some of those cases. Deployment roadmaps are very hard to predict.
> 
> 
> * Whereas the 'forward compatibility' approach tries to ensure that AccECN implementations that are being built today will be ready to interwork with a future extension. 
> * In contrast, if your hypothetical new AccECN codepoint did need to be deployed later, it would need to include a fall-back to the current AccECN option when one peer doesn't understand the new codepoint.
> 
>  
> 
> [ms] This is a reasonable point, albeit not a particularly strong one, given that the option is not really required for basic interoperability. Also, a hypothetical follow-up standard could include a way to explicitly request the option…
> 
> 
> * Fall-back would need to avoid a second round of handshake (given this is for low latency), which would only work reliably with an option on the SYN (because the 3rd ACK is unreliable)
> 
>  
> 
> [ms] I can’t follow. If an endpoint is unhappy with getting AccECN option “A” from a peer, I could send a “please send me the better AccECN option B” after the 3-WHS. It could be an upgrade, not a fallback. Also, at this point we don’t know what option B would be and in which context it would be needed. Maybe other protocol features would require negotiation, too. IMHO it is a solvable problem.
> 
> 
> * AccECN has so far avoided using any SYN option space, which is important given that is the scarcest resource.
> 
>  
> 
> [ms] Yep, but as long as no proposal needs SYN options, this is nothing that would make one solution better than another.
> 
> These days, extending TCP is like playing chess - we have to think multiple moves ahead!
> 
>  
> 
> [ms] There is one fundamental difference between chess and TCP: In chess you precisely know the current situation and can predict all potential moves in advance (albeit there are a lot). However, we don’t know the future evolution of TCP, ECN and the Internet as a whole, nor future requirements. If we try to engineer a TCP extension for some potential future, we may figure out later that reality is different from what we designed for. As much as I appreciate your attempts to predict the future, it would be a big surprise to me if you could correctly predict what TCP will be in 10 years from now. So, instead of trying to write specifications that take into account future requirements, which are in fact just unknown, it is more important for me to write effective, efficient and secure specifications for what is currently known for sure.
> 
>  
> 
> [ms] I *don’t* know for sure whether the current AccECN option will be needed, but I see quite some evidence that support of the AccECN option(s) is not a top priority these days. That is also something to think about when designing the details.
> 
>  
> 
> I need to learn to articulate onto the list all my reasons for rejecting certain parts of the design space. Sorry about that.
> 
> Yes, I admit, the current proposal is at the cost of burning two TCP option codepoints. However, you said yourself when you proposed this approach, that they are not (currently) a scarce resource. Certainly not as scarce as option space. I just checked: roughly 15% of the option codepoint allocation space has been used so far. So that's where I would suggest we compromise first.
> 
> [ms] I made a similar calculation some years ago and came to a similar conclusion. Investing two codepoints for AccECN is something we can probably afford. But nonetheless we have to reason carefully why it is really worth the effort.
> 
> 
>  
> 
> Regarding the resulting codepoint consumption (two option kinds) and the addition of a flag byte, this approach would basically end up with the same result as the current proposal in -13 plus some hypothetical future addition leveraging the length field beyond the currently known use.
> 
>  
> 
> And, yes, the length encoding variant may be less flexible and/or consume some more bits in some cases, but on the plus side it only needs one codepoint now – and given that it is entirely unclear whether any AccECN option will indeed be widely used in future, this is a big plus. The fact that implementers are quite silent on the option design is not a good sign.
> 
> 
> [BB] Here's another possibility then. 
> 
> a) [K|L| EE1B | ECEB | EE0B ]
> b) [K|L| EE1B | ECEB | EE0B |F]    if F = 0bXXXXXXX0
> c) [K|L| EE0B | ECEB | EE1B |F]    if F = 0bXXXXXXX1
> 
> We'd have to decide which order alternative (a) takes. I've picked EE1B first only 'cos you're right that there is not currently as much pressure (on the public Internet) for AccECN with ECT(0).
> 
> ==Legend==
> K:    Kind (1B)
> L:    Length (1B)
> ExxB: Echo xx byte counter
> F:    Flags (1B)
> 
> Alternatives a) and b) are equivalent, at least while the only allocated flag is for field order.
> However, alternative b) is available in case other flags are allocated in future
> The flags byte can only be omitted if (L % 3 == 2).
> The flags byte is considered present if (L % 3) == 0.
> 
> Forward compatibility: Options with (L % 3 == 1) MUST be assumed to include a flags byte, and current implementations ignore the last byte.
> 
> The flags byte is optional to implement (even if the AccECN option is implemented)
> If the server includes a flags byte on the SYN/ACK but the client does not include one on the 3rd ACK or the first data packet, the server assumes the client does not implement the flags byte and uses only alternative a) for the remainder of the connection.
> 
> 
> [ms] I am not sure if we need that, but I take this as existence proof that other encodings are possible. Personally, I think that other solutions would be doable if you took one bit from one or multiple counters for format encoding. At least for some of the byte counters I don’t think taking one bit for other purposes would result in an Internet congestion collapse… But, I have already said that the -13 proposal works for me, so I’ll not try to come up with another variant myself.
> 
>  
> 
> To me, one potential difference between two proposals would be incremental deployment. The proposal in -13 only has an advantage if middleboxes such as firewalls will indeed pass TCP options with a format that contains content beyond the (first) Accurate ECN standard (i.e., currently unused length values). IMHO it is too early to know whether firewalls would indeed allow this in future.
> 
>  
> 
> From a security perspective, it is not clear to me whether allowing arbitrary unspecified bytes in a TCP option is a good idea *at all*. It will be interesting to hear the opinion from SEC area on that. Personally, I am not convinced that this really makes sense, but I my concerns are not strong enough to formally push back. I’ll leave it to others to think about whether this is a bug or a feature.
> 
> 
> [BB] Well, let's first try to deal with security ourselves:
> * Octets that are explicitly declared as part of an option cannot be used for a buffer overflow attack. I don't really need to, but I could add the following text to the forward compatibility wording: 
>     A receiver considers octets beyond those it uses, but within the specified length, as if they are padding. 
> * And such octets cannot be any different from the current ability of a sender to add padding. So there's no new attack possible here.
> * And there's no need to specify a max length for any AccECN Option, which would just unnecessarily limit flexibility.
> 
> [ms] I have more thought about the covert channel that some unspecified bytes in a standards track TCP option could offer… In particular if the AccECN option gets widely deployed and used, would some malicious users find those bytes useful? For what? For instance, what in the AccECN protocol design prevents use of these “padding” bytes as super-cookie? To me, it is much more important to think about abuse of whatever we standardize *now*, instead of engineering for some entirely unknown future. I am not a security expert, but I suggest a careful analysis of any security implications of the AccECN option. We seldomly specify new TCP options and malicious users will certainly look for “opportunities” in the spec. So far, I don’t know whether that whole idea of forward compatibility is a feature or a bug.
> 
>  
> 
> Michael
> 
>  
> 
> Maybe one lesson learnt is that the document could have a non-normative appendix that explains the rationale for the finally picked TCP option encoding. That may also help if there are further questions whether two codepoints are really required, e.g. by the IESG (if two codepoints are still the design after WGLC). At least for past TCP option codepoint allocations I recall some discussions late in the IETF process. In those past cases, good arguments in an appendix and running code has helped a lot.
> 
> 
> [BB] I can do that. Appx B is already similar, giving Design Rationale for the ACE field.
> 
> 
> Bob
> 
> 
> 
>  
> 
> Michael
> 
>  
> 
>  
> 
>  
> 
> From: Bob Briscoe <[e-mail]> 
> Sent: Tuesday, November 17, 2020 6:10 AM
> To: Scharf, Michael <[e-mail]>; Yoshifumi Nishida <[e-mail]>
> Cc: Michael Tuexen <[e-mail]>; [e-mail]; Mirja Kuehlewind <[e-mail]>; Scheffenegger, Richard <[e-mail]>; tcpm IETF list <[e-mail]>
> Subject: Re: AccECN field order
> 
>  
> 
> Michael,
> 
> On 16/11/2020 17:36, Scharf, Michael wrote:
> 
> Bob,
> 
>  
> 
> One proposal using the length field with *one option codepoint only* is detailed in: https://mailarchive.ietf.org/arch/msg/tcpm/zo-1OR0nRfhHocX8yvTvpC4BNMo/
> 
>  
> 
> It is the third option mentioned in this e-mail. One example would be to use option length values 5/8/11 for one encoding type and option length values 6/9/12 for the other encoding type (i.e., order of fields). Or one could use some other combination of length values – the only requirement is that a certain value for the option length is only used by one of the option formats. In that approach, the value of the length field would thus directly describe the encoding of the option. Unless I miss something, this would work and it would just require one option codepoint.
> 
>  
> 
> Thus, alternatives to two option codepoints exist and I have explained them on the list in March 2020.
> 
> 
> OK, sorry, yes, I remember this now.
> 
> As I will explain in the AccECN status update talk today in virtual Bangkok, the draft has made provision for different length values than 5/8/11. It says existing implementations MUST accept length values other than those currently defined. But then read in as many whole 3-byte fields as they can.
> 
> This can be used to add a flags byte on the end in future, for extensibility. Or any other form of extensibility the WG might decide in the future.
> 
> I know a flags byte at the end seems odd compared to at the beginning. But (if decided it's needed in future) it's reasonably easy to implement by reading the whole option, then processing the last byte, before reading the rest of the option.
> 
> I believe you will agree that this is a better way to utilize different lengths.
> 
> And thank you for repeatedly emphasizing that you're happy with the 2-kind scheme, or other alternatives.
> 
> 
> Bob
> 
> 
> 
> 
>  
> 
> Anyway, I don’t really care how the options are encoded as long as the receiver doesn’t need per-connection state for decoding a TCP option. So, personally, I would be fine with using e.g. the length field as described in my old e-mail. Or an additional flag byte. And one could come up with further encodings, e.g., by using one or a few bits as a short “type” field for each counter. This is all about protocol engineering. And all these variants have their pros and cons.
> 
>  
> 
> I am also fine with using two option codepoints as specified in -13; this is probably the approach that consumes the least number of bits.
> 
>  
> 
> Michael (w/o any hat)
> 
>  
> 
>  
> 
>  
> 
> From: Bob Briscoe <[e-mail]> 
> Sent: Monday, November 16, 2020 5:52 PM
> To: Yoshifumi Nishida <[e-mail]>
> Cc: Scharf, Michael <[e-mail]>; Michael Tuexen <[e-mail]>; [e-mail]; Mirja Kuehlewind <[e-mail]>; Scheffenegger, Richard <[e-mail]>; tcpm IETF list <[e-mail]>
> Subject: AccECN field order
> 
>  
> 
> Yoshi, (adding the tcpm list in cc)
> 
> On 05/11/2020 06:58, Yoshifumi Nishida wrote:
> 
> Hi Bob,
> 
>  
> 
> On Wed, Nov 4, 2020 at 3:29 PM Bob Briscoe <[e-mail]> wrote:
> 
> Yoshi,
> 
> On 04/11/2020 06:51, Yoshifumi Nishida wrote:
> 
> Hi, folks,
> 
> 
> 
> 
> 
> 
> In my understanding, I'm not sure if we settled down on using two option kinds or encoding schemes for 24bits fields in acc ecn draft.
> 
> So, I think there're still something to be clarified and hope things will be settled at the meeting.
> 
> 
> [BB] I know a WG can change it's mind at any time. But I'd rather we just clarified what a previous decision was, to avoid the need to keep re-opening discussion on a question that have been decided then changed three different ways already.
> 
> My memory is not so good these days. I trusted that Michael S remembered the decision correctly, and I seem to remember that decision being made.
> I've just checked the minutes of the last interim:
>     https://datatracker.ietf.org/doc/minutes-interim-2020-tcpm-01-202004291600/
> and they mention Michael's proposal to use two kinds, but don't record any decision.
> The jabber log gives no clues about any decision.
> 
> I can't find an audio or video recording. Can you point me at one?
> 
>  
> 
> I thought that it's because there was no clear decision at the meeting. 
> 
> But, you can check https://www.youtube.com/watch?v=KsB0_Nb8-kA
> 
> Please let us know if you have any questions or opinions with regard to this.
> 
> 
> [BB] I checked the Youtube link you sent below.
> 
> First I think we're agreed no-one was fighting for us to keep the previous way we did this (using the initial value of the field to set the order for the connection). 
> In my presentation I said there was strong resistance from Michael to do it a different way. 
> (also, offlist, the co-authors including me also didn't like this so much. And Ilpo said it made the implementation complex.)
> 
> Then came the question of what we do instead. There were three alternative proposals:
> a) use 2 option kinds
> b) add a flags byte
> c) somehow use the length field maybe
> 
> Michael raised (c) in the meeting as a possibility, but no-one could think how to distinguish two options of the same length but a different field order using the length field. Michael said he'd post any ideas to the list if he could think of any, but that didn't happen.
> 
> So we're left choosing between (a) and (b).
> I said in the meeting (and on the list when discussing with Ilpo) that I'd be happy to go with (b), but only if there was another use for a flag. Because it would consume 1B more options space in many packets, which is a scarce resource.
> 
> Ilpo had a proposed use for another flag (to help synch counters after a loss), but I think the discussion about it ended that it wouldn't be helpful, 'cos the way it worked depended on itself (circular logic).
> 
> In conclusion, I don't think there was an explicit decision to go with 2 option kinds, but it ended up as the 'last person standing'.
> I like it. It's simple. And apparently option kinds are not such a scarce resource.
> 
> Perhaps we can ratify this in the WG tomorrow.
> 
> 
> Bob
> 
> 
> 
> 
> 
>  
> 
> Thanks,
> 
> --
> 
> Yoshi
> 
> 
> 
> 
> 
> 
> -- 
> ________________________________________________________________
> Bob Briscoe                               http://bobbriscoe.net/
> 
> 
> 
> 
> -- 
> ________________________________________________________________
> Bob Briscoe                               http://bobbriscoe.net/
> 
> 
> 
> -- 
> ________________________________________________________________
> Bob Briscoe                               http://bobbriscoe.net/
> _______________________________________________
> tcpm mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/tcpm