Re: [tsvwg] [Technical Errata Reported] RFC7141 (7237)

[Bob Briscoe <[e-mail]>]

Sebastian,

I've just read your erratum, which is about §2.2 & §2.3:
https://www.rfc-editor.org/errata/rfc7141
and I just read this thread, which is more about §2.4 (sorry I missed 
these at the time).
See [BB]



On 04/11/2022 13:37, Sebastian Moeller wrote:
> Hi Gorry,
>
>
>> On Nov 4, 2022, at 14:03, Gorry Fairhurst<[e-mail]>  wrote:
>>
>> On 04/11/2022 12:42, Sebastian Moeller wrote:
>>> Hi Gorry,
>>>
>>>
>>>> On Nov 4, 2022, at 11:56, Gorry Fairhurst<[e-mail]>  wrote:
>>>>
>>>> On 04/11/2022 10:43, Sebastian Moeller wrote:
>>>>> Hi Gorry,
>>>>>
>>>>> See [SM] below.
>>>>>
>>>>> On 4 November 2022 11:20:56 CET, Gorry Fairhurst<[e-mail]>  wrote:
>>>>>> Commenting as an individual on the Errata filing:
>>>>>>
>>>>>> On 04/11/2022 09:40, RFC Errata System wrote:
>>>>>>> The following errata report has been submitted for RFC7141,
>>>>>>> "Byte and Packet Congestion Notification".
>>>>>>>
>>>>>>> --------------------------------------
>>>>>>> You may review the report below and at:
>>>>>>> https://www.rfc-editor.org/errata/eid7237
>>>>>>>
>>>>>>> --------------------------------------
>>>>>>> Type: Technical
>>>>>>> Reported by: Sebastian Moeller<[e-mail]>
>>>>>>>
>>>>>>> Section: 2
>>>>>>>
>>>>>>> Original Text
>>>>>>> -------------
>>>>>>> 2.2.  Recommendation on Encoding Congestion Notification
>>>>>>>
>>>>>>>      When encoding congestion notification (e.g., by drop, ECN, or PCN),
>>>>>>>      the probability that network equipment drops or marks a particular
>>>>>>>      packet to notify congestion SHOULD NOT depend on the size of the
>>>>>>>      packet in question.
>>>>>>> [...]
>>>>>>> 2.3.  Recommendation on Responding to Congestion
>>>>>>>
>>>>>>>      When a transport detects that a packet has been lost or congestion
>>>>>>>      marked, it SHOULD consider the strength of the congestion indication
>>>>>>>      as proportionate to the size in octets (bytes) of the missing or
>>>>>>>      marked packet.
>>>>>>>
>>>>>>>      In other words, when a packet indicates congestion (by being lost or
>>>>>>>      marked), it can be considered conceptually as if there is a
>>>>>>>      congestion indication on every octet of the packet, not just one
>>>>>>>      indication per packet.
>>>>>>>
>>>>>>>      To be clear, the above recommendation solely describes how a
>>>>>>>      transport should interpret the meaning of a congestion indication, as
>>>>>>>      a long term goal.  It makes no recommendation on whether a transport
>>>>>>>      should act differently based on this interpretation.  It merely aids
>>>>>>>      interoperability between transports, if they choose to make their
>>>>>>>      actions depend on the strength of congestion indications.
>>>>>>>
>>>>>>> Corrected Text
>>>>>>> --------------
>>>>>>> I am not sure the text is actually salvageable, as it appears ti be a logic disconnect at the core of the recommendations.
>>>>>>>
>>>>>>> Notes
>>>>>>> -----
>>>>>>> The recommendations seem not self consistent:
>>>>>>> A) Section 2.2.  recommends that CE marking should be made independent of packet size, so a CE-mark carries no information about packet size.
>>>>>> I did not understood that it needed to.

[BB]
1/ I've emphasized the words in your erratum that help me see where a 
first misunderstanding might be:
> A) Section 2.2. recommends that CE marking should be made independent 
> of packet size, *so a CE-mark carries no information about packet size*.
> B) Section 2.3 then recommends to use the size of marked packets as 
> direct indicators of congestion strength.

An analogy might help explain: When a tail-drop queue discards a packet, 
or when a flow-agnostic AQM (like PIE or RED) ECN-marks a packet, they 
do so independent of flow ID. Nonetheless, because there is a flow-ID in 
each packet, these arbitrary/random algorithms blindly pick the flows 
that should respond, even though these algorithms don't read the flow-ID 
(it can even be encrypted).

Returning to packet-size, most AQMs don't mark dependent on packet size. 
But packets do have a size (and a size field). And larger packets do 
contribute more per packet to link congestion than smaller packets. So, 
if we assume that the marking algorithm has not already taken packet 
size into account, the strength of a congestion signal for the purpose 
of congestion response can be taken to depend on the size of the packet 
it is applied to.

And that can be done, even if the people who originally developed the 
AQM algorithm thought that packet-size dependence ought to be done by 
the AQM (but it wasn't enabled).

Next step: If this does capture the your misunderstanding on /this/ 
point, pls say, then either you or I can suggest an edit to avoid the 
misunderstanding.

2/ A second problem might be that RFC7141 doesn't actually outline the 
original packet-size problem that byte-mode RED was trying to solve. I 
detect you're not familiar with that history, and why should you be? 
That /is/ something that could be corrected with an erratum.


3/ Then this thread shows further misunderstanding regarding §2.4 on 
splitting and merging - see [BB] below...

>>>>>> This RFC I think was intended to be independent of the transport.  I see the transport sender as responsible for determining the packetisation of the transport segments, and the (S)ACKs can often identify segments, hence the sender can determine the segments that have been acknoweldged or times when ECN marking was seen.
>>>>> [SM] This assumes that relevant segment size does not change along the path. Which generally is not true. Just think fragmentation, if the sender sends a packet that gets fragmented along the path and only a single fragment gets CE marked the sender will see this as the whole packet being marked. Or from the other side of the issue, if say a Linux router uses GRO/GSO and queues a larger meta packet and CE marks that, receiver and sender at best see a sequence of CE marked packets. So the recommendation would need to be changed to calculate the consecutive sequence of CE marked octets and take these as correlate for congestion strength. So no, the sender really has no reliable knowledge about the size of the data unit the marking node marked.
>>>>>
>>>> I suggest IETF transports treat all IP fragments as one unit of retransmission/congestion at the transport layer.
>>> 	[SM2] But what if the re-segmentation does not happen at the receiver, but say a fragmenting and CE-marking path tries to act transparently. According to the rules both in RFC3168 and RFC7141 a re-segmented packet containing even a single CE-marked fragment is to be CE-marked (or dropped). So the AQM might have marked a 576 octet segment but all the endpoint sees is a marked ~1460 octet segment.
>>> 	This also illustrates how section 2.4 of RFC7141 proposes a method that does not achieve its aim, of giving veridical "number of market octets" information. It simply is impossible to do so generally (often it will work, but the endpoints can not even know when it was correct and when not).
>>> Section 2.4 has more issues BTW, it tries to give recommendation how to deal with splitting and merging but fails to achieve its goals of giving a veridical account of the marked octets:
>>>
>>> Let's see what happens when applying the proposed counter method in regards to number of marked octets under the conditions this section addresses
>>> Here let's look at a toy problem with 20 byte headers and a total payload of 1200 octets that is split in or merged out of 3 fragments/segments with 400 octets payload each
>>>
>>> Merging multiple segments pre-marking:
>>> (20+400) + (20+400)-20 + (20+400)-20 -> 1220 total 1200 payload + CE
>>> -> AQM marks 1220 or 1200 octets
>>> (12+1200)+CE
>>> receiver sees 1200 octets with CE and ACKs these with ECE
>>> sender can assume 1200 octets where marked
>>> CORRECT
>>>
>>> Merging multiple segments post-marking ():
>>> -> AQM marks segment 2 of 420 or 400 octets
>>> (20+400) + (20+400+CE) + (20+400)
>>> (20+400)+(20+400+CE)-20+(20+400)-20 = 1220 total 1200 payload + CE
>>> receiver sees 1200 octets with CE and ACKs these with ECE
>>> sender must assume 1200 octets where marked
>>> FALSE
>>>
>>> Fragmenting a segment pre-marking
>>> 1220 -> (20+400) + (20+400) + (20+400)
>>> -> AQM marks segment 2 of 420 or 400 octets
>>> (20+400) + (20+400+CE) + (20+400)
>>> Resegmentation happens before protocol sees marking
>>> (20+400) + (20+400+CE)-20 + (20+400)-20 -> 1220 total 1200 payload + CE
>>> receiver sees 1200 octets with CE and ACKs these with ECE
>>> sender must assume 1200 octets where marked
>>> FALSE
>>>
>>> Fragmenting a segment post-marking
>>> (20+1200)
>>> -> AQM marks 1220 or 1200 octets
>>> (12+1200)+CE
>>> fragmentation happens:
>>> (20+400+CE) + (20+400+CE) + (20+400+CE)
>>> Resegmentation happens before protocol sees marking
>>> (20+400=CE) + (20+400+CE)-20 + (20+400+C)-20 -> 1220 total 1200 payload + CE
>>> receiver sees 1200 octets with CE and ACKs these with ECE
>>> sender must assume 1200 octets where marked
>>> CORRECT
>>>
>>> So only in two out of four conditions does the proposed method actually achieves its goal.

[BB] You seem to have interpreted §2.4 as applying to just each frame in 
isolation, which would make absolutely no sense at all. It applies to a 
stream of frames that are being split or merged. Indeed, the text 
already makes this clear, as follows:

    even the smallest
    positive remainder in the conceptual counter should trigger the next
    outgoing packet to be marked (causing the counter to go negative)


I detect that you prefer to be fed every little detail, so here's some 
pseudocode that might help.
https://bobbriscoe.net/projects/netsvc_i-f/consig/encap/ecn-encap-reframing_goal2_pseudo.c
It's written in terms of frame decapsulation, not fragment reassembly, 
but you can see it works on the principle of ignoring the sizes of any 
headers that are not preserved during the process.

Applying the pseudocode to your examples:
Ignoring the timeout block for a moment, you should be able to see that, 
in the two cases you've tagged 'FALSE', _diff will become negative 
(-800) after the reassembled packet in either of your 'FALSE' examples 
is marked. Then if more 400-byte-payload fragments arrived, it would 
take three marked ones to make _diff positive again, causing the packet 
reassembled from last to be marked before forwarding.

This carrying over of the balance only continues as long as the time 
between incoming marks is less than CE_TIMEOUT. If not, you can see that 
the counters reset to become equal. The value of CE_TIMEOUT is just for 
illustration.

The use of WRAP_GUARD is just a cheap emulation of modulo arithmetic 
that allows either counter to wrap to zero before the other and still 
work regardless.

>>>
>>> Now add the complication that the RFC fails to mention what it considers marked octets, just the payload or payload+headers.

[BB]  This RFC is not specifying a protocol, it's giving a statement of 
principle as a goal of future protocol design. Any protocol designer (or 
professional implementer) would be able to fill in details like which 
headers to include or ignore; or how to confine each instance of the 
algorithm to a stream of packets that had passed through the same AQM 
with the same type of ECN codepoint.

Next steps:  Everyone else who discussed this section grocked the idea 
immediately. So I don't think more explanation is needed to understand 
§2.4. Certainly pseudocode would seem overkill, given splitting and 
merging is a fairly minor part of this RFC. But I'll leave that decision 
for whoever has to decide on errata - the AD I think?


>>> This is important as the sum of payload + headers of X fragments is larger than the sum payload + header of the single packet re-constituted out of these fragments. So the de-fragmenting process arguably needs to only look at payload size, but RFC7141 section 2.4 does not make that explicit.
>>> If an implementation actually uses the full size instead of the payload size now the last condition also gets it wrong:
>>>
>>> Fragmenting a segment post-marking
>>> (20+1200)
>>> -> AQM marks 1220 or 1200 octets
>>> (12+1200)+CE
>>> fragmentation happens:
>>> (20+400+CE) + (20+400+CE) + (20+400+CE)
>>> Resegmentation happens before protocol sees marking
>>> (20+400+CE) + (20+400+CE) + (20+400+C) -> 1220 total 1200 payload + CE
>>> but (20+400)*3 = 1260 marked octets
>>> receiver sees 1200 octets with CE and ACKs these with ECE
>>> sender must assume 1200 octets where marked
>>> CORRECT
>>> But now the left over 40 bytes in the marked-octet budget will result in CE feedback for the next (re-assembled) packet.
>>> FALSE
>>>
>>> For rfc3168 that will not matter much as ECE is sustained until CWR is received anyway, but L4S style signaling now acquired an erroneous CE mark.
>> Network fragmentation be it in tunnels, extension headers or IPv4 fragments is indeed thwarted with all manner of issues. Nothing new - the IETF has long recommended the unit of loss/marking to be the same as the end to end PDU. PMTU is tricky, but does have benfits:-)
> 	[SM3] Not a fan of fragmentation either, but I assume that fragmentation will stay a fact of life over the internet independent of my opinion. My point here is if the IETF proposes a method that aims to correctly account for CE-marked octets, that method should actually deliver on its premise. Failing in 2-3 out of 4 conditions the method is designed to handle is IMHO a sign that the proposed method is/was not in proper shape for becoming an official recommendation.
> This is fine in an informational RFC as documenting subjective opinion, but problematic in a standards or BCP type document, if like in this case we feel that BCP methods (even incomplete or impossible to achieve ones) are binding precedence that need to be respected in later RFC drafts.

[BB] Instead of implying that everyone involved has been acting 
unprofessionally, or incompetently, it is good practice to word emails 
in such a way that allows for the possibility that you just haven't 
understood something.

>
>>>
>>>> GSO/GRO and variants would/could change the fragmentation, that is true and need to be considered.
>>> 	[SM2] I am confused? How do GRO/GSO affect fragmentation, IMHO these two will cause larger aggregates that exist only locally (Linux will segment meta-packets in the sending process and will not sent out say a large 64K TCP packet in fragments, but will re-segment the meta-packet into a neat sequence of complete self-sustained TCP packets)? IMHO they affect primarily the unit size the AQM might CE-mark on, in a way that is in-transparent to the end points. My point is the unit size an AQM acts on is generally unknowable precisely be the end-points. At which point making the end-points pretend that congestion strength somehow correlates with size of marked packets really stops making sense.
>>>
>> The segment delivered can be a different size to the unit of transmission. This is an implementation optimisation - if this done without regard to the marking, then the results will be different and likely do not deliver what is expected - optimisations need to understand what they optimise.
> 	[SM3] Yes, we seem to agree that it is impossible for endpoints to veridically measure the amount of octets actually CE-marked for a number of reasons. IMHO from this observation it follows directly that basing end-point decisions on number of marked octets is not going to generally work, as that number is not robustly and reliably available at the end-point. That is end-points do see a number but that number is not guaranteed to actually match what happened at the marking entity, and hence this number can not be a correlate of congestion strength. Interpreting that number never the less as indicator of congestion strength seems hence sub-optimal and not something to unconditionally recommend.

[BB] With a better understanding of preserving marking probability when 
re-framing, I hope you can now see that it would be possible for 
endpoints to respond based on packet size of congestion indications 
(even if any re-framing includes approximations).

>
>>>>>>> B) Section 2.3 then recommends to use the size of marked packets as direct indicators of congestion strength.
>>>>>>>
>>>>>>> C) Section 2.3 then later clarifies that transports should interpret the size of CE-marked packets as correlate for congestion strength but are in no way required to take this interpretation into account when acting based on the congestion signal.
>>>>>>>
>>>>>>>
>>>>>>> This has several problems:
>>>>>>> 1) A) and B) are in direct contradiction to each other. If we ask marking nodes to ignore packet size while marking, but end nodes to take it into account we basically create random congestion strength "information" by the pure chance of a specific packet of a specific size "catching" a CE mark. At which point we might as well simply draw a random number at the end-point to interpret congestion strength (except that packet sizes are not distributed randomly).
>>>>>>>
>>>>>>> 2) Asking endpoints to interpret CE_marks in this way but not act on it, is hardly actionable advice for potential implementers. If we can not recommend a specific way, we should refrain from offering recommendations at all to keep things as simple as reasonably possible.
>>>>>> This doesn't appear to be textual errata, it seems more like the request is for more clarification or motivating an alternative?

[BB] I hope I have explained for Sebastian how A) & B) being different 
is a way of exploiting random selection, not a contradiction.
Re #2, the draft doesn't say "don't act on it"; it says it's not 
mandatory, which is very different (and necessary when protocols already 
exist that do not comply).


Bob

>>>>> [SM] What alternatives to changing incorrect text do exist? I do not think changing the status to historic is a realistic option in spite of the text recommending the impossible.
>>>>>
>>>> Put simply:
>>>>
>>>> An Erratum would normally specify either:
>>>>
>>>>      a direct change of text to fix a mistake in production, but a change of the spec from the original intended method;
>>>>
>>>>      or specify something to inform a future revision.
>>>>
>>>> An update in a new RFC is needed to change the method, or a process request to mark an RFC as historic.
>>> 	[SM2] Would it also be possible to request to re-classify as informative? This RFC with its impossible recommendations is causing issues with other RFCs and I think it would help if this could be ameliorated by moving away from BCP status.
>>>
>> If there is consensus an RFC shouldn't be associated with a BCP, we can examine what to do. The first thing is to write a (short) ID and see if you can gain sufficient attention from the WG to enable this to be discussed.
> 	[SM3] Thank you very much. Is there an example for a similar process I could look at and take inspiration from?
>
> Regards
> 	Sebastian
>
>
>> Gorry
>>
>>>> Gorry
>>>>
>>>>>>> Instructions:
>>>>>>> -------------
>>>>>>> This erratum is currently posted as "Reported". If necessary, please
>>>>>>> use "Reply All" to discuss whether it should be verified or
>>>>>>> rejected. When a decision is reached, the verifying party
>>>>>>> can log in to change the status and edit the report, if necessary.
>>>>>>>
>>>>>>> --------------------------------------
>>>>>>> RFC7141 (draft-ietf-tsvwg-byte-pkt-congest-12)
>>>>>>> --------------------------------------
>>>>>>> Title               : Byte and Packet Congestion Notification
>>>>>>> Publication Date    : February 2014
>>>>>>> Author(s)           : B. Briscoe, J. Manner
>>>>>>> Category            : BEST CURRENT PRACTICE
>>>>>>> Source              : Transport Area Working Group
>>>>>>> Area                : Transport
>>>>>>> Stream              : IETF
>>>>>>> Verifying Party     : IESG

-- 
________________________________________________________________
Bob Briscoehttp://bobbriscoe.net/