Re: [v6ops] More IPv6 fragmentation in draft-ietf-intarea-tunnels

[Joseph Touch <[e-mail]>]

Hi, Eduard,

> On Mar 22, 2021, at 9:26 AM, Vasilenko Eduard <[e-mail]> wrote:
> 
> Hi Joseph,
> We are in sync now. I do understand you. It looks like you understand me.
>  
> Let me repeat what draft-ietf-intarea-tunnels proposes in super-short:
> -          2 MTUs introduced instead of one for the virtual tunnel interface

Not introduced. Just explained. They already existed.

> -          Lowest one is managed by PMTUD as usual (that you claim is dead anyway, because ICMP filtered)

Yes.

> -          Highest one is static because nobody knows how to correct it in a more automatic way (maybe in the future or maybe not …)

Yes.

> -          Everything in between these 2 MTUs are fragmented – it would be all attempts from hosts to send bulk traffic (ftp, etc)

Yes.

>  
> You propose to completely finish PMTUD because:
> -          PMTUD is already basically gone from the Internet due to ICMP blocking

Yes.

> -          It is so nice to have 2 MTUs per 1 virtual interface!

No - those already exist. I did not invent them; I just gave them names so we can discuss them clearly.

> Sorry. I do not agree with both:
> -          Conscious efforts to scrape IPv6 PMTUD completely is the big architecture decision that is better to discuss openly

You’re welcome to do that, but that’s the reason PLPMTUD exists. Again, that’s not my battle and has no relation to draft-tunnels.

Nobody is scraping PMTUD; we’re just all recognizing that it largely doesn’t do anything. Relying on it results in black holes (again, see the PLPMTUD RFC).

> -          I have read draft-ietf-intarea-tunnels very carefully – I was not capable to understand why 2 MTUs per virtual interface is better

It isn’t better. It just IS.

Every path in the Internet has two packet sizes:
	- the size that can traverse the hops
	- the size that can be reassembled at the receiver

For tunnels, we need names for the path experienced by the user of the tunnel AND for packets inside the tunnel. Those need to be different names.

I just gave them different names. They always existed.

E.g., Tunnel MTU - what does that mean to you?
	- to some, it means the endpoint source MTU that goes over a tunnel without tunnel ingress fragmentation
	- to others, it means the endpoint source MTU that a tunnel egress can reassemble
	- to others, it means the MTU of the path between ingress and egress of encapsulated packets
	- to others, it means the MTU that the egress can reassemble of encapsulated packets

That’s the only reason we introduced new terms *for existing issues*.

> Reminder: all current tunneling technologies have just one MTU for the virtual interface that is automatically corrected by PMTUD.

They use at least two, sometimes all four of the ones I note above.

> The majority needs 2 packets lost before PTB would reach the host.
> RFC 2473 is capable of proxy the 1st PTB message to ask the host to decrease MTU immediately.

Strictly following RFC2473 means there can never be IPv6-in-IPv6 because 1280 would be PTB after encapsulation, which means the response to the source would be “send only up to 1240”. But that then means the source is told there is an IPv6 path that cannot support 1280, which means it would need to shut down that path.

Is that your intended outcome?
 
> PS: [e-mail] <mailto:[e-mail]> is a closed community – my messages are refused – this group demands permanent subscription. I could not just send the message there for a small discussion. All or nothing.

Yes; that is not uncommon for IETF WGs. That doesn’t mean this discussion belongs in this WG. 

Joe

> I hope that your replay would leave some history there.
>  
> Eduard
> From: Joseph Touch [mailto:[e-mail] <mailto:[e-mail]>] 
> Sent: Monday, March 22, 2021 6:23 PM
> To: Vasilenko Eduard <[e-mail] <mailto:[e-mail]>>
> Cc: [e-mail] <mailto:[e-mail]>; int-area <[e-mail] <mailto:[e-mail]>>
> Subject: Re: More IPv6 fragmentation in draft-ietf-intarea-tunnels
>  
> Hi, Eduard,
> 
> 
> On Mar 22, 2021, at 5:11 AM, Vasilenko Eduard <[e-mail] <mailto:[e-mail]>> wrote:
>  
> Hi Joseph,
> Truly speaking – I have not understood many of your comments below. I did try – read a few times. It is my fault. Sorry.
> Therefore, let’s do it slowly step by step.
> -          draft-ietf-intarea-tunnels has introduced 2 MTUs for one virtual tunnel interface:
> o   “Tunnel MTU in draft-ietf-intarea-tunnels“ (managed by somebody else by section 4.2.2, and with deep explanation in section 4.2.3 why it could not be managed effectively) – it is the analog of host EMTU_R buffer for reassembly. The default “Tunnel MTU in draft-ietf-intarea-tunnels” is initialized to EMTU_R (1500B) minus tunnel outer headers overhead
> o   MAP (maximum atomic packet) that other people do know now as “Ordinary tunnel MTU”. MAP should be corrected by PMTUD feedback from inside the tunnel (like it is now for “Ordinary tunnel MTU”)
>  
> Two *different* terms are introduced because “tunnel MTU” is ambiguous - it means both “largest segment traversing a tunnel without tunnel fragmentation” and “largest segment traversing a tunnel with tunnel fragmentation”. 
>  
> If we use the analogy of a link layer, ATM reports a MTU of 9K, not 48bytes - because traversing the link is successful based on reassembly, not traversal without link fragmentation.
>  
> That is why we need two *different* terms and why we should not use “tunnel MTU” for the former.
> 
> 
> o   No feedback is assumed between these 2 MTUs for the one interface – this is the root cause of this problem.
>  
> No feedback mechanism *is currently available*. That’s not to say there can’t be a new one, but it would need to be the tunnel configuration protocol that does this. ICMPs alone are insufficient/
>  
> -          If any additional overhead would happen on the tunnel path (iOAM, SRv6, any EH) then the packet would be dropped with PTB to the tunnel source. Path MTU would be corrected for “Ordinary tunnel MTU” that draft-ietf-intarea-tunnels has renamed to MAP.
>  
> That would imply, e.g., that ATM should send PTBs for packets larger than 48B. That also means that there cannot ever be IPv6 over IPv6 tunnels, because the tunnel would force a PTB to the source of 1240B, not 1280.
>  
> Again, that’s why you cannot force the tunnel MTU upstream.
> 
> 
> -          But MTU from MAP would not be propagated to “Tunnel MTU in draft-ietf-intarea-tunnels”. The real traffic source would never be informed by the tunnel end about Oversized packet. Looking to the logic of other tunneling mechanisms – PTB would never be delivered to the real traffic source.
>  
> That’s correct; we can discuss a way to allow the tunnel to try to help the source upstream, but we need to clamp the PTBs upstream to the MTU of the protocol. Again, the net result for IPv6 over IPv6 would be that PTBs cannot be propagated back to the source. And that’s the *correct* behavior, unless you want to make IPv6 tunnels break IPv6 requirements.
> 
> 
> -          Finally, draft-ietf-intarea-tunnels has no choice except to propose fragmentation in section 4.2.2 for all such cases (algorithm proposes to fragment if MAP is not enough, but “Tunnel MTU in draft-ietf-intarea-tunnels“ is already accepted)
>  
> Agreed that it might be useful to discuss ways to propagate the MAP upstream, but MAP cannot be relayed as PTB *unless* it remains below the max MTU.
>  
> —
>  
> To the broader point, yes - it would be useful to talk about *new* extensions, but none of these are *existing* mechanisms.
> 
> 
> Net result:
> If any other overhead would happen on the tunnel path (that is very probably with SRv6, iOAM, and many other IPv6 EHs in development)
> Then such traffic would be initially dropped (it is normal for the 1st packet – needed to correct MTU on the virtual interface) then it would be fragmented for all other packets,
> Instead of informing the real source to decrease MTU.
> The real source would never understand that this packet size does create fragmentation.
>  
> Conclusion: draft-ietf-intarea-tunnelsdoes breaks PMTUD completely. It is replaced by fragmentation.
>  
> First, PMTUD is already basically gone from the Internet due to ICMP blocking.
>  
> But I agree that draft-tunnels does say that tunnel fragmentation should not drive end-to-end fragmentation. That is a FEATURE, not a bug, e.g., in allowing IPv6 over IPv6.
>  
> I agree we can talk about a NEW mechanism to relay PTB upstream *when it follows protocol requirements*, but we need to figure out how to do this. We ALSO need to relay EMTU_R upstream at the IP layer - but have NO mechanism for that right now.
>  
> Joe
> 
> 
> Eduard
> From: Vasilenko Eduard 
> Sent: Monday, March 22, 2021 2:29 PM
> To: 'Joseph Touch' <[e-mail] <mailto:[e-mail]>>
> Cc: [e-mail] <mailto:[e-mail]>; int-area <[e-mail] <mailto:[e-mail]>>
> Subject: Proxy function for PTB messages on the tunnel end
>  
> Hi Joseph,
> I probably need to tell why I like the initial RFC 2473: it requests PTB “proxy” functionality from tunnel ends. (well, it is not called “proxy” inside RFC 2473 – it just discusses how to re-create PTB from PTB on the tunnel end). This initial architecture decision of IPv6 (to inform the real traffic source) is the basement for PMTU to work. It is better to fix it, not to invent some other patches for MTU discovery.
> Longer explanation:
> If PTB message would be created on the tunnel path – it would easily inform “tunnel end” of Oversized packet – tunnel MTU could be decreased.
> It is already a solution because the next Oversized packet from the source would get PTB response from the tunnel end itself – the source would get PTB after the second oversized packet.
> But it was better to inform the real traffic source after 1st Oversized packet. Hence, PTB proxy on the tunnel end is better.
>  
> I am not ready to discuss all corrections of draft-ietf-intarea-tunnels section 5.2 to RFC 2473 – they do not have relationships to PMTUD.
> Except one that would lead to the massive fragmentation that I would discuss in the next email.
> Eduard
> From: Joseph Touch [mailto:[e-mail] <mailto:[e-mail]>] 
> Sent: Sunday, March 21, 2021 7:23 PM
> To: Vasilenko Eduard <[e-mail] <mailto:[e-mail]>>
> Cc: [e-mail] <mailto:[e-mail]>; int-area <[e-mail] <mailto:[e-mail]>>
> Subject: Re: [v6ops] draft-vasilenko-v6ops-ipv6-oversized-analysis-00
>  
> Hi, all,
>  
> Spoiler alert if you don’t want to read the whole post:
>             - draft-vasilenko makes erroneous claims as to the content in draft-tunnels
>             - draft-tunnels and draft-vasilenko are consistent (once the latter is corrected) in their mutual conclusions
>                         - draft-tunnels on the need for fragmentation over finite MTU paths
>                         - draft-vasilenko in encouraging increases in those finite MTUs
>  
> Joe
>  
> ---
>  
> First, draft-ietf-intarea-tunnels is discussed on the int-area list; after review of the information below, if you still believe there are issues to be addressed in that doc, you should post them there.
>  
> The technical errors in RFC2473 have been indicated in that document since draft-ietf-intarea-tunnels-01, posted in July 2015. They remain accurate, IMO. 
>  
> Note that I ceased performing in-place updates of that document because of *lack of active discussion* and because in-place updates are a waste of my time.
>  
> I am glad to see someone in IPv6 interested now, and would be glad to update my draft as needed.
>  
> FWIW, having read your doc, here are its errors in misstating the content of my draft:
>  
> - your doc mistakenly assumes that mine requires IPv6 hosts to send 1500B packets if they can, even if tunnels are on the path
>             as with any IPv6 path, the source should send fragments no larger than the entire path can transit, whose reassembled size is no larger than the receiver can reassemble
>             those original fragments are what enter the on-path tunnels, so they should be no larger than the tunnel egress can fragment
>             and those original fragments would be encapsulated and then source fragmented by the tunnel according to the same (recursive) policy
>  
> - nothing in draft-tunnels assumes ICMP PTB cannot adjust these sizes or that the tunnel cannot use PLPMTUD
>             see sec 4.3.1 of v10
>  
> - draft-tunnels does not “introduce” a new variable called tunnel MTU; I introduced the terminology, but the concept is as old as tunnels
>             I coined that term to refer to the MTU across the tunnel with reassembly at egress (which already exists), as different from the MTU between ingress and egress (which I call tunnel MAP)
>             sec 4.2.3 of v10 doesn’t claim this value cannot be set; in explains that PMTUD has no role in discovering its value:
>           Note, however, that PMTUD never discovers
>           EMTU_R that is larger than the required minimum; that information is
>           available to some upper layer protocols, such as TCP [RFC1122 <https://tools.ietf.org/html/rfc1122>], but
>           cannot be determined at the IP layer.
>             I never said it cannot be discovered
>                         it should be (e.g., by a tunnel configuration protocol)
>                         note that there are no current protocols that do this, even without tunnels (i.e., discover larger EMTU_R)
>                         I can add that point as clarification
>  
> - draft-tunnels does not increase IPv6 fragmentation
>             please indicate why you believe it would (notably here "a considerable increase in fragmentation is proposed for the reasons of academic purity”)
>  
> - draft-tunnels does not claim fragmentation is the only solution to oversize packets
>             it addresses how and where to handle tunnels in the presence of packet limits, of which path MTU is only one
>  
> - ICMP PTB is not a solution out to the origin source
>             that would potentially drop the IPv6 path MTU below 1280, given enough tunnel overhead (or layers thereof), a violation of IPv6
>             so yes, in that case, the ONLY solution that preserves IPv6 in the presence of tunnels with that much overhead would be ingress source fragmentation
>  
> - sec 3.3 of my doc DOES allow ICMPs to be relayed back to the source
>             it merely states that they should be generated when a packet too large to ingress arrives, 
>             NOT when an internal tunnel ICMP is received by the ingress
>  
>             the point is that the origin source sees the ingress as a router on the path,
>             so it should get ICMPs from that router only when packets arrive at that router, not when its tunnel fails downstream
>  
>             this makes ICMP relay *easier* and more reliable to implement; the ingress gets tunnel ICMPs to learn the tunnel’s effective link MTU,
>             then uses that link MTU to send ICMPs back
>  
>             yes, this is to allow the tunnel to act as the link *that it is*, but it does not prohibit ICMP info from flowing back to the source
>  
> And finally:
>  
> - nobody is claiming we shouldn’t increase link MTU
>             draft-tunnels would still be relevant, no matter how large the MTU is, for the reasons I state in that doc
>  
> One other observation:
>  
> - your statistics for fragment drops apply only when the fragment is visible to the IP layer
>             there are intermediate layers that hide fragmentation for exactly this reason, e.g., UDP tunnels, GRE, etc.
>  
> ---
>  
> 
> On Mar 21, 2021, at 1:59 AM, Vasilenko Eduard <[e-mail] <mailto:[e-mail]>> wrote:
>  
> Dear Experts,
> I have seen many recent activities in IETF related to MTU problems. Well, maybe not so active as some others, but active anyway. Many other active drafts are evaluated in this draft.
> I had an idea what is the right way to solve problems in this area, but after the research, it has been found that foundations were discussed in RFC 2473 (Dec 1998). Just people have forgotten about it.
> We have discussed it with co-authors and we have decided that it make sense to publish the research because it looks at the problem in a systematic approach.
> 
> The one thing that is alarming in this research: draft-ietf-intarea-tunnels is pushing for much more fragmentation for pure Academic reasons. This draft is already referenced by many other documents.
> I believe that not many people have spent enough time to understand it's complexity to reveal the truth: the majority of the IPv6 traffic would be fragmented if it would follow draft-ietf-intarea-tunnels.
> 
> Thanks to everybody who would spend enough time to produce comments.
> Eduard
> -----Original Message-----
> From: [e-mail] <mailto:[e-mail]> [mailto:[e-mail] <mailto:[e-mail]>] 
> Sent: Friday, March 19, 2021 11:07 PM
> To: Dmitriy Khaustov <[e-mail] <mailto:[e-mail]>>; Vasilenko Eduard <[e-mail] <mailto:[e-mail]>>; Vasilenko Eduard <[e-mail] <mailto:[e-mail]>>; Xipengxiao <[e-mail] <mailto:[e-mail]>>; Xipengxiao <[e-mail] <mailto:[e-mail]>>
> Subject: New Version Notification for draft-vasilenko-v6ops-ipv6-oversized-analysis-00.txt
> 
> 
> A new version of I-D, draft-vasilenko-v6ops-ipv6-oversized-analysis-00.txt
> has been successfully submitted by Eduard Vasilenko and posted to the IETF repository.
> 
> Name:             draft-vasilenko-v6ops-ipv6-oversized-analysis
> Revision:         00
> Title:               IPv6 Oversized Packets Analysis
> Document date:          2021-03-19
> Group:             Individual Submission
> Pages:              19
> URL:            https://www.ietf.org/archive/id/draft-vasilenko-v6ops-ipv6-oversized-analysis-00.txt <https://www.ietf.org/archive/id/draft-vasilenko-v6ops-ipv6-oversized-analysis-00.txt>
> Status:         https://datatracker.ietf.org/doc/draft-vasilenko-v6ops-ipv6-oversized-analysis/ <https://datatracker.ietf.org/doc/draft-vasilenko-v6ops-ipv6-oversized-analysis/>
> Htmlized:       https://datatracker.ietf.org/doc/html/draft-vasilenko-v6ops-ipv6-oversized-analysis <https://datatracker.ietf.org/doc/html/draft-vasilenko-v6ops-ipv6-oversized-analysis>
> Htmlized:       https://tools.ietf.org/html/draft-vasilenko-v6ops-ipv6-oversized-analysis-00 <https://tools.ietf.org/html/draft-vasilenko-v6ops-ipv6-oversized-analysis-00>
> 
> 
> Abstract:
>   The IETF has many new initiatives relying on IPv6 Enhanced Headers
>   added in transit: SRv6, SFC, BIERv6, iOAM. Additionally, some recent
>   developments are overlays (SRv6, VxLAN) over IPv6. It could create
>   oversized packets that need to be dealt with. This document analyzes
>   available standards for the resolution of oversized packet drops.
> 
> 
> 
> 
> Please note that it may take a couple of minutes from the time of submission until the htmlized version and diff are available at tools.ietf.org <http://tools.ietf.org/>.
> 
> The IETF Secretariat
> 
> 
> _______________________________________________
> v6ops mailing list
> [e-mail] <mailto:[e-mail]>
> https://www.ietf.org/mailman/listinfo/v6ops <https://www.ietf.org/mailman/listinfo/v6ops>