Re: [Pals] [Int-area] L2TP sequencing: Commonly disabled for IP data? Or always?

[Bob Briscoe <[e-mail]>]

Giles,

On 29/06/2021 12:41, Giles Heron wrote:
> Hi Bob,
>
>> On 28 Jun 2021, at 00:23, Bob Briscoe <[e-mail] 
>> <mailto:[e-mail]>> wrote:
>>
>> Giles, Mark,
>>
>> On 10/06/2021 13:22, Giles Heron wrote:
>>> So AFAIK SP networks don’t generally reorder packets in the steady 
>>> state, but of course reordering can occur under rerouting.
>>
>> [BB] The cases I'm concerned about are where the operator
>> * deliberately reorders packets using a multi-queue scheduler at a 
>> node contrived to act as the bottleneck (the BNG)
>> * AND the node is within an L2TPv2/3 tunnel
>> * AND the tunnel needs sequencing at the egress for some other reason.
>
> If that’s the case then the node almost certainly won’t re-order 
> packets within the tunnel, even if it is deliberately reordering 
> packets between clients.  As far as the node is concerned the L2TPv3 
> tunnel is one flow within one subscriber’s traffic.   BNG reordering 
> is generally between subscribers (e.g. where subscribers get serviced 
> round-robin to ensure fairness), or potentially within classes of 
> service per subscriber (e.g. strict priority for on-net voice and IPTV 
> over Internet traffic).

[BB] Not so. There are certainly cases where packets /are/ reordered 
within a per-customer tunnel. For instance, in the BT wholesale case, 
the BNG uses the BBF framework to schedule a set of QoS queues for each 
customer CVLAN. I know this intimately, because it was what I had been 
asked to simplify when we came up with L4S in the first place. Indeed, I 
was given permission to include a picture and description of it in this 
public report we did for the EU-funded project: 
https://riteproject.files.wordpress.com/2015/12/rite-deliverable-3-3-public1.pdf#12

Also, there are probably many cases of accidental reordering too, e.g. 
due to channel bonding or multipath links that are left out of order 
rather than resequenced, or reroutes, including those at L7, when the 
traffic source moves to a CDN after a flow starts, etc. etc.


Whatever, the question is the converse: where packets are /not/ 
deliberately re-ordered by the network operator today, might these 
operators be sequencing IP data emerging from an L2TP tunnel? If so, 
they would have to disable sequencing if they wanted to introduce 
deliberate reordering (to transition to L4S). But (based partly on the 
insights you've given already) I doubt there is much if any sequencing 
going on of IP data over tunnels.

>
>>
>> In many cases, such a scheduler would be located prior to the tunnel 
>> ingress, so not a concern. I believe the DOCSIS rPHY case below falls 
>> into that category (both downstream and up).
>
> Agreed
>
>>
>> In contrast, where a BNG sits /within/ the span of an L2TP tunnel, I 
>> think it will often (or at least sometimes?) have been constructed as 
>> the bottleneck. Any operator having designed such a QoS arrangement 
>> would not want to support sequencing...
>
> Yes, BNGs are often planned to be the bottleneck.  But as I say that’s 
> about fairness between subscribers and potentially prioritising 
> different traffic for each subscriber.
>
>>> As noted by Derek I’m guessing reordering is an issue for L2TPv2 if 
>>> stateful PPP compression schemes are in play (which I suspect is 
>>> unlikely to be the case given we usually have abundant bandwidth in 
>>> the aggregation network, and given that compression can occur at 
>>> other layers).  Though given that BNGs inherently keep state per 
>>> subscriber maybe the NPU scaling issues that Stewart mentioned are 
>>> less of an issue in that use case than for MPLS PEs doing PWE?
>>
>> My concern was that 'keep it simple' operators that are using 
>> L2TPv2/3 and had not previously bothered with the complexities of QoS 
>> might want to support L4S, because it has the potential to cut out 
>> queue delay for /all/ traffic. Altho' L4S is eventually for all 
>> traffic, it still requires two queues at the bottleneck for 
>> transition - one for L4S and one for not-yet-L4S ('Classic') traffic 
>> flows, and therefore introduces reordering at the aggregate level...
>
> Again I presume any single tunnel being passed *through* a BNG would 
> either be L4S or not-yet-L4S, and hence not subject to reordering?

[BB] Not so, I'm afraid. L4S is enabled on a per-host basis, and during 
the transition, some traffic in the IP aggregate to (or from) a customer 
site will be L4S, and some Classic (not-yet-L4S).

It's this reordering scenario that begged my original question - whether 
sequencing might be used anywhere for IP data over a tunnel.

Cheers



Bob

>
> Giles
>
>>
>> From the replies so far, even if such 'keep it simple' operators were 
>> using compression, I can't see any reason why having to turn off 
>> compression and sequencing (in order to support L4S) would be a 
>> significant problem nowadays.
>>
>> So, in conclusion, I don't think we even need to raise any concerns 
>> about L2TP sequencing in the L4S specs.
>>
>> If anyone here thinks otherwise, pls speak now.
>>
>> Thank you everyone who has contributed to this discussion.
>>
>> Cheers
>>
>>
>>
>> Bob
>>
>>
>>
>>>
>>> From a quick look at the DOCSIS rPHY specs it seems they do require 
>>> support for L2TPv3 sequence numbers.  Of course in that case the 
>>> payload is the DOCSIS MAC rather than IP (even though, of course, 
>>> most DOCSIS frames ultimately carry an IP payload).
>>>
>>> Giles
>>>
>>>> On 10 Jun 2021, at 12:49, Andrew G. Malis <[e-mail] 
>>>> <mailto:[e-mail]>> wrote:
>>>>
>>>> (resending with cc: list trimmed to pass the too many recipients 
>>>> filter)
>>>> Mark,
>>>>
>>>> The original question was, how many (if any) of these L2TPv2 and v3 
>>>> use cases use sequencing, especially when carrying IP?
>>>>
>>>> Cheers,
>>>> Andy
>>>>
>>>> On Wed, Jun 9, 2021 at 6:32 PM Mark Townsley <[e-mail] 
>>>> <mailto:[e-mail]>> wrote:
>>>>
>>>>     Hi folks,
>>>>
>>>>     In addition to the DSL arena, L2TP is still in use with
>>>>     host-based VPN clients as it is embedded in Apple, Android, and
>>>>     Windows based operating systems (new and old). Despite most VPN
>>>>     solutions preferring their own client software that must be
>>>>     installed on hosts to operate, there are still admins that
>>>>     appreciate not having to ask their employees to download an app
>>>>     for the VPN to work - in which case PPTP and L2TP with
>>>>     transport-mode IPsec are your most widely available options.
>>>>
>>>>     Regarding L2TPv3 replacing L2TP: L2TPv2 (RFC 2661) was PPP
>>>>     only. L2TPv3 generalized L2TP to support other L2 (including
>>>>     MPLS, but I don’t want to argue what layer MPLS operates within
>>>>     here). There was never a strong push to replace L2TPv2 used in
>>>>     DSL, Dialup and host VPN software with L2TPv3 (there was one
>>>>     use case for an important L2TP operator that wanted to carry
>>>>     PPPoE over L2TPv3 in DSL, but that was overcome by RFC3817
>>>>     which achieved the same goal without altering the dataplane).
>>>>     Ironically, I would expect to see very little PPP over L2TPv3
>>>>     in the wild, though obviously it is possible.
>>>>
>>>>     In the cable broadband world, the DOCSIS DEPI “Remote PHY”
>>>>     specification (similar I suppose to the split BNG spec Joel is
>>>>     referring to) standardized on L2TPv3 and is in active use.
>>>>
>>>>     I also know of at least one vendor that uses Ethernet over
>>>>     L2TPv3 for some wifi backhaul use cases.
>>>>
>>>>     There could be more, this is just what I am personally aware of
>>>>     off the top of my head. Even I am surprised to see how much
>>>>     L2TP is still out there once you start really looking around ;-)
>>>>
>>>>     Best Regards,
>>>>
>>>>     - Mark
>>>>
>>>>
>>>>
>>>>
>>>>     > On Jun 9, 2021, at 6:10 AM, Joel Halpern Direct
>>>>     <[e-mail]
>>>>     <mailto:[e-mail]>> wrote:
>>>>     >
>>>>     > BNGs are still a big busienss.  And BNG resale /emote control
>>>>     uses L2TP in many cases.  The BBF has been working on (and
>>>>     published a first version of) protocol for control of split
>>>>     BNG.  L2TP is commonly used for these use cases.
>>>>     >
>>>>     > Yours,
>>>>     > Joel
>>>>     >
>>>>     > On 6/9/2021 7:50 AM, Stewart Bryant wrote:
>>>>     >> Which applications still use it Joel?
>>>>     >> Stewart
>>>>     >>> On 9 Jun 2021, at 12:42, Joel M. Halpern
>>>>     <[e-mail] <mailto:[e-mail]>> wrote:
>>>>     >>>
>>>>     >>> There is plenty of L2TP still in use.
>>>>     >>> Yours,
>>>>     >>> Joel
>>>>     >>>
>>>>     >>> On 6/9/2021 6:23 AM, Stewart Bryant wrote:
>>>>     >>>> Sequence number checking in the forwarder is always a
>>>>     problem because it is stateful so I doubt that many high-scale
>>>>     or high-speed forwarders ever did this.
>>>>     >>>> I think there is an undisclosed assumption that go up
>>>>     enough levels and its IP so sequence number checking in the
>>>>     transport network (as opposed to the transport layer) is not
>>>>     really needed.
>>>>     >>>> I doubt that there is much L2TP still out there. It was in
>>>>     its prime with dialup modems. L2TPv3 which was intended to
>>>>     replace it became niche with, as Andy says, operators who did
>>>>     not want MPLS. Much of what L2TPv3 was intended for was
>>>>     actually done with PW over MPLS with some replacement with by
>>>>     Mac in Mac for cost reasons.
>>>>     >>>> If Carlos does not know the answer, Mark T would be my
>>>>     next port of call.
>>>>     >>>> Stewart
>>>>     >>>>> On 8 Jun 2021, at 22:41, Andrew G. Malis
>>>>     <[e-mail] <mailto:[e-mail]>
>>>>     <mailto:[e-mail] <mailto:[e-mail]>>> wrote:
>>>>     >>>>>
>>>>     >>>>> Bob,
>>>>     >>>>>
>>>>     >>>>> In addition to the cases listed by Derek, L2TPv3 can also
>>>>     carry non-IP pseudowire data, such as Ethernet frames (see RFC
>>>>     4719 for example). Even though 4719 says that sequencing is
>>>>     optional, I would certainly recommend it :-).
>>>>     >>>>>
>>>>     >>>>> But I guess that's really not what you were asking about,
>>>>     since you specifically mentioned IP data. But it is a case
>>>>     where you would probably see sequencing in use.
>>>>     >>>>>
>>>>     >>>>> Back in the day, Sprint made good use of Ethernet over
>>>>     L2TPv3, as they were in the anti-MPLS camp at the time. But
>>>>     that's water over the bridge, and I really don't know if this
>>>>     solution continues to be in active use. Mark Townsley might know.
>>>>     >>>>>
>>>>     >>>>> Cheers,
>>>>     >>>>> Andy
>>>>     >>>>>
>>>>     >>>>>
>>>>     >>>>> On Sat, Jun 5, 2021 at 10:07 AM Derek Fawcus
>>>>     <[e-mail]
>>>>     <mailto:[e-mail]>
>>>>     <mailto:[e-mail]
>>>>     <mailto:[e-mail]>>> wrote:
>>>>     >>>>>
>>>>     >>>>>    On Fri, Jun 04, 2021 at 03:13:15PM +0100, Bob Briscoe
>>>>     wrote:
>>>>     >>>>>    > The L2TP RFC says sequencing /can/ be disabled for
>>>>     IP data, but it
>>>>     >>>>>    > doesn't say SHOULD or MUST. Is it possible that some
>>>>     operators
>>>>     >>>>>    enable
>>>>     >>>>>    > L2TP sequencing for IP data? And if so, do you know
>>>>     why they would?
>>>>     >>>>>    > Also, are you aware of any other types of tunnel
>>>>     that might try
>>>>     >>>>>    to keep
>>>>     >>>>>    > IP data packets in sequence?
>>>>     >>>>>
>>>>     >>>>>    How many intermediate headers are you considering
>>>>     between L2TP and
>>>>     >>>>>    where
>>>>     >>>>>    a carried IP header may exist?
>>>>     >>>>>
>>>>     >>>>>    Maybe I'm getting the wrong end of the stick, but
>>>>     surely this engages
>>>>     >>>>>    the text from section 5.4 of RFC 2661:
>>>>     >>>>>
>>>>     >>>>>      "For example, if the PPP session being tunneled is not
>>>>     >>>>>       utilizing any stateful compression or encryption
>>>>     protocols and is
>>>>     >>>>>       only carrying IP (as determined by the PPP NCPs
>>>>     that are
>>>>     >>>>>       established), then the LNS might decide to disable
>>>>     sequencing as IP
>>>>     >>>>>       is tolerant to datagram loss and reordering."
>>>>     >>>>>
>>>>     >>>>>    This would then suggest if L2TP is carrying PPP, the
>>>>     PPP session
>>>>     >>>>>    is not
>>>>     >>>>>    multi-link, and is making use of compression
>>>>     (including one of the
>>>>     >>>>>    versions of IP header compression) in some form for IP
>>>>     packets, then
>>>>     >>>>>    reordering will impact the ability to decompress.
>>>>     >>>>>
>>>>     >>>>>    So such an L2TP data session may well make use of
>>>>     sequence numbers to
>>>>     >>>>>    prevent reordering.
>>>>     >>>>>
>>>>     >>>>>    I guess similarly in L2TPv3 when the PW is for PPP,
>>>>     and possibly also
>>>>     >>>>>    the fragmentation scheme in RFC 4623 which requires
>>>>     sequence numbers;
>>>>     >>>>>    and such PWE3 links could ultimately be carrying IP
>>>>     packets.
>>>>     >>>>>
>>>>     >>>>>
>>>>     >>>>>    DF
>>>>     >>>>>
>>>>     >>>>>    (not an operator)
>>>>     >>>>>
>>>>     >>>>> _______________________________________________
>>>>     >>>>>    Int-area mailing list
>>>>     >>>>> [e-mail] <mailto:[e-mail]>
>>>>     <mailto:[e-mail] <mailto:[e-mail]>>
>>>>     >>>>> https://www.ietf.org/mailman/listinfo/int-area
>>>>     <https://www.ietf.org/mailman/listinfo/int-area>
>>>>     >>>>>    <https://www.ietf.org/mailman/listinfo/int-area
>>>>     <https://www.ietf.org/mailman/listinfo/int-area>>
>>>>     >>>>>
>>>>     >>>>> _______________________________________________
>>>>     >>>>> Int-area mailing list
>>>>     >>>>> [e-mail] <mailto:[e-mail]>
>>>>     <mailto:[e-mail] <mailto:[e-mail]>>
>>>>     >>>>> https://www.ietf.org/mailman/listinfo/int-area
>>>>     <https://www.ietf.org/mailman/listinfo/int-area>
>>>>     >>>> _______________________________________________
>>>>     >>>> Int-area mailing list
>>>>     >>>> [e-mail] <mailto:[e-mail]>
>>>>     >>>> https://www.ietf.org/mailman/listinfo/int-area
>>>>     <https://www.ietf.org/mailman/listinfo/int-area>
>>>>
>>>> _______________________________________________
>>>> Pals mailing list
>>>> [e-mail] <mailto:[e-mail]>
>>>> https://www.ietf.org/mailman/listinfo/pals
>>>
>>
>> -- 
>> ________________________________________________________________
>> Bob Briscoehttp://bobbriscoe.net/
>

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/