Re: [tsvwg] On packet reordering in 5G

[Bob Briscoe <[e-mail]>]

Ingemar,

Thanks muchly for this comprehensive explanation of the state of the art.
I have posed a question for any folks from mobile operators watching 
this thread - inline.

On 03/08/2019 01:06, Ingemar Johansson S wrote:
>
> Hi
>
> Long post and I probably missed something..
>
> I was asked to clarify various aspects of packet reordering in 5G. I 
> mentioned a little on this in 
> https://lists.bufferbloat.net/pipermail/bloat/2019-March/008976.html
>
> I’ll try to keep the discussion around default bearers which are the 
> typical bearers allocated for general mobile broadband (MBB) 
> connectivity. Acknowledged mode (AM) transmission is used.
>
> 4G networks (and before this 3G) ensured in sequence delivery. The 
> main reason behind this was that the TCP DupACK mechanism was 
> sensitive to packet reordering. At that time it was likely a sensible 
> thing to do as most of the traffic was mainly file transfers (and RACK 
> was not discovered)
>
> Things have however changed over time.
> 1) Increased share of interactive services that require low latency. 
> In-sequence delivery leads to HoL blocking issues and that often for 
> no big use as transport protocols are better off to decide how to deal 
> with out of order packets. This applies especially to QUIC, in which 
> case many streams may be transmitted over one connection.
>
> 2) Higher link speeds means more memory is required in network nodes 
> for the sole purpose of in-sequence delivery. For downlink traffic 
> this means more memory in UE (User Equipment = USB modems, CPEs and 
> other cellular devices). One can here argue that if in-sequence 
> delivery is not enforced in on the RLC or PDCP layer, then it is 
> likely needed to happen on the transport layer (e.g. TCP). But then 
> again, for some transport protocols like QUIC where multiple streams 
> are transmitted over the same connection, then it is likely better to 
> get the packets delivered out of order.
>
> First a short recap of the 3GPP user plane protocol stack (GTP stuff 
> omitted), it goes like
> | IP | PDCP | RLC | MAC | PHY |
>
> Where can packet reordering occur in 5G?
>
>  1. On the MAC layer: Data from the RLC layer is put on transport
>     blocks. If a transmission fails, a new transmission of the same
>     data is repeated (but with a slightly different error control
>     coding, incremental redundancy is used here to improve the
>     spectrum efficiency. The max number of retransmits can be for
>     instance 4, but this is configurable and vendor specific. Up to 16
>     HARQ (Hybrid AQR) processes can be run in NR (LTE has a limit of
>     8), this means that one chunk of data can be successfully
>     transmitted before a previous chunk. This causes packet
>     reordering. In LTE the reordering depth could be 8ms if one
>     retransmission occurred, 16ms in case the same chunk of data is
>     retransmitted twice and so on. In NR this reordering jitter is
>     shorter, both as a result of a faster HARQ processing and shorter
>     transmission intervals. As an example, with the highest numerology
>     (0.125ms TTI=transmission interval) the jitter on the MAC layer is
>     typically less than 1ms.
>  2. On the RLC layer: The MAC layer transmission can fail if the max
>     number of retransmission attempts is reached, this can occur if
>     for instance the link adaptation (adjustment of the modulation and
>     coding) is not optimal, in this case it is up to the RLC layer to
>     retransmit the missing data.
>  3. Dual Connectivity: There can be cases where LTE is used for wide
>     area coverage and NR is used for small cell coverage. One example
>     is LTE running in the 2.4MHz band with 20MHz bandwidth and NR
>     running on the 28GHz band with a 100MHz bandwidth. With Dual
>     Connectivity, the data can be transmitted over both legs. Flow
>     control is used to portion the data between the two legs.
>     Depending on the efficiency of the  flow control and how the link
>     bandwidth over the two legs vary, there will be packet reordering
>     to a smaller or larger degree. I cannot give any example of
>     typical reordering depths here as this reordering depends on the
>     flow control algorithms in use.
>
> In LTE it was up to the RLC layer to ensure in-sequence delivery up to 
> higher layers. In 5G, this role is moved up to the PDCP layer, one 
> reason is to make Dual Connectivity easier to implement.
>
And this makes more sense once the RLC frames map 1:1 to IP packets in 5G.

> In addition, it is optional to implement in-sequence delivery. It is 
> currently not possible for me to say if in-sequence delivery is 
> implemented as it is vendor specific.
>
Assuming equipment vendors implement in-sequence delivery, the real 
question is whether mobile operators will still want it enabled on the 
default (MBB) bearer. Does anyone (e.g. from an operator) have a feel 
for an answer?

I assume that operators will keep PDCP in-sequence delivery enabled 
until the prevalence of modern re-ordering tolerant transports combined 
with the benefits to them outweigh the prevalence and detriments to 
conventional (3DupACK) transports.

This messy compromise is why we took the opportunity to draw a line in 
the sand with L4S, by requiring sender algorithms that have to be 
modified to support L4S to also have modern reordering tolerance from day-1.

I know there are objections on editorial, architectural and procedural 
grounds, so I shall be working through that minefield next week, because 
I've seen a lot of support on technical and pragmatic grounds - the two 
types are hard to weigh against each other.


> For the interested, this book gives more insight to the NR standard.
>
> https://www.elsevier.com/books/5g-nr-the-next-generation-wireless-access-technology/dahlman/978-0-12-814323-0
>
> OK, looks like shameless marketing but I can assure you that the 
> authors (Ericsson employees) will pay me nothing for this ☹. Still the 
> book is quite comprehensible and explains many of the design 
> considerations.
>
> Hope that this sheds at least some light
>
Indeed it does. Thank you again.



Bob


> /Ingemar
>
> ================================
>
> Ingemar Johansson  M.Sc.
>
> Master Researcher
>
> Ericsson Research
>
> Network Protocols & E2E Performance
>
> Labratoriegränd 11
>
> 971 28, Luleå, Sweden
>
> Phone [phone]
>
> SMS/MMS [phone]
>
> [e-mail]
>
> www.ericsson.com
>
> You’ve got to live with
>
>          What you can’t rise above
>                 Bruce Springsteen
>
> =================================
>

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/