Re: [tsvwg] On packet reordering in 5G

Bob Briscoe <> Sat, 03 August 2019 13:20 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id A2E1A1201EC for <>; Sat, 3 Aug 2019 06:20:49 -0700 (PDT)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -1.998
X-Spam-Status: No, score=-1.998 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (2048-bit key)
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id FORYL0ZvFJ9W for <>; Sat, 3 Aug 2019 06:20:46 -0700 (PDT)
Received: from ( []) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 6C87812003F for <>; Sat, 3 Aug 2019 06:20:45 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;; s=default; h=Content-Type:In-Reply-To:MIME-Version:Date: Message-ID:From:References:Cc:To:Subject:Sender:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Id: List-Help:List-Unsubscribe:List-Subscribe:List-Post:List-Owner:List-Archive; bh=SGBrD93SgqqxRg1y7/HedKjzBcyvR5Zq5jugqv9pDrs=; b=YdC9xlsNdWqfc6wqgVqfjev8k f9WqyduuWQ1eHMgvCxG1VOzDvXlrDgxuyvoAHLyX5Hf0HoL0RttGHI5QuD1G9HILl8DY91pt58XmJ iNODJr5+foFDBVxdgJDCI1Gb9mQrO57qRQOMC2YE4cTKPQP6zO7ZAImCeZgtNMuQ+Y0zajpMRfLL8 ld1jRuEs5+rgZSO0pJvE39pfD3pHUZzFHk8MXSVKvRMRoO7xo+IlvKv1ZnSv52D43gBvOkV2MuWdV 1IEJU8iu7Q/jrJiVludpIHZMMY2AvvEKuFLCjbVxGwkeW9ORxbMMRHmgyDR2+La0zXqZQVBu5Ym5A To8qL0WUQ==;
Received: from [] (port=56902 helo=[]) by with esmtpsa (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.92) (envelope-from <>) id 1httxh-0003VU-C2; Sat, 03 Aug 2019 14:20:41 +0100
To: Ingemar Johansson S <>, "" <>
Cc: Greg White <>, "Black, David" <>, "" <>
References: <>
From: Bob Briscoe <>
Message-ID: <>
Date: Sat, 3 Aug 2019 14:20:40 +0100
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1
MIME-Version: 1.0
In-Reply-To: <>
Content-Type: multipart/alternative; boundary="------------646AE4F1C7DF97DB8A28F647"
Content-Language: en-GB
X-AntiAbuse: This header was added to track abuse, please include it with any abuse report
X-AntiAbuse: Primary Hostname -
X-AntiAbuse: Original Domain -
X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12]
X-AntiAbuse: Sender Address Domain -
X-Get-Message-Sender-Via: authenticated_id:
Archived-At: <>
Subject: Re: [tsvwg] On packet reordering in 5G
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: Transport Area Working Group <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Sat, 03 Aug 2019 13:20:50 -0000


Thanks muchly for this comprehensive explanation of the state of the art.
I have posed a question for any folks from mobile operators watching 
this thread - inline.

On 03/08/2019 01:06, Ingemar Johansson S wrote:
> Hi
> Long post and I probably missed something..
> I was asked to clarify various aspects of packet reordering in 5G. I 
> mentioned a little on this in 
> I’ll try to keep the discussion around default bearers which are the 
> typical bearers allocated for general mobile broadband (MBB) 
> connectivity. Acknowledged mode (AM) transmission is used.
> 4G networks (and before this 3G) ensured in sequence delivery. The 
> main reason behind this was that the TCP DupACK mechanism was 
> sensitive to packet reordering. At that time it was likely a sensible 
> thing to do as most of the traffic was mainly file transfers (and RACK 
> was not discovered)
> Things have however changed over time.
> 1) Increased share of interactive services that require low latency. 
> In-sequence delivery leads to HoL blocking issues and that often for 
> no big use as transport protocols are better off to decide how to deal 
> with out of order packets. This applies especially to QUIC, in which 
> case many streams may be transmitted over one connection.
> 2) Higher link speeds means more memory is required in network nodes 
> for the sole purpose of in-sequence delivery. For downlink traffic 
> this means more memory in UE (User Equipment = USB modems, CPEs and 
> other cellular devices). One can here argue that if in-sequence 
> delivery is not enforced in on the RLC or PDCP layer, then it is 
> likely needed to happen on the transport layer (e.g. TCP). But then 
> again, for some transport protocols like QUIC where multiple streams 
> are transmitted over the same connection, then it is likely better to 
> get the packets delivered out of order.
> First a short recap of the 3GPP user plane protocol stack (GTP stuff 
> omitted), it goes like
> | IP | PDCP | RLC | MAC | PHY |
> Where can packet reordering occur in 5G?
>  1. On the MAC layer: Data from the RLC layer is put on transport
>     blocks. If a transmission fails, a new transmission of the same
>     data is repeated (but with a slightly different error control
>     coding, incremental redundancy is used here to improve the
>     spectrum efficiency. The max number of retransmits can be for
>     instance 4, but this is configurable and vendor specific. Up to 16
>     HARQ (Hybrid AQR) processes can be run in NR (LTE has a limit of
>     8), this means that one chunk of data can be successfully
>     transmitted before a previous chunk. This causes packet
>     reordering. In LTE the reordering depth could be 8ms if one
>     retransmission occurred, 16ms in case the same chunk of data is
>     retransmitted twice and so on. In NR this reordering jitter is
>     shorter, both as a result of a faster HARQ processing and shorter
>     transmission intervals. As an example, with the highest numerology
>     (0.125ms TTI=transmission interval) the jitter on the MAC layer is
>     typically less than 1ms.
>  2. On the RLC layer: The MAC layer transmission can fail if the max
>     number of retransmission attempts is reached, this can occur if
>     for instance the link adaptation (adjustment of the modulation and
>     coding) is not optimal, in this case it is up to the RLC layer to
>     retransmit the missing data.
>  3. Dual Connectivity: There can be cases where LTE is used for wide
>     area coverage and NR is used for small cell coverage. One example
>     is LTE running in the 2.4MHz band with 20MHz bandwidth and NR
>     running on the 28GHz band with a 100MHz bandwidth. With Dual
>     Connectivity, the data can be transmitted over both legs. Flow
>     control is used to portion the data between the two legs.
>     Depending on the efficiency of the  flow control and how the link
>     bandwidth over the two legs vary, there will be packet reordering
>     to a smaller or larger degree. I cannot give any example of
>     typical reordering depths here as this reordering depends on the
>     flow control algorithms in use.
> In LTE it was up to the RLC layer to ensure in-sequence delivery up to 
> higher layers. In 5G, this role is moved up to the PDCP layer, one 
> reason is to make Dual Connectivity easier to implement.
And this makes more sense once the RLC frames map 1:1 to IP packets in 5G.

> In addition, it is optional to implement in-sequence delivery. It is 
> currently not possible for me to say if in-sequence delivery is 
> implemented as it is vendor specific.
Assuming equipment vendors implement in-sequence delivery, the real 
question is whether mobile operators will still want it enabled on the 
default (MBB) bearer. Does anyone (e.g. from an operator) have a feel 
for an answer?

I assume that operators will keep PDCP in-sequence delivery enabled 
until the prevalence of modern re-ordering tolerant transports combined 
with the benefits to them outweigh the prevalence and detriments to 
conventional (3DupACK) transports.

This messy compromise is why we took the opportunity to draw a line in 
the sand with L4S, by requiring sender algorithms that have to be 
modified to support L4S to also have modern reordering tolerance from day-1.

I know there are objections on editorial, architectural and procedural 
grounds, so I shall be working through that minefield next week, because 
I've seen a lot of support on technical and pragmatic grounds - the two 
types are hard to weigh against each other.

> For the interested, this book gives more insight to the NR standard.
> OK, looks like shameless marketing but I can assure you that the 
> authors (Ericsson employees) will pay me nothing for this ☹. Still the 
> book is quite comprehensible and explains many of the design 
> considerations.
> Hope that this sheds at least some light
Indeed it does. Thank you again.


> /Ingemar
> ================================
> Ingemar Johansson  M.Sc.
> Master Researcher
> Ericsson Research
> Network Protocols & E2E Performance
> Labratoriegränd 11
> 971 28, Luleå, Sweden
> Phone +46-1071 43042
> SMS/MMS +46-73 078 3289
> You’ve got to live with
>          What you can’t rise above
>                 Bruce Springsteen
> =================================

Bob Briscoe