Re: Proposal: Run QUIC over DTLS

Subodh Iyengar <> Tue, 06 March 2018 18:31 UTC

Return-Path: <>
Received: from localhost (localhost []) by (Postfix) with ESMTP id 64A0212D779 for <>; Tue, 6 Mar 2018 10:31:10 -0800 (PST)
X-Virus-Scanned: amavisd-new at
X-Spam-Flag: NO
X-Spam-Score: -0.731
X-Spam-Status: No, score=-0.731 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, HTTPS_HTTP_MISMATCH=1.989, RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_PASS=-0.001] autolearn=unavailable autolearn_force=no
Authentication-Results: (amavisd-new); dkim=pass (1024-bit key) header.b=khMlLmDO; dkim=pass (1024-bit key) header.b=OV6ctJ9d
Received: from ([]) by localhost ( []) (amavisd-new, port 10024) with ESMTP id BCjYkcfYLpbS for <>; Tue, 6 Mar 2018 10:31:08 -0800 (PST)
Received: from ( []) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by (Postfix) with ESMTPS id 69F6D12D7E8 for <>; Tue, 6 Mar 2018 10:31:03 -0800 (PST)
Received: from pps.filterd ( []) by ( with SMTP id w26IIsea010522; Tue, 6 Mar 2018 10:30:58 -0800
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : mime-version; s=facebook; bh=G9EhbArdT8Ksgv3O6Hg5MESYxikkhq6/kDa3Uh5znHk=; b=khMlLmDOBPQbYlXMa4jGGVsV2fPxlbM1k3LIHiofCbYj0/oQ88QEDrE8DMEpW22CZISx QTEm9ULJdrnIKHwbeTEa2Kv7G7CYp2nofwL2VIB0nazbTGW3pXJyPgWwSJBAVVLoBUz5 XZsU+IWWQVIHkjl/G2IYUGtFb0uM9KkSbOc=
Received: from ([]) by with ESMTP id 2ghy138aas-3 (version=TLSv1 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=NOT); Tue, 06 Mar 2018 10:30:45 -0800
Received: from ( by ( with Microsoft SMTP Server (TLS) id 14.3.361.1; Tue, 6 Mar 2018 10:30:16 -0800
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;; s=selector1-fb-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version; bh=G9EhbArdT8Ksgv3O6Hg5MESYxikkhq6/kDa3Uh5znHk=; b=OV6ctJ9d8RH8UM38eClDLwGQUUGSxj9hVHSvOqPLfMa3mKnAfYl3UL34vRoCt0DS9ByJfYjcHvLYDmfEexSUk4egl3bJKsQGjNtoFYVtIit+DtRIw+UhoYPHYb9br0eNcUskTmnbp/u94E5JBm2SO2Rb+QFh9z64qNv9fWx0K3g=
Received: from ( by ( with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384_P256) id 15.20.548.13; Tue, 6 Mar 2018 18:30:14 +0000
Received: from ([fe80::b054:d63c:f848:809d]) by ([fe80::b054:d63c:f848:809d%17]) with mapi id 15.20.0548.016; Tue, 6 Mar 2018 18:30:14 +0000
From: Subodh Iyengar <>
To: Victor Vasiliev <>, Eric Rescorla <>
Subject: Re: Proposal: Run QUIC over DTLS
Thread-Topic: Proposal: Run QUIC over DTLS
Thread-Index: AQHTtNa4B4uCeAaGiUaku+WfqQMubKPDhkgAgAABBQU=
Date: Tue, 6 Mar 2018 18:30:14 +0000
Message-ID: <>
References: <>, <>
In-Reply-To: <>
Accept-Language: en-US
Content-Language: en-US
x-originating-ip: [2620:10d:c090:180::1:f522]
x-ms-publictraffictype: Email
x-microsoft-exchange-diagnostics: 1; MWHPR15MB1598; 7:0Wbe2CRAV7Js/Y95FkARNor8OeCCzMQmh7a3Ecg2cysv9qTreBrEq+Y48PIvR78primXaUHYt0YOyhyElTApQmHImjQYtbiyvVM/vXeHlmxq6rO6GefO2fGUsMKePQly5rA+9n1N0baWB9v7q2LD2uLXUlaDZ5lV8+oDzd9v58C5OQC9fuPKIW5KwRyjYqSC/UYBei2sY1yCDuxhceWle2Q3ZQZMaV+Xd26/5mSTvoou8ZGMfu6AHBqdEaueAUSO; 20:5xCos9Va+S00lmzEQ5C/Fu1evb1r8kgW5x4Q8S6gdF1CPubwZ1/83CBHMkcJZfo+F+VlMvUu5Bt3AlRJiSksavBr3BvaFrDPNKDsVgVaQo8MU27+7hInO5V4fsoMYIKIuYSPjSUZJu9QEa5CxRYEXLgYe7aTVFNCAO0YY104cf8=
x-ms-exchange-antispam-srfa-diagnostics: SSOS;
x-ms-office365-filtering-correlation-id: 9761a859-1521-4536-1fea-08d583904d9a
x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(5600026)(4604075)(3008032)(4534165)(4627221)(201703031133081)(201702281549075)(2017052603328)(7153060)(7193020); SRVR:MWHPR15MB1598;
x-ms-traffictypediagnostic: MWHPR15MB1598:
x-microsoft-antispam-prvs: <>
x-exchange-antispam-report-test: UriScan:(10436049006162)(120809045254105)(166708455590820)(160773892695471)(240460790083961)(17755550239193);
x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(6040501)(2401047)(5005006)(8121501046)(3231220)(11241501184)(944501244)(52105095)(3002001)(10201501046)(93006095)(93001095)(6041288)(20161123562045)(20161123558120)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123560045)(20161123564045)(6072148)(201708071742011); SRVR:MWHPR15MB1598; BCL:0; PCL:0; RULEID:; SRVR:MWHPR15MB1598;
x-forefront-prvs: 06036BD506
x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(39380400002)(346002)(396003)(376002)(366004)(39860400002)(51874003)(199004)(189003)(7736002)(68736007)(106356001)(6246003)(6306002)(2900100001)(6436002)(3280700002)(54896002)(8676002)(110136005)(86362001)(99286004)(316002)(2906002)(33656002)(53936002)(7696005)(236005)(105586002)(3660700001)(55016002)(81156014)(9686003)(8936002)(229853002)(46003)(6116002)(97736004)(76176011)(5660300001)(19627405001)(81166006)(25786009)(186003)(5250100002)(6506007)(102836004)(2950100002)(59450400001)(606006)(53546011)(966005)(561944003)(14454004)(478600001)(74316002)(4326008)(6606003); DIR:OUT; SFP:1102; SCL:1; SRVR:MWHPR15MB1598;; FPR:; SPF:None; PTR:InfoNoRecords; A:1; MX:1; LANG:en;
received-spf: None ( does not designate permitted sender hosts)
x-microsoft-antispam-message-info: VS6LF2/7k9SBS2DNa4ghP4LHLSIkhm5CvQtSx0rQ59VSRvY2d3AShoSrVl8rbqtM9yHOhh8GRbwCgzaS0E6BEAfKl7E6qwXo5sc0TCqxWTEfsudNRVxDJ2yR1LHPN4c3pP+LN4zKx2AOvFwKFjpTH/+DJYrtQHUdLwsczq0FlGTPjQ8ON4G4a+pBizl4hwzEx8L3Qmle0oR7TTPFWVs9XhI2QLaTeisPKiVaE3sOQ0viDwQxqAW0+DBHuY7MLX0hlmxv8t2tbfAvOIeF3GZKfXg2FOtiY4mEe8dCzVbZuW0h3qoFvlYbCuKjljrbbf+AHZQnxc1PLwGK+v6N1yqxlA==
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: multipart/alternative; boundary="_000_MWHPR15MB1821C14B6122682C24D27A65B6D90MWHPR15MB1821namp_"
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-Network-Message-Id: 9761a859-1521-4536-1fea-08d583904d9a
X-MS-Exchange-CrossTenant-originalarrivaltime: 06 Mar 2018 18:30:14.0612 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 8ae927fe-1255-47a7-a2af-5f3a069daaa2
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR15MB1598
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:, , definitions=2018-03-06_10:, , signatures=0
X-Proofpoint-Spam-Reason: safe
X-FB-Internal: Safe
Archived-At: <>
X-Mailman-Version: 2.1.22
Precedence: list
List-Id: Main mailing list of the IETF QUIC working group <>
List-Unsubscribe: <>, <>
List-Archive: <>
List-Post: <>
List-Help: <>
List-Subscribe: <>, <>
X-List-Received-Date: Tue, 06 Mar 2018 18:31:10 -0000

 > I'm thinking about this question as "Is Stream 0 a Design Flaw?".. The DTLS part is largely putting a solution to that problem out front and isn't really the determining question to me. Its a flashy distraction.

 > I think we achieve better clarity at little to no cost by treating these transport-level control data as something 100% decoupled from the application-layer data, i.e. not a stream.

Calling the crypto stream a stream 0 makes implementors naturally assume that the protocol is messy and intertwined, however that is not true in practice. Practically as we've implemented iQUIC, we've realized that crypto streams are special and dealt with as special cases in an implementation and they don't need to be stream 0. Calling the crypto stream as just another reliably delivered frame in the transport with a very similar structure to streams would be one way to signal to implementors that this is a fundamentally different thing to a stream and make the layering structure align better with implementations.

I think the problem of matching the state of the crypto stream and the packetization format will exist in any crypto core layer, DTLS or TLS encryption if we are not willing to give up
1. flexibility of future versions as Mike put it.
 2. The ability for load balancers and implementations to have a cheap way to check the beginning of the connection

 The implementation complexity is a solvable problem, but I think the question here I have is does the editorial representation of the protocol in terms of layering match up with the practical layering of implementations, which I believe is not the case. A few minor changes to the drafts and the protocol might make it match up better.


From: QUIC <>; on behalf of Victor Vasiliev <>;
Sent: Tuesday, March 6, 2018 10:21:44 AM
To: Eric Rescorla
Subject: Re: Proposal: Run QUIC over DTLS

On Mon, Mar 5, 2018 at 6:05 PM, Eric Rescorla <<>> wrote:
Hi folks,

Sorry to be the one randomizing things again, but the asymmetric
conn-id thing went well, so here goes....

I'd like to discuss refactoring things to run QUIC over DTLS.

Hi Ekr,

Thanks for sending out the proposal!

I appreciate the idea of using DTLS, but I'm afraid it just makes things worse.  Comments inline.

When we originally designed the interaction between TLS and QUIC,
there seemed like a lot of advantages to embedding the crypto
handshake on stream 0, in particular the ability to share a common
reliability and congestion mechanism. However, as we've gotten further
along in design and implementation, it's also become clear that it's
archictecturally kind of crufty and this creates a bunch of problems,

  * Stream 0 is unencrypted at the beginning of the connection, but
    encrypted after the handshake completes, and you still need
    to service it.

I am not sure where the complexity comes from?  If anything, not closing the stream seems easier than closing it.

  * Retransmission of stream 0 frames from lost packets needs special
    handling to avoid accidentally encrypting them.

Agreed, you have to handle retransmission of encrypted and handshake data differently.

  * Stream 0 is not subject to flow control; it can exceed limits and
    goes into negative credit after the handshake completes.

Again, we can close stream 0 -- I am not entirely sure what's the cost you're concerned about here, though.

  * There are complicated rules about which packets can ACK other
    packets, as both cleartext and ciphertext ACKs are possible.

I can see the complexity here, but I don't think there are that many issues with it -- if anything, this sounds like a reason to simplify the ACK rules.

  * Very tight coupling between the crypto stack and the transport
    stack, especially in terms of knowing where you are in the
    crypto state machine.

I'm slightly confused here -- by crypto stack, you mean TLS or the crypto framing state machine?

I've been looking at an alternative design in which we instead adopt a
more natural layering of putting QUIC on top of DTLS. The basic
intuition is that you do a DTLS handshake and just put QUIC frames
directly in DTLS records (rather than QUIC packets). This
significantly reduces the degree of entanglement between the two
components and removes the corner cases above, as well as just
generally being a more conventional architecture. Of course, no design
is perfect, but on balance, I think this is a cleaner structure.

I have a draft for this at:<>

And a partial implementation of it in Minq at:


I can't speak for anyone else's implementation, but at least in my
case, the result was considerable simplification.

I think this is actually making layering worse than in current
situation.  See this diagram:<>

The proposed tradeoff is that we get rid of the complexity of having to
deal with both encrypted and unencrypted packets, but as a result we
have to implement two transport stacks -- one inside the TLS library and
one inside the QUIC library.  Having one transport stack is painful
enough.  In practice, DTLS handshake transport mechanisms are always
less polished than commonly used transport stacks.

The layering will still be ugly and leaky, just in different ways -- for
example, you'd have to export the RTT information from your DTLS stack
to your QUIC stack.

It might be interesting to play around with idea of whether we can
actually make unencrypted and encrypted part behave as "two connections
in one" -- i.e.  a cleaner way to reuse the QUIC transport logic while
avoiding painful cross-interactions between them.

It's natural at this point to say that this is coming late in the
process after we have a lot invested in the current design, as well as
to worry that it will delay the process. That's not my intention, and
as I say in the draft, many of the issues we have struggled over
(headers especially) can be directly ported into this architecture (or
perhaps just reused with QUIC-over-DTLS while letting ordinary DTLS do
its thing) and this change would allow us to sidestep issued we are
still fighting with, so on balance I believe we can keep the schedule
impact contained.

Agreed that we should discuss the merits of the proposal before talking
about the schedule.  That said.  If this miraculously solves all of our
problems, we should definitely go for it; if this kinda-sorta solves
some of the problems we've already worked through while introducing a
whole host of new yet-untackled problems, I am much more skeptical.

We are designing a protocol that will be used long into the future, so
having the right architecture is especially important. Our goal has
always been to guide this effort by implementation experience and we
are learning about the deficiencies of the Stream 0 design as we go
down our current path. If the primary concern to this proposal is
schedule we should have an explicit discussion about those relative
priorities in the context of the pros and cons of the proposal.

Is there that much of implementation and deployment experience with DTLS 1.3?

The hackathon would be a good opportunity to have a face to face chat
about this in addition to on-list discussion.

Thanks in advance for taking a look,