Re: Proposal: drop QPACK encoder stream framing

Alan Frindell <afrind@fb.com> Fri, 08 June 2018 20:49 UTC

Return-Path: <prvs=96974df54d=afrind@fb.com>
X-Original-To: quic@ietfa.amsl.com
Delivered-To: quic@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A17A3130DC4 for <quic@ietfa.amsl.com>; Fri, 8 Jun 2018 13:49:44 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.721
X-Spam-Level:
X-Spam-Status: No, score=-2.721 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001, T_DKIMWL_WL_HIGH=-0.01, T_DKIMWL_WL_MED=-0.01] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=fb.com header.b=dyqT5hV0; dkim=pass (1024-bit key) header.d=fb.onmicrosoft.com header.b=KVUQMyrI
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id AEY59_duXJL6 for <quic@ietfa.amsl.com>; Fri, 8 Jun 2018 13:49:42 -0700 (PDT)
Received: from mx0a-00082601.pphosted.com (mx0a-00082601.pphosted.com [67.231.145.42]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id C6FDC12F1A5 for <quic@ietf.org>; Fri, 8 Jun 2018 13:49:42 -0700 (PDT)
Received: from pps.filterd (m0109334.ppops.net [127.0.0.1]) by mx0a-00082601.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w58KmGET015669; Fri, 8 Jun 2018 13:49:40 -0700
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-id : content-transfer-encoding : mime-version; s=facebook; bh=8774ZncPc7bQGAAqSw73RaFaTJAKMmER43yGj8Hwbjw=; b=dyqT5hV0GXJlWlow1ks+u/VXd14IyUpIWYgfiL8f8pzd14bPfXz5ox3krhknmy/LHA3T ax1TLSRKxHH4HSN8slKpPumzC2AKtcs/lUlX3CFKkrs+FC1sgJEkhj4axfeNiA1X0Nq6 CTwCvwsSCbB9d9ffpF4Ds0U0XnYQoqaO6uY=
Received: from maileast.thefacebook.com ([199.201.65.23]) by mx0a-00082601.pphosted.com with ESMTP id 2jg007r6ay-1 (version=TLSv1 cipher=ECDHE-RSA-AES256-SHA bits=256 verify=NOT); Fri, 08 Jun 2018 13:49:40 -0700
Received: from NAM05-DM3-obe.outbound.protection.outlook.com (192.168.183.28) by o365-in.thefacebook.com (192.168.177.34) with Microsoft SMTP Server (TLS) id 14.3.361.1; Fri, 8 Jun 2018 16:49:38 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=fb.onmicrosoft.com; s=selector1-fb-com; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=8774ZncPc7bQGAAqSw73RaFaTJAKMmER43yGj8Hwbjw=; b=KVUQMyrILd7ITmnhbngjSjFGHMP6mgRmZjz5fGNQHLkIQ1QFEg1uSlS5twArfbIvf7A4bXz01dIz3vxF5dHPlCtBDdhDTvnymAttMtxXej3OMdC2n0Ei/4tapwpoNguzP2GxNm85+/mgIjtc5GzP2UVgBO0/YGy/0IscOn9xyZ8=
Received: from MWHPR15MB1181.namprd15.prod.outlook.com (10.175.2.135) by MWHPR15MB1167.namprd15.prod.outlook.com (10.175.2.21) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.841.14; Fri, 8 Jun 2018 20:49:37 +0000
Received: from MWHPR15MB1181.namprd15.prod.outlook.com ([fe80::399c:db50:dfc0:3860]) by MWHPR15MB1181.namprd15.prod.outlook.com ([fe80::399c:db50:dfc0:3860%4]) with mapi id 15.20.0841.015; Fri, 8 Jun 2018 20:49:37 +0000
From: Alan Frindell <afrind@fb.com>
To: Kazuho Oku <kazuhooku@gmail.com>, Martin Thomson <martin.thomson@gmail.com>
CC: QUIC WG <quic@ietf.org>
Subject: Re: Proposal: drop QPACK encoder stream framing
Thread-Topic: Proposal: drop QPACK encoder stream framing
Thread-Index: AQHT/nHqXJJQhdl+r0afCsPvVPdYN6RWJZAAgAAStYCAACnpgA==
Date: Fri, 08 Jun 2018 20:49:37 +0000
Message-ID: <F04D1BA6-8EC6-43EB-ABEC-34C4B30352AB@fb.com>
References: <20180607151112.GA28823@ubuntu-dmitri> <CABkgnnXbfkVbq6vCvud100h6wr3+9ir3iO7dD6-qaykOmK3JBQ@mail.gmail.com> <CANatvzzzXF8WCXgDCahTa_7JWZ1c1i-9rU_c8c9QWk5_O5u4Ow@mail.gmail.com>
In-Reply-To: <CANatvzzzXF8WCXgDCahTa_7JWZ1c1i-9rU_c8c9QWk5_O5u4Ow@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
user-agent: Microsoft-MacOutlook/10.c.0.180410
x-originating-ip: [2620:10d:c090:200::6:31fc]
x-ms-publictraffictype: Email
x-microsoft-exchange-diagnostics: 1; MWHPR15MB1167; 7:5Tg5u0iXipKWx7rKNLcvGzD+8TllbipJwbFrQDcsGJt8AsvZ/RFOUYiGIQArUhUssJiBxo/jqQHQtTOcF/OFzhKK4nYBqqJI9ThdXil2E5qfK6MWHrjpnNBUFeho4oQ8dmq7FrXkLtatmCaOUkiKbPBhuZH3VLX4QTw1d5z4M+B5nexFyfgPnorG1n/pFD6cOEhiPOS+iG2JCGmavJWE04cbQL0VjRWB62KTX6LQ1hD/m28aSWV+A7KJnifBhOw7; 20:yX9/aFonUC1pDAGIQ1Kbsfq7DiS02IPfRO5RqSWwjPRfcMVqYeGcbRAb8OYB6JoW4F52M87euRgPxjoCSeThKnxrSeKBqY2FUqgWAWum7y0sSRfgbozdWc8I5Wr4BONAD1fe4aKYMuxqbAPy3p1/HfmZfbhWgx9IhFBsmx5iaks=
x-ms-exchange-antispam-srfa-diagnostics: SOS;
x-microsoft-antispam: UriScan:; BCL:0; PCL:0; RULEID:(7020095)(4652020)(5600026)(4534165)(4627221)(201703031133081)(201702281549075)(2017052603328)(7153060)(7193020); SRVR:MWHPR15MB1167;
x-ms-traffictypediagnostic: MWHPR15MB1167:
x-microsoft-antispam-prvs: <MWHPR15MB1167AFA8B3B1B0D6A01784F0A77B0@MWHPR15MB1167.namprd15.prod.outlook.com>
x-exchange-antispam-report-test: UriScan:;
x-ms-exchange-senderadcheck: 1
x-exchange-antispam-report-cfa-test: BCL:0; PCL:0; RULEID:(8211001083)(6040522)(2401047)(8121501046)(5005006)(93006095)(93001095)(10201501046)(3002001)(3231254)(11241501184)(944501410)(52105095)(149027)(150027)(6041310)(201703131423095)(201702281528075)(20161123555045)(201703061421075)(201703061406153)(20161123560045)(20161123562045)(20161123558120)(20161123564045)(6072148)(201708071742011)(7699016); SRVR:MWHPR15MB1167; BCL:0; PCL:0; RULEID:; SRVR:MWHPR15MB1167;
x-forefront-prvs: 06973FFAD3
x-forefront-antispam-report: SFV:NSPM; SFS:(10019020)(39380400002)(396003)(366004)(376002)(39860400002)(346002)(189003)(199004)(446003)(11346002)(2616005)(486006)(14454004)(105586002)(106356001)(2906002)(46003)(86362001)(58126008)(39060400002)(305945005)(53936002)(6512007)(5660300001)(7736002)(6246003)(83716003)(5250100002)(4326008)(82746002)(110136005)(6486002)(316002)(81156014)(478600001)(8676002)(186003)(102836004)(229853002)(8936002)(3280700002)(97736004)(36756003)(3660700001)(6436002)(81166006)(6506007)(6116002)(476003)(68736007)(33656002)(2900100001)(76176011)(99286004)(25786009); DIR:OUT; SFP:1102; SCL:1; SRVR:MWHPR15MB1167; H:MWHPR15MB1181.namprd15.prod.outlook.com; FPR:; SPF:None; LANG:en; PTR:InfoNoRecords; MX:1; A:1;
received-spf: None (protection.outlook.com: fb.com does not designate permitted sender hosts)
x-microsoft-antispam-message-info: QudqTcagfPNnI/15qwfeq/nCeHQn4jA4s/f7uOn9+tnfSdYoczvWX5eGw+dzez5IIvXUgQLP7hqQ8rJenxTM4PJdluc2HSQT8REUZMshVUxHmUz5t40fS38SDELDjg0GANMIwW3zd7fZKPaig7XwOR1FjSW/tac/adOoHORHcgFReaIJ3vFoMsMNoMDuhIO1
spamdiagnosticoutput: 1:99
spamdiagnosticmetadata: NSPM
Content-Type: text/plain; charset="utf-8"
Content-ID: <46A9CDAC1283D440BBBD3A7576329825@namprd15.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-MS-Office365-Filtering-Correlation-Id: 4f541b21-adeb-4fa6-4fe5-08d5cd815926
X-MS-Exchange-CrossTenant-Network-Message-Id: 4f541b21-adeb-4fa6-4fe5-08d5cd815926
X-MS-Exchange-CrossTenant-originalarrivaltime: 08 Jun 2018 20:49:37.1220 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 8ae927fe-1255-47a7-a2af-5f3a069daaa2
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR15MB1167
X-OriginatorOrg: fb.com
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2018-06-08_09:, , signatures=0
X-Proofpoint-Spam-Reason: safe
X-FB-Internal: Safe
Archived-At: <https://mailarchive.ietf.org/arch/msg/quic/2HQ-MVLEH9lyykP7O9GZMPEtoJ0>
X-BeenThere: quic@ietf.org
X-Mailman-Version: 2.1.26
Precedence: list
List-Id: Main mailing list of the IETF QUIC working group <quic.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/quic>, <mailto:quic-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/quic/>
List-Post: <mailto:quic@ietf.org>
List-Help: <mailto:quic-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/quic>, <mailto:quic-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 08 Jun 2018 20:49:45 -0000

    1a. buffer the entire frame, and then iterate though the instructions,
    while checking for every instruction that it does not go beyond the
    end of the frame
    1b. iterate though the instructions as they arrive, while checking
    that the entire instruction is available and that the instruction does
    not go beyond the end of the frame
    
    If we remove framing, the decoder can be implemented as:
    
    2. iterate through the instructions as they arrive, while checking
    that the entire instruction is available

Consider the most complicated instruction, which is Inserting a pure literal (all fields have variable length)

| Name Length | Name | Value Length | Value |

When you know the length of the instruction block in advance, you know that an incomplete instruction is a decoding error which you can check as you decode, and fail immediately on underflow.  Here, you either need to do a trial decode of the variable lengths to see if you have enough bytes, or check while decoding, and decide whether you want to save the state for later or discard it and reparse from the beginning of the instruction.  I'm not even sure reparsing is workable since it might open you up to doing N^2 work -- at least you need to be careful how you implement it.  It's worth calling out that discovering the length of a QPACK varint is more complicated than a QUIC varint.
    
It’s not impossible to solve of course.  “It’s just software” as Dmitri likes to say.  It’s just one piece of complexity I was hoping to avoid.

-Alan