Re: [Cbor] cddl 0.8.17: Add .abnf (draft-ietf-cbor-cddl-control)

Paul Kyzivat <pkyzivat@alum.mit.edu> Sat, 27 February 2021 15:50 UTC

Return-Path: <pkyzivat@alum.mit.edu>
X-Original-To: cbor@ietfa.amsl.com
Delivered-To: cbor@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A40363A0BF0 for <cbor@ietfa.amsl.com>; Sat, 27 Feb 2021 07:50:19 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.003
X-Spam-Level:
X-Spam-Status: No, score=-2.003 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, NICE_REPLY_A=-0.001, RCVD_IN_MSPIKE_H2=-0.001, SPF_PASS=-0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=alum.mit.edu
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id S7OEiAZkeKOw for <cbor@ietfa.amsl.com>; Sat, 27 Feb 2021 07:50:18 -0800 (PST)
Received: from NAM12-MW2-obe.outbound.protection.outlook.com (mail-mw2nam12on2049.outbound.protection.outlook.com [40.107.244.49]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id CB0FF3A0BEF for <cbor@ietf.org>; Sat, 27 Feb 2021 07:50:17 -0800 (PST)
ARC-Seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=Suo+uY37aRxtHpVFPWYI9S8NAeaTDGgxLJajbmavrQGcIOnadNy4EBXeKVlwZjCxAqdyXwLYOTqfKdRBzI2x16oLrNeVsOgZptUeHdw//0hRGXQQR7Tm2mC52dQIKeiIaxHPuzzAcvM3PlFW2o2zUpeTimfXwy2WSdaHH0I/aCk0ENhhmomnUyvIPaibO4B9hTVwV3HCo4eD837unbXkQ9JBaCkzOWvaOf5yWM/+d+qm+kXzj1UnwrBGW6PnF0eeJ/MOoFzQUfIb8nsTu0CUh9No0Shv2Xdi7yVOEcmAxj95ip43hL7Bq3pgYbucsSg38bQDXIHjzXSxlR96+k9+uw==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=y+hNF6Umw6TdZBs9RlxRIGinIDne9J0BpjiJYmZJNpU=; b=PE7NW/hXlacYCLZ5yxQlKcdN2S5b056BwgyF2tg7Ue/3X34f7dFNZ3L5xKoDbSiYuR4uwCO9xfXqrNxFCBbNOgXQDYrs+jjKlySPSrkOXLoNAjG2NqJrRPObro9DJarYA3BRApzZKifqUQi3sX0nHdBx/xJDP0jMOU3MspNFVRFZHcV80xtoFXONzH1TSaOGWK4XoFmC/VKhgP85Jr4S8vjRvvg9mtR27QQTwsR9ogCTb/SsBUwwFmLB3DifIoJgmJ0BvIa9sqGN6uBrtSFDuzubPA2sYc5nWeY/wrawYa0LrRqi7VRCm66oNjPkcfqRAeXKsK5K2/U0WQ8dXUHt6w==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 18.7.68.33) smtp.rcpttodomain=tzi.org smtp.mailfrom=alum.mit.edu; dmarc=bestguesspass action=none header.from=alum.mit.edu; dkim=none (message not signed); arc=none
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=alum.mit.edu; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=y+hNF6Umw6TdZBs9RlxRIGinIDne9J0BpjiJYmZJNpU=; b=V/PjiKg4SrChRWBwrlZ9+SWFcnUZnCkaChs2i2cGj0Ccjrbzzqba29DjB6NmH4bqRNGUvX2ab4j6lcVfo5Pq+qA8Gid+NotEZ1+11266vuISgCnVmEo3loaNuUEMMhptuCZ0Xjls+WNy0rW3lAvGgWqccJbkD5Y79OFROqI16RQ=
Received: from SN4PR0501CA0050.namprd05.prod.outlook.com (2603:10b6:803:41::27) by DM5PR12MB4662.namprd12.prod.outlook.com (2603:10b6:4:a6::13) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3890.20; Sat, 27 Feb 2021 15:50:15 +0000
Received: from SN1NAM02FT044.eop-nam02.prod.protection.outlook.com (2603:10b6:803:41:cafe::35) by SN4PR0501CA0050.outlook.office365.com (2603:10b6:803:41::27) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3912.9 via Frontend Transport; Sat, 27 Feb 2021 15:50:15 +0000
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 18.7.68.33) smtp.mailfrom=alum.mit.edu; tzi.org; dkim=none (message not signed) header.d=none;tzi.org; dmarc=bestguesspass action=none header.from=alum.mit.edu;
Received-SPF: Pass (protection.outlook.com: domain of alum.mit.edu designates 18.7.68.33 as permitted sender) receiver=protection.outlook.com; client-ip=18.7.68.33; helo=outgoing-alum.mit.edu;
Received: from outgoing-alum.mit.edu (18.7.68.33) by SN1NAM02FT044.mail.protection.outlook.com (10.152.72.173) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.3890.19 via Frontend Transport; Sat, 27 Feb 2021 15:50:14 +0000
Received: from MacBook-Air.localdomain (c-24-62-227-142.hsd1.ma.comcast.net [24.62.227.142]) (authenticated bits=0) (User authenticated as pkyzivat@ALUM.MIT.EDU) by outgoing-alum.mit.edu (8.14.7/8.12.4) with ESMTP id 11RFoCuZ011247 (version=TLSv1/SSLv3 cipher=AES128-GCM-SHA256 bits=128 verify=NOT); Sat, 27 Feb 2021 10:50:12 -0500
To: Carsten Bormann <cabo@tzi.org>
Cc: cbor@ietf.org
References: <AC771EE9-9672-4B2D-B66A-2C815D102687@tzi.org> <358dd1e3-74a5-c32e-235a-c0a6c308ae6f@alum.mit.edu> <CD82E5A7-4D46-412F-B45F-B4970C2FF8E8@tzi.org> <3a80f922-cfc9-2831-c4f0-7c918e211d74@alum.mit.edu> <8B7422CF-6DE1-4ACB-B6A8-BE0B466BB072@tzi.org> <DDB153F6-B2AA-4D12-BB1C-9C2D90AD6DA3@tzi.org>
From: Paul Kyzivat <pkyzivat@alum.mit.edu>
Message-ID: <56cea100-d3b2-148e-3701-5587672db481@alum.mit.edu>
Date: Sat, 27 Feb 2021 10:50:12 -0500
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.16; rv:78.0) Gecko/20100101 Thunderbird/78.7.1
MIME-Version: 1.0
In-Reply-To: <DDB153F6-B2AA-4D12-BB1C-9C2D90AD6DA3@tzi.org>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 8bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-Office365-Filtering-Correlation-Id: fb368c6d-2437-45f4-f02e-08d8db375f6a
X-MS-TrafficTypeDiagnostic: DM5PR12MB4662:
X-Microsoft-Antispam-PRVS: <DM5PR12MB46622D944766D75D9E32CCD5F99C9@DM5PR12MB4662.namprd12.prod.outlook.com>
X-MS-Oob-TLC-OOBClassifiers: OLM:9508;
X-MS-Exchange-SenderADCheck: 1
X-Microsoft-Antispam: BCL:0;
X-Microsoft-Antispam-Message-Info: quJ8HZw8VNoSk8d+nyl6q9MDV3omqfoJTFOtm5d8xjjUS+SMWufSNMyTAxDeHWNwv1IpbuBNcPiRb+r7IXaZRe60lZZcEauWVGoverRwNlmM3NB4lX4pY733fBr4cIVN/zv1NmpZjwIE9/7Cyvv7ZkplTQpTETHBzQ1I/OQq/qpx9oUwYTw8X5lPpwXg4NFVadcJVa01+q3K+Pl18A6a2lbU+OJlEoliwcgGe1aR8J7y6Es55WoPH/KW+vY7Qu3t8cXnghNQ3gaOBMvaTGpcXi1tResdvZXPX0nE5oSN6/pz6ehhHBzxess6Vl96b5TYfJQbskx0A2HqGKdmUw0jy2Wk7GhbjfObUQKEf0BZxDQ+G4CLLn+DF+yz8mUcc1sa0Kx9ju27MnNcn/z76/CQpo+axw2JCWBq9Vl01wyKUWonmH5jgHFhG5pSqtwBVXkrN/YdqXi3YrKy/YDCQ7AN0LJUdEqGY1DWbL4Os3MIznChYUJsgsct9vhjQTnnFWAJbGsBAxWH9Yw8RU8Kv+Kb96p2hhWWV/Fd3fLceBfl/RMDVf25xNHJZTePU1csWhDdHfdVKz//VVgNuyCBcx4lq9OkP+P3svksbInpaFUmnUWBcTl0l8+jnSctlXI41UCf7VpNbpmMRq2zK9EjTklYhdYsR0TQ/9FN4Fb4ihshw/29fz6R1U90S6lDXMNrwJMO0WAk5p79Pd3D6th0iES7Aw==
X-Forefront-Antispam-Report: CIP:18.7.68.33; CTRY:US; LANG:en; SCL:1; SRV:; IPV:CAL; SFV:NSPM; H:outgoing-alum.mit.edu; PTR:outgoing-alum.mit.edu; CAT:NONE; SFS:(136003)(346002)(396003)(39860400002)(376002)(46966006)(36840700001)(66574015)(336012)(356005)(2906002)(186003)(7596003)(4326008)(26005)(31696002)(83380400001)(47076005)(82310400003)(5660300002)(82740400003)(478600001)(31686004)(53546011)(6916009)(70206006)(2616005)(316002)(786003)(8676002)(75432002)(36906005)(70586007)(956004)(86362001)(36860700001)(8936002)(142923001)(43740500002); DIR:OUT; SFP:1101;
X-OriginatorOrg: alum.mit.edu
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Feb 2021 15:50:14.1464 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: fb368c6d-2437-45f4-f02e-08d8db375f6a
X-MS-Exchange-CrossTenant-Id: 3326b102-c043-408b-a990-b89e477d582f
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3326b102-c043-408b-a990-b89e477d582f; Ip=[18.7.68.33]; Helo=[outgoing-alum.mit.edu]
X-MS-Exchange-CrossTenant-AuthSource: SN1NAM02FT044.eop-nam02.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR12MB4662
Archived-At: <https://mailarchive.ietf.org/arch/msg/cbor/ZMPugbccTN_xZtt3pUhtV30DwoM>
Subject: Re: [Cbor] cddl 0.8.17: Add .abnf (draft-ietf-cbor-cddl-control)
X-BeenThere: cbor@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Concise Binary Object Representation \(CBOR\)" <cbor.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/cbor>, <mailto:cbor-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/cbor/>
List-Post: <mailto:cbor@ietf.org>
List-Help: <mailto:cbor-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/cbor>, <mailto:cbor-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sat, 27 Feb 2021 15:50:20 -0000

Fixing the indentation problem by removing common leading whitespace 
breaks if both spaces and HT are used in varying combinations to achieve 
the indentation. You can try to fix that by transforming HT into spaces 
using the algorithm of your choice, but it fails if your algorithm 
differs from that used by the author of the abnf.

That argues for "fixing" abnf itself. But I agree that won't happen 
quickly if at all.

	Thanks,
	Paul

On 2/27/21 8:36 AM, Carsten Bormann wrote:
> So RFC 5234 has this indentation problem:
> 
> oid = bytes .abnfb ("oid" .cat cbor-tags-oid)
> 
> cbor-tags-oid = '
>    oid = 1*arc
>    roid = *arc
>    arc = [nlsb] %x00-7f
>    nlsb = %x81-ff *%x80-ff
> '
> 
> …does not work, as the ABNF is indented, which is not allowed by RFC 5234.
> Outdenting the ABNF works, but reduces readability (it also is too easy to confuse  ABNF with CDDL rules!).
> 
> So how do we make something like the above work?
> 
> (1) We could fix ABNF.  This is an Internet Standard, so I’d expect this to be a slow process; we want to be done with the new controls earlier than that.
> (2) We could make CDDL work with ABNF as is, but still allow something like the above.
> 
> Now how can we do (2):
> 
> (2a) We define .abnf/.abnfb to ignore leading whitespace.  This requires some elaborate rules like the ones bap uses (see previous mail).  The main problem with this is that if the ABNF is concatenated together from different sources, these must agree on the amount of leading whitespace for this to work.  And this is rather likely as 5234 and other places define “libraries” that need to combined with ABNF from protocol spec RFCs.
> 
> (2b) We leave .abnf/.abnfb as is, and define an outdenting version of .cat.
> Let’s call that .bat for the moment.
> 
> Fix the above example:
> oid = bytes .abnfb ("oid" .bat cbor-tags-oid)
> Done.
> 
> More precisely, .bat takes its controller (right hand side argument), computes the indent (leading whitespace of least-indented non-blank line), and reduces the amount of leading whitespace on each line by the indent.
> 
> This is only done on the right hand side as it fits the most common use cases.  It also allows keeping some leading whitespace for the target (left hand side argument).  If really both sides need to be outdented, use something like
> 
> ant<v1, v2> = ("" .bat v1) .bat v2
> 
> As the outdenting is done before concatenation, .bat has no problems dealing with pieces that are indented by different amounts.
> 
> I have little problem with introducing this control operator, as there are features in other languages that have similar outdenting semantics, e.g., <<~ in Ruby.
> 
> CDDL tool 0.8.20 has .bat, if you want to play with that.
> 
> The retronym generation process for .bat is now officially started (or maybe you have a different name for this that also is short and to the point; .cat-with-undenting-of-right-hand-side is not it).
> 
> Grüße, Carsten
>