[Tools-discuss] Re: [art] Re: Exploring ABNF extracts from RFCs

Paul Kyzivat <pkyzivat@alum.mit.edu> Mon, 29 July 2024 13:58 UTC

Return-Path: <pkyzivat@alum.mit.edu>
X-Original-To: tools-discuss@ietfa.amsl.com
Delivered-To: tools-discuss@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 3309AC1840EE; Mon, 29 Jul 2024 06:58:51 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.01
X-Spam-Level:
X-Spam-Status: No, score=-2.01 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_BLOCKED=0.001, RCVD_IN_MSPIKE_H2=-0.001, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=alum.mit.edu
Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id bfMG1Z8CQBFC; Mon, 29 Jul 2024 06:58:50 -0700 (PDT)
Received: from NAM11-CO1-obe.outbound.protection.outlook.com (mail-co1nam11on2127.outbound.protection.outlook.com [40.107.220.127]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 16AB4C1CAE67; Mon, 29 Jul 2024 06:58:38 -0700 (PDT)
ARC-Seal: i=1; a=rsa-sha256; s=arcselector10001; d=microsoft.com; cv=none; b=swKzwpkkNh/8of8ugBjJkdH9qnzeecm3rxjdiAfu44DD/ICgeOfwxHozcNlOrb60D+ag6NFzTA9W7fAR+uWFR7ZHjVfubF5xno4u2Ku1kz1wmmtxAkQJtT0L4X8ouQsMbF2aTqYWaa9i357Z2yvCjaGG8XGYXPjN7mjuT+J0Bi42S9wYrOc3kv0RfmCKeW4ci7VET3ZptQqVmRdu7x3YtPW2DtiT8n652YlefqT9b5GExpUbciRNONNv3fFb2XrSvRYo8N36COYWVq2WKrmANstgU3udQL+ydcFz4w4zYcaGEVjhBH0R1sLIlVXHBv6h+jpUMc+SvLbPMtmrOrlW3w==
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector10001; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-AntiSpam-MessageData-ChunkCount:X-MS-Exchange-AntiSpam-MessageData-0:X-MS-Exchange-AntiSpam-MessageData-1; bh=XLvYNSZWXha2k4Q7lbZXTZZL8xU/XBmO0b/JrPF7RHw=; b=NOWbjm87JTd4H5vcisWGVU4Q8YFjAKIYPAiJaq0o5pVW1/jdwU8Agp6Zw+bqjXnrmrbEU513Nd10fuJyjrZHctZPdXMGYTE1lKsH1wGmi/vlLObbT3k8e4FJ8iyyjNc4uxGi3vbquqcYz5prYEkMQ8o5QNcfrDVF5TiS7siasBfSeXyYPhS5tEo6tCkPlgA/X6qGKiiPg38iVq/xB0zzI4+cmcmJWNIKhzNvutRfrIsc9kfZIIQp0Yb0bfXnoeySztSZiadlE+lU8oDGcHVZCEXzPzIFOhXxnGDyzY1i/c4/KunJrI8yfMc+gDO66EnJvXRbj0QDIly195h5QfWewQ==
ARC-Authentication-Results: i=1; mx.microsoft.com 1; spf=pass (sender ip is 18.7.68.33) smtp.rcpttodomain=ietf.org smtp.mailfrom=alum.mit.edu; dmarc=pass (p=none sp=none pct=100) action=none header.from=alum.mit.edu; dkim=none (message not signed); arc=none (0)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=alum.mit.edu; s=selector2; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=XLvYNSZWXha2k4Q7lbZXTZZL8xU/XBmO0b/JrPF7RHw=; b=g/u5kKEI+WSyUy31YDeFLepf1OEhIKyRTxvc7aVGFUJr81ZsnYR4gcrV4bRqp354VAFkYa3MbrK9by4LXyH356oW6+vgFgwrG3L0CaT5NijIkdbhr6CyojKX0QtJQHF8fQjdFYgDubEJdNe2TRDtiNz4sJeKydz+EcbyhjeOhKw=
Received: from PH0PR07CA0055.namprd07.prod.outlook.com (2603:10b6:510:e::30) by MW4PR12MB5644.namprd12.prod.outlook.com (2603:10b6:303:189::15) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7807.27; Mon, 29 Jul 2024 13:58:37 +0000
Received: from SA2PEPF000015CB.namprd03.prod.outlook.com (2603:10b6:510:e:cafe::4c) by PH0PR07CA0055.outlook.office365.com (2603:10b6:510:e::30) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7784.35 via Frontend Transport; Mon, 29 Jul 2024 13:58:37 +0000
X-MS-Exchange-Authentication-Results: spf=pass (sender IP is 18.7.68.33) smtp.mailfrom=alum.mit.edu; dkim=none (message not signed) header.d=none;dmarc=pass action=none header.from=alum.mit.edu;
Received-SPF: Pass (protection.outlook.com: domain of alum.mit.edu designates 18.7.68.33 as permitted sender) receiver=protection.outlook.com; client-ip=18.7.68.33; helo=outgoing-alum.mit.edu; pr=C
Received: from outgoing-alum.mit.edu (18.7.68.33) by SA2PEPF000015CB.mail.protection.outlook.com (10.167.241.201) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.20.7828.19 via Frontend Transport; Mon, 29 Jul 2024 13:58:35 +0000
Received: from [192.168.1.52] (c-76-19-71-248.hsd1.ma.comcast.net [76.19.71.248]) (authenticated bits=0) (User authenticated as pkyzivat@ALUM.MIT.EDU) by outgoing-alum.mit.edu (8.14.7/8.12.4) with ESMTP id 46TDwVU0016637 (version=TLSv1/SSLv3 cipher=AES128-GCM-SHA256 bits=128 verify=NOT); Mon, 29 Jul 2024 09:58:32 -0400
Message-ID: <b864092b-477b-4866-a2b1-481313545917@alum.mit.edu>
Date: Mon, 29 Jul 2024 09:58:31 -0400
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
From: Paul Kyzivat <pkyzivat@alum.mit.edu>
To: Dominique Hazael-Massieux <dom@w3.org>, Carsten Bormann <cabo@tzi.org>, art@ietf.org
References: <89a4a566-8ffe-413e-9196-3f08bebe8d20@w3.org> <99E8B992-AF60-4CD6-9786-2EC180E95E4D@tzi.org> <2867abac-8c62-41b2-a20c-cb9fe8d3736d@alum.mit.edu> <ff29ebe0-efbc-4bfc-8c79-f09ccbe3ffe3@w3.org>
Content-Language: en-US
In-Reply-To: <ff29ebe0-efbc-4bfc-8c79-f09ccbe3ffe3@w3.org>
Content-Type: text/plain; charset="UTF-8"; format="flowed"
Content-Transfer-Encoding: 8bit
X-EOPAttributedMessage: 0
X-MS-PublicTrafficType: Email
X-MS-TrafficTypeDiagnostic: SA2PEPF000015CB:EE_|MW4PR12MB5644:EE_
X-MS-Office365-Filtering-Correlation-Id: 1549d5cd-ccb0-4ef7-4094-08dcafd68a50
X-MS-Exchange-SenderADCheck: 1
X-MS-Exchange-AntiSpam-Relay: 0
X-Microsoft-Antispam: BCL:0;ARA:13230040|36860700013|82310400026|376014|1800799024|41320700013;
X-Microsoft-Antispam-Message-Info: gR6YBjBvY1x5Br5a9Mk/lMrjrVTXZvRyw9VNii5+b79uPmHlxUNqzl05LGboEEN+tehQ8yBnij16BEe5t5zLNsc36N7tIDIn8Wmg4rPwaS8LVJ0gzncgF/FifjUtQRIYPoQWo7QpmHzM0ab+URcqhHUxHAsqcDcpd+3X5Ob9YTYr/ic8+M0G0lcwu4Bww4zATlkuE/C2k2TEP9Jhl4eiA7TaycK8wBcXqOy6c6aAg6+Bt6vhGs+K+TMPhtaWs+y41fJbEkkCQGQS8or+gJp54Iupzi+R8lvmqvWwwaAXdRS8lKkFvdiTqI+PnM6D604ulcuwh2iAgJgwe4xE2scB6dVn3CRCS3DNkTWc1Kk+cCU8aV05i7Zw4T6MzkO//52nYQzP3K2GGhEkBFXh90fA/iq/GiKNcq3Tj8PSBGyZZ8+4fWCvkoqMIEX5S3WYHbGs4KnXhGBFxfuaI2RDRnWNo5mEt9luSTXFUCMCh2lczZNpHwQPkyf3TplOTqpuZzv8cygQuaaEM4v+V8ubmIJNlREjGBqO1jK+09xFv1KoWA0qW/vkOe8ZyDkGacssOvO3O0c2z8Uvj5NC9zqLWbg8ojQYILosQuQyGGkvEnlWbd+/qoAFvoDJl5EtuH5x9kBpv8o/9K2kfnNBVW51MONWwvOjpXR1kVExFO04Nk0paW/KkhjbKtcCYbgC7tdkYp1FeSTcVvvyloY/qBeAW8JCP8Rsdb6oTPNNZu7+ka0UO2eiQJ3ZnPFdVj+u4RLUVQTiYDqMJq363lKkgmu1GS0LPLdeK+SRoZsugOV3dJB6IC9VNk9MBHLn2qiAKLT4039Kyc+zjLtXnlhiiXFYnkrzrSH1jY4PxNxNlTz1onMrUooaHQ4QeF3T0RDZx+PTMpqNrl4ayc/Ois3yorIX7VV4knF8b0e0hnQrdttYC0DbTO0zsdDHlQOgIkWctPRIUWZSzFU3JDbpRO6aT2VBXWEjcfWq5dNFf2qPwEtA11M7bNJ0rwllf01bD5RxVoTCh1pcGmPReBl8AzSuJuEio9XuUg37ShpQJGhYvy2/ker7AqgCyg+4+C7aID5D6cbIG4xFLg1E5E1IBGzlczWwXXHxXnjgNXiVjEHE/msiQWcvRqIfj7/3/MqJF6RFXbvGOD4vUQavJVIMW3tylXO6CsYHnejOe02klvYSFG0gFE8pN923Ve8evVPVsQqHcrIRpTq/yU1FWZj9gmDLwLmPLzuZoFgfGqdGQABj7JpIIFky3e6xHrtrUdrGZPgU7NYd2RkSdnXgI67digu+bBLdw1B2DDSj8gFII4Yo5lFq9o3/rzucHh8BSAuq0zP5dxLKJjfS6QkjbH9n+ng23mBlf62qhvD3Fa4vPgn1t2J1Brn/NqJ7fgHzeohguIeyRQUE4373WtELPomOJMoQeCPuGYmb0SmuXKPrCKiW5kPMPJV1zKE3vsmVwpZ+EjqokWYKvhCNoBMz0ApsOuBu5QwBQN3IeQ==
X-Forefront-Antispam-Report: CIP:18.7.68.33;CTRY:US;LANG:en;SCL:1;SRV:;IPV:CAL;SFV:NSPM;H:outgoing-alum.mit.edu;PTR:outgoing-alum.mit.edu;CAT:NONE;SFS:(13230040)(36860700013)(82310400026)(376014)(1800799024)(41320700013);DIR:OUT;SFP:1102;
X-OriginatorOrg: alum.mit.edu
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Jul 2024 13:58:35.4707 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 1549d5cd-ccb0-4ef7-4094-08dcafd68a50
X-MS-Exchange-CrossTenant-Id: 3326b102-c043-408b-a990-b89e477d582f
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3326b102-c043-408b-a990-b89e477d582f;Ip=[18.7.68.33];Helo=[outgoing-alum.mit.edu]
X-MS-Exchange-CrossTenant-AuthSource: SA2PEPF000015CB.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR12MB5644
Message-ID-Hash: GGGI5FXTSG54FVJD33EALV4W2RR6Z4UI
X-Message-ID-Hash: GGGI5FXTSG54FVJD33EALV4W2RR6Z4UI
X-MailFrom: pkyzivat@alum.mit.edu
X-Mailman-Rule-Misses: dmarc-mitigation; no-senders; approved; emergency; loop; banned-address; member-moderation; header-match-tools-discuss.ietf.org-0; nonmember-moderation; administrivia; implicit-dest; max-recipients; max-size; news-moderation; no-subject; digests; suspicious-header
CC: tools-discuss <tools-discuss@ietf.org>, CBOR <cbor@ietf.org>, abnf-discuss@ietf.org
X-Mailman-Version: 3.3.9rc4
Precedence: list
Subject: [Tools-discuss] Re: [art] Re: Exploring ABNF extracts from RFCs
List-Id: IETF Tools Discussion <tools-discuss.ietf.org>
Archived-At: <https://mailarchive.ietf.org/arch/msg/tools-discuss/1xBbfCFnsDhWf3VxW3zAFLyMRO0>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tools-discuss>
List-Help: <mailto:tools-discuss-request@ietf.org?subject=help>
List-Owner: <mailto:tools-discuss-owner@ietf.org>
List-Post: <mailto:tools-discuss@ietf.org>
List-Subscribe: <mailto:tools-discuss-join@ietf.org>
List-Unsubscribe: <mailto:tools-discuss-leave@ietf.org>

[This message apparently failed to be sent, so I'm trying again.]

On 7/26/24 12:49 PM, Dominique Hazael-Massieux wrote:
> Thanks Carsten for the pointers to the CDDL work and the existing 
> extractions you've built!
> 
> Reactions of my own to Paul's questions based on my automated ABNF 
> consolidation experience:
> 
> Le 26/07/2024 à 18:09, Paul Kyzivat a écrit :
>> There are interesting open issues around abnf dependencies:
>>
>> * If I want to use a rule defined in another document, do I add to my 
>> abnf *all* of the foreign abnf that contains the rule definitions I 
>> need? Or do I only import the specific rules I need and any other 
>> rules those depend on?
> 
> The consolidater takes the approach of importing only the specific rules 
> needed and their dependencies.

If we had namespaces this wouldn't be necessary. But for your work this 
seems the practical way.

I would like to see namespaces added to abnf. But will only help in the 
future. We are still stuck with all the usage enshrined in RFCs. So the 
hacks will persist.

...

>> - Stating the dependency in an abnf prose-val. E.g.,
>>      foo = <foo as defined in RFCnnn>
> 
> The consolidater tool as it tries to determine whether a given ABNF can 
> be consolidated identifies not only rules that are undefined, but also 
> rules that are defined only as prose as indeed they often contain 
> references to other RFCs.

While many do, they still do so in ad hoc ways. Have you found a way to 
automate this, or do you use a manual assist?

	Thanks,
	Paul