IETF Logo Mail Archive

Re: [Tools-discuss] Advice on DT schema changes

"Salz, Rich" <rsalz@akamai.com> Sun, 17 November 2019 22:06 UTC

Return-Path: <rsalz@akamai.com>
X-Original-To: tools-discuss@ietfa.amsl.com
Delivered-To: tools-discuss@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id A3BEB12008C for <tools-discuss@ietfa.amsl.com>; Sun, 17 Nov 2019 14:06:00 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.7
X-Spam-Level:
X-Spam-Status: No, score=-2.7 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIMWL_WL_HIGH=-0.001, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=akamai.com
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id erVuPPHjCT_o for <tools-discuss@ietfa.amsl.com>; Sun, 17 Nov 2019 14:05:58 -0800 (PST)
Received: from mx0b-00190b01.pphosted.com (mx0b-00190b01.pphosted.com [IPv6:2620:100:9005:57f::1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id B26EA12006B for <tools-discuss@ietf.org>; Sun, 17 Nov 2019 14:05:58 -0800 (PST)
Received: from pps.filterd (m0122330.ppops.net [127.0.0.1]) by mx0b-00190b01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id xAHM4khw005181; Sun, 17 Nov 2019 22:05:54 GMT
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=akamai.com; h=from : to : cc : subject : date : message-id : references : in-reply-to : content-type : content-id : content-transfer-encoding : mime-version; s=jan2016.eng; bh=ThJQ0FfavGLnxOoWAmg+NW6sEuTDrIWtZzSnc+RsAow=; b=TahJLn/eV/KvfpQaC0F/U4uCRqTJYvtcNiaP8IJ3xYqG8+FdjOGPx45+9jRzIFzNvQpO 1O/o5AaNs6NnXn6K/Ry1cZGPfLp875iuokB/zWGSnkz5kz67YToZNAbItnpRi9Y4nobH Dw0fWY4NWuhkc/eHm+wcvh1KmZ1fWl4Xq+FgEmTQGBiHPrmXCabHKLLxAou9CBMeSh6a XXNoYBoipj7NzGxSqAMSNo4Fvn318A7Fr4cEjHCKGHSigounX8MuXuYDUYZmS1m9+5eo GffVRxM9kANsvoeG26WR5kCwsfIgs0SNnGAGdWrCJOAa1L3G79Y+lfjKW22S7iUUBtMa ag==
Received: from prod-mail-ppoint2 (prod-mail-ppoint2.akamai.com [184.51.33.19] (may be forged)) by mx0b-00190b01.pphosted.com with ESMTP id 2wag31d3m9-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Sun, 17 Nov 2019 22:05:54 +0000
Received: from pps.filterd (prod-mail-ppoint2.akamai.com [127.0.0.1]) by prod-mail-ppoint2.akamai.com (8.16.0.27/8.16.0.27) with SMTP id xAHM1tgj017683; Sun, 17 Nov 2019 17:05:53 -0500
Received: from email.msg.corp.akamai.com ([172.27.123.33]) by prod-mail-ppoint2.akamai.com with ESMTP id 2wadaxy592-1 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-SHA384 bits=256 verify=NOT); Sun, 17 Nov 2019 17:05:53 -0500
Received: from USMA1EX-DAG1MB3.msg.corp.akamai.com (172.27.123.103) by usma1ex-dag1mb4.msg.corp.akamai.com (172.27.123.104) with Microsoft SMTP Server (TLS) id 15.0.1473.3; Sun, 17 Nov 2019 17:05:53 -0500
Received: from USMA1EX-DAG1MB3.msg.corp.akamai.com ([172.27.123.103]) by usma1ex-dag1mb3.msg.corp.akamai.com ([172.27.123.103]) with mapi id 15.00.1473.005; Sun, 17 Nov 2019 17:05:52 -0500
From: "Salz, Rich" <rsalz@akamai.com>
To: Michael Richardson <mcr@sandelman.ca>
CC: Tools Team Discussion <tools-discuss@ietf.org>
Thread-Topic: [Tools-discuss] Advice on DT schema changes
Thread-Index: AQHVnRmR/xhGV4DYk0y2uc9ZEHzWy6ePxFeAgAAnzoA=
Date: Sun, 17 Nov 2019 22:05:52 +0000
Message-ID: <5CCFDA2C-3431-43AB-8CFA-44301A89F819@akamai.com>
References: <310B3662-BF5A-4943-9107-198A725EEEA3@akamai.com> <7326.1574001803@dooku.sandelman.ca>
In-Reply-To: <7326.1574001803@dooku.sandelman.ca>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
user-agent: Microsoft-MacOutlook/10.1f.0.191110
x-ms-exchange-messagesentrepresentingtype: 1
x-ms-exchange-transport-fromentityheader: Hosted
x-originating-ip: [172.19.216.83]
Content-Type: text/plain; charset="utf-8"
Content-ID: <286BC96DDF53A242B5F50899104CB9F0@akamai.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2019-11-17_05:, , signatures=0
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 malwarescore=0 phishscore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1911140001 definitions=main-1911170209
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.95,18.0.572 definitions=2019-11-17_05:2019-11-15,2019-11-17 signatures=0
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 priorityscore=1501 malwarescore=0 mlxlogscore=999 adultscore=0 mlxscore=0 bulkscore=0 lowpriorityscore=0 clxscore=1015 spamscore=0 phishscore=0 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-1910280000 definitions=main-1911170208
Archived-At: <https://mailarchive.ietf.org/arch/msg/tools-discuss/WcM3k0GUQHYvuVTuHV1145fPVWM>
Subject: Re: [Tools-discuss] Advice on DT schema changes
X-BeenThere: tools-discuss@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: IETF Tools Discussion <tools-discuss.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tools-discuss>, <mailto:tools-discuss-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tools-discuss/>
List-Post: <mailto:tools-discuss@ietf.org>
List-Help: <mailto:tools-discuss-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tools-discuss>, <mailto:tools-discuss-request@ietf.org?subject=subscribe>
X-List-Received-Date: Sun, 17 Nov 2019 22:06:00 -0000

>    I got the general gist of it, and it seems right, but it's certainly not in
    migration style.  I guess you are initializing things from ref_names: value
    side with a unique id saved into cache, but I didn't see where you used that.
  
What is migration style?  Where can I find "typical" examples to emulate?

The idea is that existing "additional URLs" fields -- for example, look at https://datatracker.ietf.org/wg/quic/about/ or https://datatracker.ietf.org/doc/draft-ietf-tls-subcerts/ -- will have those fields that are common (as specified in the python script) moved into the new tables. I am calling those new commonly-named things "extref" for "external referfence"  The value is just a string, but the idea is that it is an identifier that is useful in some external context, and having common names for these thing makes more automation possible.

For example, a "GitHub Repository" extref.  We want that name normalized, so that tooling can fetch it. Right now the additional URL's are a set of text fields, and the names are not consistent: "Github repo" "GitHub Repository" "Repository", etc., are all used.  That is the point of the ref_names map, to capture all the existing names via regexp and map them into a common name which gets an ID in the external_refs table, which is then used as foreign keys in the "list of external references" for each of the person, group, doc, entries. It will not be perfect, but I looked at all the names and in use (the tables mentioned in the python outline) and I think it gets almost all things right.  I am unsure, for example, of the YANG links; what tooling of theirs breaks if the name becomes generic?  Perhaps I just don't remove a link when migrating.

What is not shown or done is mapping these extrefs into the source, such as ietf/doc/models.py, and displaying them. That's the real work, I just want commentary on the data structures first.  For example, in https://datatracker.ietf.org/person/rsalz@akamai.com I could add a "standard" extref that says "On github I am richsalz"  Then this could be used by scripts to populate repo's and auto-link back to the DT accounts.

Hope that helps you help me help the IETF. :)

v2.23.0 | Report a Bug | By Email