Re: [Tools-discuss] [rfc-i] missing line ends in generated TXT (from datatracker)

Julian Reschke <julian.reschke@gmx.de> Mon, 04 November 2019 18:20 UTC

Return-Path: <julian.reschke@gmx.de>
X-Original-To: tools-discuss@ietfa.amsl.com
Delivered-To: tools-discuss@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id AD07B120255 for <tools-discuss@ietfa.amsl.com>; Mon, 4 Nov 2019 10:20:06 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.898
X-Spam-Level:
X-Spam-Status: No, score=-1.898 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=gmx.net
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Gj8nWX-sCwrt for <tools-discuss@ietfa.amsl.com>; Mon, 4 Nov 2019 10:20:03 -0800 (PST)
Received: from mout.gmx.net (mout.gmx.net [212.227.15.18]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id D9E0D120120 for <tools-discuss@ietf.org>; Mon, 4 Nov 2019 10:20:02 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1572891535; bh=58vYvuDCMboxdq0Z8c/MdB5Li//abqzBC2MVE5jCCG8=; h=X-UI-Sender-Class:Subject:From:To:Cc:References:Date:In-Reply-To; b=YSdWmPXASkOyBkfrPA3lUgo0FR5kclVK41tEZYjrtlo2+mQIfbQ1mqCpx15aQhOpx gLGdAzCVj7zIQ4Jba6LrRFxIXfJ5VDVxB5Dsdd+C/zYiNcLL0pisEFy/QRda3j8LoW mwbDnwfXwL4XDmuNLTtXor/CAPIkpfM+hTi+JPxg=
X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c
Received: from [192.168.178.124] ([84.171.149.92]) by mail.gmx.com (mrgmx005 [212.227.17.190]) with ESMTPSA (Nemesis) id 1Mqb1W-1i5oSt27hw-00md8x; Mon, 04 Nov 2019 19:18:55 +0100
From: Julian Reschke <julian.reschke@gmx.de>
To: Dave Rice <dave@dericed.com>, Henrik Levkowetz <henrik@levkowetz.com>
Cc: rfc-interest <rfc-interest@rfc-editor.org>, tools-discuss@ietf.org
References: <20285.1568652805@localhost> <705AA16B-68B1-4B5E-AE47-0714F6781C1A@dericed.com> <26105.1568818137@localhost> <E70CD0FF-0294-47E5-873F-5E184A248577@dericed.com> <1268.1568828850@localhost> <B4047303-E2B9-4B9D-B368-2ACC85A06D15@dericed.com> <15814.1569013442@localhost> <3a8f5865-9f5c-8c65-63be-bca31c5f1c86@levkowetz.com> <3650B5EF-A442-4291-928A-2FCB2144134C@dericed.com> <06122b6d-d5f9-30b9-2db4-a2427d299a1f@levkowetz.com> <CB32A50B-D6F2-41C4-8AC7-750E35F4941D@dericed.com> <97e68838-ac0c-25e4-67a9-a2a998b6e8af@gmx.de> <3ef6a53c-e18a-9fc7-8324-67f388acd32f@gmx.de>
Message-ID: <60a9d29f-03ef-5842-0c21-6930293759da@gmx.de>
Date: Mon, 4 Nov 2019 19:18:51 +0100
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:60.0) Gecko/20100101 Thunderbird/60.9.0
MIME-Version: 1.0
In-Reply-To: <3ef6a53c-e18a-9fc7-8324-67f388acd32f@gmx.de>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: quoted-printable
X-Provags-ID: V03:K1:2bWdI3XzXGHF5Bk7lT5J2VuaMwV+MCGCh+7gnXbWOB5HzFt52Xf VUyAaHivLA3HRxzk4HCm2TqxfZtNLmshllY4+DZUsMW7rnnjNRnD9VYFzB7ZMPKh2+lsxrC M9COGc90Tw1zmprmgwGTCEbEqD/Jl3Mse8Rtn5uJf4K4gPVBtqG0qqvcyOeRu94ZCN8oRb3 7tR3HDHjSHmNCNcRKMCnQ==
X-UI-Out-Filterresults: notjunk:1;V03:K0:5PgecwWHAWY=:DKo3+CTE9ljttCXKr9ODxp T3G3u5KlCJxwIllW6u+l9OIzAR5/iacRC7tKI4b2tnqiPgMEpUg6CFOw4Ot3gUwlphNXNNIXw QQfQlNmYCU6EdiGcSyNPlkkNbKw48lY6/MgTz6KCx1Ze60x+qfAXdfOwvl/zUP11LzliCTKT2 ZYzzazhnvuTzc2SdYwQuYNx4Llswv1sy3Do9FCNOwelISApZMOjxDdh93t+qNwVQ4aYLIpLD4 +QhT2qg/CjwFNEkO/lcWsACbX91moxfMePl5wljxd0ft5dKwxgtdHWD0UKSpSRgBpVNqPaEKr /LeRkHaDuChSjtKJp+aVdMR1dvBD7jICX/5DrnKNdgM/alcVzPfsxxHjgRXywN9aqZw0Qq4JK vk4FOeEGX0FMc2dsdO0UWdCoaxS/F4ujbcQvo6UbDhQz1Gxkv3TOWvV34COwDrOdzuDJvzF25 8mIqjjQb897AXs+pw/h1BP3uPfm+RX0yAidb645lF/ZZJH5bblVRoEGRZ7vutYjMxQs28didj y5ueLCMd0PGwQr5gHUwB0foBSSxSvxKQgA7RHveQpPxlwFDd6zXmqkI2qRzDdwGZhd1P1f34T PJg4aHDtbF/S4bcF+Z4q1jbwpoLhNKGRX0xgKMffRzpIiIkCyC6K/k5CbkvUELovGOJkyRJpa B7noIWWX0LgLISaW854Nqg/mTJf5fAj2OnkJBk1DjaoLmmzPdoGdbxsjLoRf6HswIotkEEhpl zSKcoS9Iiv+f44OKudCemhfJagPHezOm93K+Anwih019MNr6DDQSq2WU/g5j6l2jDAYk2WmtL NoAGPDpFetkPtyGxi2gEsSnUeF3+8Gym9DLqCEyh8e8LJ2w3EmRXgeC39klytnY9RXePExiWL K6l8m2iER6dYkw//uLDafW7tRs6iC5sLg3DhoNbJmjifJCzvDVhGxYkkjNfcIpEqFW+RrS+SM feYemUD67kVTkD1TQPqf9iTkarK0ZKE/0CJjWSeJxk6tovMUgw39/nGNFXJ69nBg3J0arjvfr jxAWzyJdnOSAakcMX68BON2QSILjt0DQe/7u7jgJvAn51ELMsDgonT/UzV9ullHpC3zuPy/Qu YPV8M1OmBE3LKYqXRwra97wQIqO1ULrlJhyNKaRm93R397h2CLYVpLGFg5fIi/C5RJrpdw2+X N7LrwTAaAQp45hovg1Hi2HTJXHQ8QhMeF0Oqc4kX0q6U621wQPrnc3dILo6uIJ8Bgnwwr5hyN d1ErFu3yNQICO/OYo/B5OEpUrRl9L3vKpTfxR8lkZisZDAKXGXghrn2ePqLg=
Archived-At: <https://mailarchive.ietf.org/arch/msg/tools-discuss/XCdMqx_ADVg8FEsbGKc90LHdwVs>
Subject: Re: [Tools-discuss] [rfc-i] missing line ends in generated TXT (from datatracker)
X-BeenThere: tools-discuss@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: IETF Tools Discussion <tools-discuss.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tools-discuss>, <mailto:tools-discuss-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tools-discuss/>
List-Post: <mailto:tools-discuss@ietf.org>
List-Help: <mailto:tools-discuss-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tools-discuss>, <mailto:tools-discuss-request@ietf.org?subject=subscribe>
X-List-Received-Date: Mon, 04 Nov 2019 18:20:07 -0000

On 25.10.2019 08:16, Julian Reschke wrote:
> On 21.10.2019 18:46, Julian Reschke wrote:
>> On 21.10.2019 18:14, Dave Rice wrote:
>>> Hi all,
>>>
>>>> On Oct 9, 2019, at 1:38 PM, Henrik Levkowetz <henrik@levkowetz.com
>>>> <mailto:henrik@levkowetz.com>> wrote:
>>>>
>>>> Hi David,
>>>>
>>>> On 2019-10-09 17:32, Dave Rice wrote:
>>>>> Thanks Henrik,
>>>>>
>>>>>> On Sep 20, 2019, at 5:26 PM, Henrik Levkowetz <henrik@levkowetz.com
>>>>>> <mailto:henrik@levkowetz.com>> wrote:
>>>>>>
>>>>>> Hi Michael, David,
>>>>>>
>>>>>> On 2019-09-20 23:04, Michael Richardson wrote:
>>>>>>>
>>>>>>> Dave Rice <dave@dericed.com <mailto:dave@dericed.com>> wrote:
>>>>>>>> Another annoyance here is the the xml processor in the submission
>>>>>>>> tool
>>>>>>>> and the xml processor at
>>>>>>>> https://xml2rfc.tools.ietf.org/experimental.html
>>>>>>>> <https://xml2rfc.tools.ietf.org/experimental.html> perform
>>>>>>>> differently,
>>>>>>>> so there’s effectively no test environment to use before
>>>>>>>> submitting.
>>>>>>>> Dave
>>>>>>>
>>>>>>> Dave, can you please try xml2rfc.ietf.org
>>>>>>> <http://xml2rfc.ietf.org>;, which is a different installation,
>>>>>>> supported by a different entity?
>>>>>>
>>>>>> Michael:  Umm, no, that redirects to tools.ietf.org
>>>>>> <http://tools.ietf.org>;, too, and the processor
>>>>>> behind that, and behind
>>>>>> https://xml2rfc.tools.ietf.org/experimental.html,
>>>>>> _and_ behind the submission tool are all the latest release of
>>>>>> xml2rfc.
>>>>>>
>>>>>> David: I'd be happy to look into any discrepancies you see, but I
>>>>>> can only
>>>>>> do so if you provide observations and input XML files for me to work
>>>>>> with.  It won't happen if I don't know about it :-)
>>>>>
>>>>
>>>>> When I use the tool at https://xml2rfc.tools.ietf.org or
>>>>> https://xml2rfc.tools.ietf.org/experimental.html, the plain text
>>>>> output appears ok, but when I use
>>>>> https://datatracker.ietf.org/submit/
>>>>> <https://datatracker.ietf.org/submit/>;, the contents of the
>>>>> <sourcecode> elements (which are new in xml2rfc version 3) have their
>>>>> line breaks stripped out and thus cause an error when the line length
>>>>> limits are exceeded.
>>>>
>>>> That sounds like a bug.
>>>>
>>>>> For the moment, I’m proposing a find and replace to convert the newer
>>>>> <sourcecode> elements back to <artwork> elements as in
>>>>> https://github.com/FFmpeg/FFV1/pull/171/commits/a018f091534678f45cc157bfd154518ad6267c7e
>>>>>
>>>>>
>>>>> <https://github.com/FFmpeg/FFV1/pull/171/commits/a018f091534678f45cc157bfd154518ad6267c7e>;,
>>>>>
>>>>>
>>>>> thus an xml2rfc version 2 processor or version 3 processor should
>>>>> proceed accurate results though not as semantically clear in the
>>>>> xml.
>>>>>
>>>>>> FWIW, if you have atools.ietf.org <http://tools.ietf.org/>login, you
>>>>>> can use the xml2rfc
>>>>>> issue
>>>>>> tracker athttps://trac.tools.ietf.org/tools/xml2rfc/trac/newticket,
>>>>>> for
>>>>>> xml2rfc issues, andhttps://trac.tools.ietf.org/tools/ietfdb/newticket
>>>>>> for datatracker issues.
>>>>>
>>>>> I don’t have a login, but I’ll see if I can make a tiny test file for
>>>>> this issue and file a concise ticket.
>>>>
>>>> Please don't put time into this; I'm perfectly happy to work from the
>>>> xml
>>>> file you found problematic on submission.  Just let me know the name
>>>> of the
>>>> draft in question, so I can pull it.
>>>
>>> I just uploaded this new version of the EBML specification to the
>>> datatracker.
>>>
>>>
>>> On lines 559-609 and 824-939 there are <sourcecode> nodes that contain
>>> multi-line xml expressions. During the upload the idnit test showed no
>>> issues or nits at all.  However in the resulting files at
>>> https://datatracker.ietf.org/doc/draft-ietf-cellar-ebml/12/ the outputs
>>> remove the line-breaks of the multi-line xml within the <sourcecode>
>>> nodes. So in the HTML version the <sourcecode> is shown as a single
>>> line, see
>>> https://www.ietf.org/id/draft-ietf-cellar-ebml-12.html#name-ebml-schema-example,
>>>
>>>
>>> and in the plain text as well, see Section 11.1.1 of
>>> https://www.ietf.org/id/draft-ietf-cellar-ebml-12.txt.
>>>
>>> In another document, I have been find/replacing the <sourcecode> node
>>> with the older <artwork> node as a workaround.
>>>
>>> Best Regards,
>>>
>>> Dave Rice
>>> ...
>>
>> I had the same problem and reported it as
>> <https://trac.tools.ietf.org/tools/xml2rfc/trac/ticket/444>;.
>>
>> Best regards, Julian
>
> Henrik,
>
> could you please let us know what the status is? Does it make sense to
> retry now?
> ...

Henrik,

any news on this? Any idea what could be causing this, and when it'll be
fixed?

Best regards, Julian