Re: [rfc-i] looking for a volunteer to write a simple script

> On Jul 12, 2019, at 5:19 PM, Joe Touch <touch@strayalpha.com> wrote:
> 
> I looked at this a bit and getting it closer to 100% is tricky. There are a LOT of corner cases, including the possibility that the <bcp14> tags are outside other tags, e.g., bold, etc.

The IETF absolutely *lives* for corner cases. :-) We are expecting those, and it’s ok. There is no expectation of 100% here.

> 
> I don’t know fi it’ll be worth the effort, given the need for post-checking anyway.
> 
> But I’ll take a look over the weekend and let you know if there’s a solution that gets closer without increasing falses.

You’re a gem! Thanks!

Heather

> 
> Joe
> 
>> On Jul 12, 2019, at 11:33 AM, Joe Touch <touch@strayalpha.com <mailto:touch@strayalpha.com>> wrote:
>> 
>> I can fix that in a few minutes...
>> 
>> On Jul 12, 2019, at 11:29 AM, Sandy Ginoza <sginoza@amsl.com <mailto:sginoza@amsl.com>> wrote:
>> 
>>> Thanks Joe!
>>> 
>>> I just tested this updated script and it seems to work well.  I am happy to see it catches things like “MAY” (within quotes), but does not tag items Carsten noted (e.g., “MARSHALL”).   
>>> 
>>> I note that it does double tag keywords if they’ve already been tagged, but that should be something we can check for before running the script (i.e., check whether the keywords have already been tagged).  
>>> 
>>> Thanks!
>>> Sandy
>>> 
>>>> On Jul 12, 2019, at 11:02 AM, Joe Touch <touch@strayalpha.com <mailto:touch@strayalpha.com>> wrote:
>>>> 
>>>> Quick update:
>>>> 
>>>> perl -0777 -pe "s/(\b(((MUST|SHOULD|SHALL)(\s+NOT)?)|((NOT\s+)?RECOMMENDED)|MAY|OPTIONAL|REQUIRED)\b)/<bcp14>\$1<\/bcp14>/g" INFILE.xml > OUTFILE.xml
>>>> 
>>>> 
>>>> 
>>>> Joe
>>>> 
>>>>  
>>>> On 2019-07-12 10:43, Joe Touch wrote:
>>>> 
>>>>> This will do the trick:
>>>>> 
>>>>> 
>>>>> 
>>>>> perl -0777 -pe "s/(((MUST|SHOULD|SHALL)(\s+NOT)?)|((NOT\s+)?RECOMMENDED)|MAY|OPTIONAL|REQUIRED)/<bcp14>\$1<\/bcp14>/g" INFILE.xml > OUTFILE.xml
>>>>> 
>>>>> (replace INFILE.xml and OUTFILE.xml with your filenames)
>>>>> 
>>>>> If you want it to edit in-place (riskly, but simpler if you work on a copy anyway):
>>>>> 
>>>>> perl -0777 -i -pe "s/(((MUST|SHOULD|SHALL)(\s+NOT)?)|((NOT\s+)?RECOMMENDED)|MAY|OPTIONAL|REQUIRED)/<bcp14>\$1<\/bcp14>/g" INFILE.xml > OUTFILE.xml
>>>>> 
>>>>>  
>>>>> 
>>>>> 
>>>>> On 2019-07-12 10:26, Heather Flanagan wrote:
>>>>> 
>>>>> 
>>>>> 
>>>>> On Jul 12, 2019, at 10:23 AM, Julian Reschke <julian.reschke@gmx.de <mailto:julian.reschke@gmx.de>> wrote:
>>>>> 
>>>>> On 12.07.2019 18:55, Heather Flanagan wrote:
>>>>> Hola a todos!
>>>>> 
>>>>> The RFC Editor has the need for a comparatively simple script that would
>>>>> automatically add <bcp14></bcp14> tags to requirement language in v3 RFCs.
>>>>> 
>>>>> Specifically, this would take a v3 XML input file, and create a v3 XML
>>>>> output file with <bcp14></bcp14> added around each instance of a 2119
>>>>> keyword in the file. (MUST, MUST NOT, REQUIRED, SHALL, SHALL NOT,
>>>>> SHOULD, SHOULD NOT, RECOMMENDED, NOT RECOMMENDED, MAY, and OPTIONAL)
>>>>> 
>>>>> Anyone up for helping us out with that?
>>>>> 
>>>>> Thanks! Heather
>>>>> ...
>>>>> 
>>>>> The tricky part is to find the right instances. For instance, what if it
>>>>> appears in a quote, or in artwork? Or if "SHALL NOT" is across a line
>>>>> break...
>>>>> 
>>>>> So the output will require sanity checking.
>>>>>  
>>>>> Well, yes, of course. We're aiming for a rough pass to catch maybe 80% of the situations. Everything will still need to be reviewed.
>>>>> 
>>>>> 
>>>>> I assume that the tool is supposed to preserve whitespace, line breaks
>>>>> etc? This essentially rules out running the input through an XML parser...
>>>>> Seriously, we're not aiming for that robust right now. It doesn't have to be perfect, it just has to help.
>>>>>  
>>>>> -Heather
>>>>>  
>>>>> 
>>>>> 
>>>>> _______________________________________________
>>>>> rfc-interest mailing list
>>>>> rfc-interest@rfc-editor.org <mailto:rfc-interest@rfc-editor.org>
>>>>> https://www.rfc-editor.org/mailman/listinfo/rfc-interest <https://www.rfc-editor.org/mailman/listinfo/rfc-interest>
>>>>> _______________________________________________
>>>>> rfc-interest mailing list
>>>>> rfc-interest@rfc-editor.org <mailto:rfc-interest@rfc-editor.org>
>>>>> https://www.rfc-editor.org/mailman/listinfo/rfc-interest <https://www.rfc-editor.org/mailman/listinfo/rfc-interest>_______________________________________________
>>>> rfc-interest mailing list
>>>> rfc-interest@rfc-editor.org <mailto:rfc-interest@rfc-editor.org>
>>>> https://www.rfc-editor.org/mailman/listinfo/rfc-interest <https://www.rfc-editor.org/mailman/listinfo/rfc-interest>
>>> 
>> _______________________________________________
>> rfc-interest mailing list
>> rfc-interest@rfc-editor.org <mailto:rfc-interest@rfc-editor.org>
>> https://www.rfc-editor.org/mailman/listinfo/rfc-interest
> 