Re: [precis] draft-whited-precis-test-vectors-00

[Sam Whited <[e-mail]>]

On Thu, Dec 6, 2018, at 10:12, Tom Worster wrote:
> First, I also think this work is valuable but I don't understand how the 
> ID is supposed to proceed. The point of bringing it to IETF is to get 
> its authority behind the statement that these specific vectors test 
> conformance to the PRECIS RFCs. How can the IETF give this without 
> restarting the WG to push the ID forwards as a Standards Track RFC? I 
> don't think the independent submission review process can.

I don't really understand the process, but this makes sense to me and I think you're probably right. I'd be curious to hear if other IETF people who actually understand how any of this works agree or not though.

> Second, my experience is that formatting of test vectors is kinda all 
> over the place in RFCs. I would not look there for guidance. Idk what 
> XML format you refer to but I wouldn't want to go down any XML road as 
> either producer or consumer.

Sorry, I meant the rfc2xml format; I'm not really sure if there's a name for that. I was originally writing nroff format directly, but people kept telling me I should use the XML format for best results. Either way, I couldn't find a decent way to support the vectors without them wrapping awkwardly or being hard to read in the fixed-width formats that get generated. 

> I suggest you devise the simplest 
> machine-readable file format that you can imagine and publish it 
> yourself, e.g. as a Github gist. Anyone can then easily transform that 
> into any other format, for documentation, for a unit test data provider, 
> or whatever. I think this serves your and our needs best even in the 
> case that IETF would adopt the ID, since you can use the file to 
> generate IETF-format docs.

I'm actually already doing this (more or less): they're being generated from the unit tests in my Go implementation of PRECIS with a few minor changes. I should rework the tool that does the conversion at some point and publish it somewhere. However, this doesn't solve the formatting problems.

> For example, I was recently using this wonderful text file 
> https://www.cl.cam.ac.uk/~mgk25/ucs/examples/UTF-8-test.txt which is 
> both documentation and machine readable. Your ID is machine readable 
> (what isn't these days?) but it isn't so simple to either read or write. 
> In your position I'd look at UCD files for format ideas.

I would love it if the ID could be easily machine readable too, but I couldn't find a good way to do this. I can easily spit out a different format from the tools that convert the Go unit tests to xml2rfc format though, so I could probably publish something else like the UCD files if it would be helpful to others.

Thanks for the feedback and suggestions!

—Sam

-- 
Sam Whited
[e-mail]