Re: [IPFIX] Fwd: New Version Notification for draft-trammell-ipfix-ie-doctors-00

[Hadriel Kaplan <[e-mail]>]

Hi Brian,
I agree with your comments.  I find the IESpec short-hand especially useful for emails more than anything else, so even for just that I'm fine with 'em. :)
I do worry about having two forms of dictionaries for IPFIX readers, but I suppose the market will sort it out. (or not)

-hadriel

On Oct 18, 2010, at 3:21 AM, Brian Trammell wrote:

> Hi, Hadriel,
> 
> Comments per tradition inline...
> 
> On Oct 8, 2010, at 9:27 PM, Hadriel Kaplan wrote:
> 
>> 
>> Hi Brian,
>> comments inline...
>> 
>> On Oct 8, 2010, at 2:39 AM, Brian Trammell wrote:
>> 
>>> Hi, Hadriel,
>>> 
>>> Thanks for the feedback.
>>> 
>>> The primary design goal of the IESpec format is human readability in the expression of information model extensions and templates, while maintaining simple machine readability. The idea is implementors can read an Internet Draft about an IPFIX application with example templates in IESpec form, copy and paste the example templates out of it into an implementation supporting IEspec, fill in the fields and go.
>> 
>> At the end of the day what we're talking about is a form of a "dictionary" file, similar to those used for other protocols, like for SNMP, DIAMETER, RADIUS, etc.  Right?  Like not just for putting into a RFC, but more importantly for importing into a reader so it can decode and understand the IEs, right?
> 
> There are two orthogonal dimensions here, on both of which we're getting confused... First there's the human-readable versus machine-readable issue, then the dictionary (Information Model) version template issue. On reflection, the primary application for iespec (and therefore the reason to include it in iedoctors) is human-readable templates with easy machine readability.
> 
> As a dictionary file, it's basically a human-editable subset of 5610, and as such only useful for the <type> and name information. But as a dictionary, the IANA registry is of course far preferable. So if the collector device is capable of doing so, I'd recommend downloading the IANA registry and parsing it at boot or pre-parsing it (maybe into 5610)  once per fortnight or so. As long as I don't have to _hand edit_ the IANA XML, I'm happy.
> 
>> Again, I'm not a big fan of XML, but there are a ton of XML parser libraries available, and the importing of that into a reader so it can understand the new IEs is done rarely, and with a CPU (not an FPGA or network processor).  Libraries exist to not only read them, but verify their correctness using a schema.  Further, they support comments and text descriptions, which are useful for an IPFIX reader importing them.  An IPFIX reader can use those to display in a GUI as additional help, and a human can use them to understand stuff when he/she reads the XML.  
>> 
>> For example, suppose vendor BigCorp wants to give out such a "dictionary" file for some PEN IEs its BigBox generates, so that IPFIX Readers can decode them.  One of those PEN IEs is for a "camFlowId".  In IESpec, it might be:
>> camFlowId(35566/123)<unsigned32>[4]
>> 
>> That doesn't tell the human nor the IPFIX collector/reader much info.  In XML it's a lot longer text, but it can include some more info like this:
>>      <description>
>>        <paragraph>
>>          An identifier of a single Content Addressable Memory index allocated for the flow.  Two CAM entries are allocated, for each direction, unless the system is provisioned otherwise.
>>        </paragraph>
>>      </description>
>>      <reference>
>>        <paragraph>
>>        See BigCorp User Manual chapter titled "Hardware Resources" for more information.
>>        </paragraph>
>>      </reference>
>> 
>> And the file can even have a "<-- BigCorp dictionary file for BigCorp BigBox 10000 System, version 1.2.3. Created 2-10-2010 -->" comment at the top.
>> 
>> 
>> 
>>> For readability and compactness, it's intended to address shortcomings both in the XML format(s) already defined for IPFIX IEs
>> 
>> What shortcomings?  There're some fields I think it should have that it doesn't, and the structure of it is weird, but  I'm a nitpicker.  Is there something fatally wrong with it?
> 
> Eyeball-parsing the XML formats for dictionary purposes is daunting, and they're not at all adaptable to easily readable templates. Fortunately, IANA has the eyeball parsing problem fixed, 
> 
>>> and in the traditional 32-bit-wide box-and-line diagrams, which make it difficult to fit a nontrivial template onto a single page, and require some manual work to turn into an implementation.
>> 
>> This is one of the things I was worried about - that we would be saying this would avoid showing things in 32-bit-wide box-and-line diagrams.  I find those drawings are really, really useful.  For example, I couldn't make heads or tails of the structured-data draft until I saw those examples.  
> 
> The thing is, they're useful primarily when describing the _framing_ of the protocol. Here's what a template looks like. Here's what a message looks like. Given a template, here's a record. Structured data is defining new _types_, with internal structure, and displaying that structure. Of course we need 32-bit box and line diagrams in that case. With respect to simple applications of IPFIX, just defining a new template doesn't change the framing. But I hadn't considered the possibility that I'm old and jaded, and that everyone else enjoys seeing the same boxes I'm getting bored with. :) So that's a valid argument against more compact templates in drafts...
> 
> The structured-data heirarchy extensions for IEspecs are meant to illustrate applications of structured-data, not the framing thereof. But there's a very valid point here -- structured-data is new enough that we really probably _do_ want to draw things out explicitly the first few times we use it.
> 
>> Fitting them on a page is not a serious issue really, is it?  I mean if it crosses pages, people can deal.  
> 
> A thirty-element template (not unthinkable) crosses two pages. At some point there's a tradeoff between verbosity for clarity and so much verbosity that you need to tape things to the wall to have any hope of seeing what's going on. 
> 
>> I know it takes more work to draw them, but you're writing an RFC for others to be able to interop with you, so that's life.  If a nontrivial template is difficult to draw, maybe that will encourage people to not make complicated templates. :)  I'm half joking, but my point is the work to write the RFC is really moot compared to helping people reading the RFC implementing it right.  
> 
> I chose the word "non-trivial" poorly -- there's not really a complexity continuum in templates, it's either "long" or "short". It's hard to draw _long_ templates, and it's a bad idea to encourage people to use short templates when what they need is a long one. :)
> 
> The reason to reduce the amount of work to write an RFC, though, is not to make _my_ life easier. It's to make it easier for IPFIX _non_-experts to write RFCs describing new applications. Less work, and less chance of making a mistake leading to ambiguity, means better, more readable, more quickly and easily edited and published RFCs.
> 
> Regards,
> 
> Brian