Re: [sip-clf] [sip-ops] [dispatch] SIP-CLF: Results on ASCII vs. binary representation

[Adam Roach <[e-mail]>]

On 05/04/2009 08:44 PM, Dean Willis wrote:
>
> On May 4, 2009, at 2:58 PM, Adam Roach wrote:
>
>> Dean Willis wrote:
>>> Right. In my experience, binary offers only marginal performance 
>>> over structured ASCII.
>>
>> I'll correct this slightly: binary offers only marginal performance 
>> over *properly* structured ASCII. I argue that the Apache log format 
>> doesn't qualify for this description. (I say this after having to 
>> deal with processing relatively large Apache logs, and suffering from 
>> the vast quantities of time such processing takes).
>>
>> I'm okay with Tom's suggestion to add ASCII length (and tag?) fields 
>> to the text format. If people want to take it that direction, I could 
>> re-cast my proposed format so it contains fixed-length ASCII number 
>> representations for pointers, lengths, and tags; and so that it 
>> terminates records with something like an ASCII 0x0D.
>
>
> One could design things such that each record contains an initial 
> "structure" filed with a fixed-format  series of indices (total 
> length, number of fields, lengths for each field, followed by 
> variable--length data fields, tab (or other reserved char) delimited 
> followed by a CRLF terminator (included in total-length index). This 
> gives us records that are fast-parseable, but still human-readable.
>

Yes. That is what I was proposing.

Followups to the sip-clf list (not sip-ops!).

/a