Re: [dmarc-ietf] DMARC bis: ticket 69: add JSON reporting format?

[Alessandro Vesely <[e-mail]>]

On Wed 20/May/2020 22:00:34 +0200 Hector Santos wrote:
> On 5/20/2020 2:43 PM, Alessandro Vesely wrote:
> 
>> I mean, what is the CSV format of the following report, that I sent yesterday
>> for this list:
> 
> Sorry, if I ignored it.
> 
> Forgetting fact that you can your report easier to read for consumers, these
> would be an example of the CSV field headers.
> 
> CSV headers:
> 
> report_metadata.org_name, report_metadata.email, report_metadata.report_id,
> report_metadata.date_range.begin, report_metadata.date_range.end
> 
> Policy_Published.domain, Policy_Published.adkim, Policy_Published.aspf,
> Policy_Published.p
> 
> record.row, record.row.source_ip.record.count. record.row.policy_evaluated,
> record.row.policy_evaluated.disposition,
> record.row,policy_evaluated.disposition, record.row.policy_evaluated.dkim,
> record.row.policy_evaluated.spf
> 
> Note: You don't have to stick to redundant "name space" field names.


You didn't include auth_results.  That's tricky because you can have zero or
more dkim and spf results.  That would bring on a variable number of columns,
with no hint about which is which.

As Freddie noted, repeating the headers for every record may sound a little bit
wasteful, but gzip may come to rescue here too.

As for readability, aggregate reports are designed to be machine-readable to
the extreme detriment of any kind of human readability.  The best samples of
such attitude are the date_range elements.  HTML provides for a much better
human readability.  (Indeed, it is one of the formats you mentioned, as it is
supported by Google Docs).  I attach the HTML equivalent of the data I sent
yesterday.  Posters whose From: was rewritten can easily spot their row
—readability meaning exactly that.

Let me note that such htmlization is the result of a DMARC XSLT applied by a
mail filter after rua messages authentication but before delivery.  That way, a
readable format of the reports is ready in the mail folder whenever I'd care to
look at it, *as if it had been written in HTML in the first place*.  By similar
techniques one can obtain JSON, SQL, TXT, ….  In the face of such flexibility,
I'm puzzled by your asking for more.


>>>> ...  Can we get back to work, please?
>>>
>>> Sorry, but I consider a rude, disrespectful and ignorant statement, to be
>>> saying that.
>>
>>
>> No personal attack intended.  I'm being rude because I have the impression that
>> you are not defending a concrete, well defined need, but instead find new
>> arguments opportunistically to pursue a vague sense of format fashion.
> 
> That's a personal attack. If you don't understand the proposal, you should back
> off or ask for clarification.


I /am/ asking for clarifications.  Please excuse my tone if it hurts you.  I'll
try and keep calm, but please restrain from technically flaky arguments such as
CSV.  I think you're perfectly aware of the capabilities I'm exemplifying here.
 They rest on the fact that a report consumer knows in advance the format of
the data inside the received gzip.  Hence, that flexibility would be destroyed
by the introduction of multiple formats, as they would come randomly at the
mercy of the report generators, any prf= notwithstanding.  Not a great loss,
given your feeling about reporting?


>> You shifted from an asserted necessity of producers to a possible desire
>> of consumers.>
> I did no such thing. I won't repeat it, but it appears you didn't understand
> the proposal.


For sure, I don't understand the proposal.  Let's start over:

On Sat 16/May/2020 19:48:25 +0200 Hector Santos wrote:
> Just consider, when the spec has XML-only, then for those who use a solid
> JSON I/O system, they are now going to be required to add XML. So for them,
> its additional development complexity.  Everything they probably do JSON
> related. The exception would be DMARC using XML. This alone can delay or
> push aside DMARC Reporting implementation.


Does DMARC Reporting implementation mean generation or consumption?


On Wed 20/May/2020 02:23:37 +0200 Hector Santos wrote:
> I suggest that there is a new tag that provides a "Preferred Report Format"
> or "prf=" tag using registered acronymns for long time "standard" formats.
> For example:>
> prf=cvs,json,xml,afrf,iodef
>
> [...]
> 
> The verifier will do what can it offer. The publisher is providing a
> preference, that it may not get.


That's 100% uncertainty of the result, isn't it?  How is flexibility improved?
 I cannot store a received report as-is in Google Docs anyway, can I?


Best
Ale
--