Re: [apps-discuss] Working Group Last Call draft-ietf-appsawg-sieve-duplicate

[t.petch <[e-mail]>]

Mostly inline, but my big comment which gets lost in the detail is about
scope of caches.  You say
"there is only one 'cache' (in the document it is called the duplicate
tracking list)"

Really, worldwide, like the DNS root?-)

Hitherto, as I understand it, sieve has had no state; a filter is
applied to a stream of messages and that is it, forget it and start
again.  This adds state.  This then introduces the question of scope.
If I get the same message from two different ISP (which I  do), will
that be detected?  I expect not.  When those ISP outsource their MX to
the same third party, will that be detected?  I expect not as long as
they have different mail domains.  At what point is there a single
cache?  When I used multiple mailboxes for the same mail domain?  Only
when I use a specific  mailbox? or when?

In a sense it does not matter because it only effects the user
experience and not the protocol on the wire, but I expect that users
will care if the implementation on ISP A gives totally different results
to ISP B.  And of course, when there is more than one cache (worldwide
:-), then operators of this facility will need more resources to
maintain them.

Tom Petch

----- Original Message -----
From: "Stephan Bosch" <[e-mail]>
To: "t.petch" <[e-mail]>; "Murray S. Kucherawy"
<[e-mail]>; "IETF Apps Discuss" <[e-mail]>
Sent: Friday, January 10, 2014 12:01 AM
> On 1/2/2014 7:24 PM, t.petch wrote:
> > This I-D seems to need more thought.
> >
> > s.1 "For example, if a member of the list decides to
> >    reply to both the user and the mailing list itself, the user will
> >    one copy of the message directly and another through the mailing
> >    list.
> >
> > Well, they MAY, but they don't on a good list system, such as the
one in
> > use here.
> >
> > "Also, if someone cross-posts over several mailing lists to
> >    which the user is subscribed, the user will receive a copy from
each
> >    of those lists."
> >
> > Ditto, not here.
>
> Ok, but these situations are quite common. I've implemented an earlier
> version of this extension based on user requests. So, what do you want
> me to do? Word this differently so that it is clear that this
shouldn't
> happen for sanely configured mailing lists?

I found the 'will' too forceful; my instant reaction was 'no it won't'
because that is my experience.  Just moderate it slightly, 'will often'
'may' 'commonly'-  just not a somewhat forceful 'will'

> > "   Duplicate messages are normally detected using the Message-ID
header
> >    field, which is required to be unique for each message.  "
> >
> > REQUIRED maybe, but I seem to recall the malformed-mail I-D raising
the
> > possibility that it was not.  In which case, ...?
>
> I'm not sure how common that is, but you are right:  as the
> specification is now, that would cause a false positive. We can make
the
> default a bit more complex by combining the Message-ID  with some
other
> header (Date perhaps?), thereby further reducing the likelihood of a
> false positive. I guess we need to think about that a little more.

Yes, I think you should allow for the possibility in the I-D - as to
how, I am less fussed.  Could be 'outside the scope of' up to 'should
detect duplicates and discard as malformed' - just show that it has been
considered.

> > s.3
> > "an earlier Sieve execution."
> > reading on it is apparent that this is any number of executions
limited
> > by the size of the FIFO cache and the maximum lifetime of entries in
the
> > cache.
>
> Yes. So, what exactly is your comment here? I don't think it is useful
> to mention such detail early in the description.
>
> > "   Usage:  [":header" <header-name: string> /
> >                           ":uniqueid" <value: string>]
> >
> > Why have two way of doing the same thing?  As I read it, this test
is on
> > a header field, so why not have just ":header" with a default of
message
> > I-D?
>
> I am not sure what you mean here. The :uniqueid argument does
explicitly
> not operate (directly) on a message header, but rather on some string
> value composed by the user (using the variables extension). This can
> consist of header field contents, but also on message body or even
some
> source other than the message being delivered.

If I understand it aright, the test for duplication can be on
 - message id
 - header field
 - something else
which are invoked by <nothing>, :header, :unique-id respectively.
Since message id is just another header field, why not merge the first
two with the message id as the default header field if none other is
specified?

> > And what happens if I use header field X in one execution and then
> > header field Y in another? I presume separate caches for X and Y, in
> > which case, duplicates may not be detected.
>
> No, there is only one 'cache' (in the document it is called the
> duplicate tracking list). See the following text:
>
>  The "duplicate" test MUST track an unique ID value independent of its
>  source.  This means that it does not matter whether values are
>  obtained from the message ID header, from an arbitrary header
>  specified using the ":header" argument or explicitly from the
>  ":uniqueid" argument.

see above


> Some examples follow this text. Do you mean that this needs to be
> clarified more?
>
> > The use of multiple fields
> > opens up all sorts of complications that need more explanation
depending
> > on the concept of the scope of the operation, which I do not see
clearly
> > explained.
>
> I am not sure what you mean here. I am assuming this comment is not
> relevant given the above. Please clarify otherwise.
>
> > "The user can explicitly control the
> >    length of this expiration time by means of the ":seconds"
argument,
> >    which is always specified in seconds.  "
> >
> > seconds seems short to me.  On the IETF lists, I typically see a gap
of
> > several hours between a message on one list and a message on another
> > list, with four hours being the norm.  I would regard 5 minutes as
the
> > minimum and 36 hours, or perhaps less, as the maximum.
>
> Given the vacation-seconds extension, the use of a seconds granularity
> is not strange in the Sieve realm.
>
> I like the flexibility of using seconds (mailinglists are not the only
> application area of this extension), but I am not against changing it
to
> :minutes per se. Do any other people have thoughts?

Ok but I just got a duplicate, once via the ietf announce list, once via
the ietf main list, and they were 10 hours apart.  For me, this is
typical, hours not seconds.

> > "By adding the ":header" argument with a message header
> >    field name, the content of the specified header field can be used
as
> > "
> >
> > Does this apply to all headers? I assume it does, since if message
I-D
> > is not used, then it would be an X- proprietary one that would seem
to
> > me to be the next best thing.
>
> Any header can be selected with this argument.
>
> > "   If the tracked unique ID value is extracted directly from a
message
> >    header field, i.e., when the ":uniqueid" argument is not used,"
> >
> > you are saying that when uniqueid is not used, then a unique ID is
used.
> > I think that this will cause confusion here and elsewhere - you need
> > another term than 'unique ID' as the collective noun for the
identifying
> > string you are using in the equality comparison.
>
> Hmm, yeah. Any suggestions? :)

No good ones:-)  'identifier' ' message identifier'  just something that
is visually different from unique-id so preferably not incorporating
'unique'.

> > "leading and trailing whitespace MUST first be trimmed from the
value"
> >
> > This is a can of worms.  Normalisation often appears on these lists
> > without, usually, a satisfactory answer, let alone the issues of
i18n.
> > More needs to be considered here.
>
> This mainly serves as a means to prevent stray white space from
messing
> with the string match. The core Sieve language also does this for
> instance for the header test. And how would i18n be relevant here?

Read RFC6532; that you have not referenced it makes me think that you
have not considered i18n which I would regard as remiss nowadays.

> Regards,
>
> Stephan.