Re: ietf-nntp Issue: reinstatement

[USENET news manager <[e-mail]>]

Chris Hall wrote:
>In article <[e-mail]>et>, "Clive D.W. Feather"
><[e-mail]> writes
>>[clarification about intended meaning of backfilling]
>>
>>My definition of reinstatement: an article is removed (either because of a
>>cancel or expiry), and then the server operator decides that this removal
>>was an error and reinstates the article *with the same number*.
>>
>>In earlier discussion, I was under the impression that people wanted to
>>allow this behaviour; the alternative is to state explicitly that, once an
>>article has been removed, it will never reappear.
>
>Which seems unnecessarily severe.
>
>>Jon Ribbens:
>>>...
>>> Even if you do want
>>> this stuff in, I don't see a need for the condition that the article
>>> number MUST be no less than the first article number.
>
>>Without that condition, the low water mark might decrease. Everyone was
>>against that.
>
>If the minimum article number is never to decrease, then it's not
>possible in every case to reinstate an article -- it's obviously not
>possible to reinstate an article that had the previous minimum article
>number [unless it can be guaranteed that no client ever said GROUP
>between the removal and the reinstatement !]. 
>
>Either one lives with the curious limitation on reinstatement, or one of
>reinstatement or increasing minimum article number has to go.
>
>What difference does it make if the "first" article number in one GROUP
>command is less than it was in the previous GROUP command ?

I can't see that it's a problem, as long as it happens only for reinstatement
of a previously-existing article with its original article number - as a
more-or-less rare event, done to reduce the impact of (e.g.) malicious
cancels, not to add new articles which some readers would miss.

>From the client's perspective the only thing that matters is that new
>articles available from the server have higher article numbers than old
>articles.  So the client can, *without* missing articles, start from the
>highest article number it saw last time (provided that is within the
>"first".."last" range).

That's the important distinction between the consequences of reinstatement
and backfilling...

>>I don't know what the concensus mechanism is for this list; when someone
>>tells me we have a consensus one way or the other on this issue, I'll
>>adjust the wording as needed.
>
>The choices appear to be:
>
>  1. ban reinstatement,
>...
>     This doesn't seem right.

Agreed.

>  2. reinstate with its original article number,
>
>     assuming there's some mechanism at the server to do that !  For
>     NEWNEWS purposes the article would have to be reinstated with its
>     original "timestamp".

With INN, unless you'd run "makehistory" to rebuild the history database
from the current spool contents in the meantime, the history entry (which
includes the timestamps) will remain until the article would have expired
(or possibly longer, for groups with very short expiry), so reinstating the
article file should plus "ctlinnd renumber <groupname>" should do it. [Just
an example to show that it's not necessarily difficult to do.]

>     For clients that remember the state of newsgroups (eg. those that
>     use NEWNEWS), there is a problem here.  Clients that look at a
>     group in the interval between removal and reinstatement may well
>     never see the article !  The longer the delay between removal and
>     reinstatement, the greater the number of people who will miss the
>     article.

Yes, but if reinstatement is used only to minimise the effects of 
accidental or malicious loss, it will benefit those people who see the
reinstated article, and the effect is no worse for others than if the 
article had not been reinstated.

>     Reinstatement is a subtle form of back-fill, against which faces
>     have been set.

Yes and no... They have properties in common, but backfilling requires 
clients to locate newly arrived but lower-numbered articles or their 
users will see a very patchy feed, missing a lot of articles. Reinstatement
is a "best efforts" fixup for loss of previously-existing articles.

>  3. reinstate with a new article number (and new "timestamp"),
>
>     which means that no client should miss the article, but some may
>     see the article twice !
>
>     Duplicating articles is anathema.  Nevertheless, I expect clients
>     cover themselves against it, and simply ignore duplicates.  But, if
>     the delay between removal and reinstatement is long, then there is
>     the risk that the client has expired its memory of the article-id,
>     and not be able to detect the duplication.

Clients using article numbers wouldn't know if a reinstated article with
a new number was a duplicate of one they'd shown the user previously (as
when broken gateways spew duplicates with new message IDs). Clients (if 
there are any?) which saved message IDs for all articles seen recently
(could be many megabytes) should be safe against duplicates, *except* that 
with a history database (underlying lookup by message ID, and NEWNEWS) like 
INN's, articles reinstated with a different article number could not be added
to the history database in any straightforward and non-disruptive way, and 
it would retain the original details including article file number (hence
also number) - so the reinstated article would never be seen (and the 
reinstated article would never be expired, since the history database is 
used by expiry to derive the list of files to unlink).

>Looks like a choice between evils to me :-(.

For anyone using INN, the choice is an easy one - reinstatement with the
original article number is the only practical option!

>If reinstating articles is a common requirement, then there is a serious
>problem here.  The client either has to be able to tolerate a form of
>back-fill, or it has to tolerate duplicate articles.

Only if you consider it essential for such articles never to be missed by 
clients, rather than viewing it as minimising the effects of losing the 
articles. Unless you could reinstate a copy of the article as it appeared on
your server (and if it's been deleted, that would be fiddly), other problems
could arise (e.g. Xref: header not matching reality on your server, Path: 
misleading, etc.) so it seems unlikely to be done very often, anyway. I've 
only ever needed to do it when something was posted in a local group which 
looked seriously inappropriate, when I mv'd it elsewhere until I'd checked 
that it was OK, then mv'd it back.

                                John Line
-- 
Cambridge University Computing Service - USENET news manager. Usually John Line
[e-mail]    (alias {newsmaster,news,usenet}@news.cam.ac.uk)