Re: ietf-nntp Backfill

[Chris Hall <[e-mail]>]

In article <3.0.32.19961218122955.00685434@lacroix>, Jack De Winter
<[e-mail]> writes
>At 04:01 PM 12/18/96 +0000, Clive D.W. Feather wrote:
>>Mark Sidell said:
>>> Anyway, as far as I know, demon's is the only server that backfills.  
>>> But, perhaps Clive could provide more information.

>>I am told that the Demon server no longer backfills; the cause of this
>>behaviour has been fixed. I can't comment on the LISTGROUPS issue you
>>mention.

>I know one question of mine has always been:
>
>if a server is supposed to be monotonically increasing and the base
>is reset (say at the beginning of the new year as was done at my
>university), should we make sure that we put wording into the spec
>that all of the articles in any group that is reset must be renumbered?
>
>Otherwise you will have: 1-X (new articles) and Y-N(old articles yet
>to expire) and essentially will have a range of 1-N.

Yers.  And it would look just as if you were backfilling :-).  No
question, the articles need to be renumbered *and* the original ordering
should be maintained.  Moreover, if clients are not to be baffled, this
would have to be done while no one was connected to the server !

Assuming a client that remembers the state of the newsgroups being
read...

The simple way to code the client is to remember the ordinal for the
highest numbered article it has fetched.  When it asks about the
newsgroup the client checks that what it last fetched is between the
current min and max -- then it can ask either for last fetched to
current max, or for current min to current max.

This can be defeated if the server resets its numbering, but the new
range overlaps the last fetched ordinal !

Moreover, fetching from current min to current max is in danger of
refetching articles which have already been expired from the client's
machine :-(.  Steps need to be taken to avoid this....

Better is to get the client to remember the ordinals and the article ids
for the last 'n' articles it fetched.  Then it can check that one or
more of the 'n' articles have the same ordinals as last time.  If the
client finds that the articles have been renumbered, it can try to
"resync" by trying to find the new ordinals for one or more of the 'n'
articles.  If that doesn't work, it's time to give up and whinge at the
user about the unreliability of servers.

Note that this assumes that the renumbering maintains the original
ordering !

This mechanism, implemented with a little latitude, can cope quite well
with changes from one news server to another, provided the ordering of
articles is generally the same between the servers.

>ideas?

Because everyone knows that servers do reset their numbering, clients
have to adopt strategies to cope.  It's not pretty, so resetting
numbering should be deprecated.  Where unavoidable the new range should
be (a) well removed from the recent range, and (b) maintain the previous
ordering [though I confess I can see that might be "tricky" if the
server's news store has to be rebuilt after some catastrophe !]

In a perfect world a unique id would be attached to a given numbering.
Something like:

  -> GROUP news.ahha
  <- 211 1234 123456 125678 news.ahha ABCDE19961218124256.00

So the client can spot a change in the numbering by checking its id.  A
complete change in ordering could be signalled by a completely new id.
A simple change in the origin could be signalled by a change in just the
part after the '.'.

Ah well.  Back to the old heuristics....

-- 
Chris Hall