[Tools-team] Datatracker Patch Report

[Glen Barney <[e-mail]>]

All -

An emergency patch has been applied to the live datatracker.  This is a
report on that patch.

We received reports today that Internet Draft announcements were not
being received by people.  Specifically, many people in regular
single-message-delivery mode were not receiving the email; however,
digest readers were receiving it.  One person reported that MIME headers
appeared to be malformed.

Nothing had changed on the AMS end since this was working.  The old ID
Submission Tool was not modified (except for a reopening-time patch,
which was not related), and mailman, postfix, and other MTA chain tools
were not modified.

What had changed, however, was that we were now processing scheduled
announcements through the datatracker.  I attempted to find Henrik, but
he was unavailable.  I did reach Robert Sparks, but he had not been
involved in this change.

I investigated backup copies of the messages on the server.  I found
that the Content-type header was being squashed to lowercase.  This was
causing the IDST "NextPart" boundary to be squashed in the header to
"nextpart".  This, in turn, was causing many MTAs (and MUAs) to fail to
receive and/or parse the message because of the malformed headers and
MIME structure.

Armed with this information I dove into the datatracker code.  I traced
things down to the following file:

	web/ietf/announcements/send_scheduled.py

Line 16 said this:
    content_type = announcement.content_type.lower()

I changed it to this:
    content_type = announcement.content_type

That was to fix the squashing, but I also noticed that later code
depended on that squashing:

Line 20 said this:
    elif 'multipart' in content_type:

Since the IDST injects records into the database with a Content Type of
"Multipart/Mixed" (instead of "multipart/mixed"), I changed it to this:
    elif 'ultipart' in content_type:

(I just removed the "M" from the test).

These two changes allowed I-D announcements to go through and be
processed and received correctly.  I then went in and requeued all the
sent mail in the database from November 1 onward, and all those messages
did subsequently get processed correctly.

Doubtless someone will want to go in and do a cleaner correction for
this problem - I was acting without Henrik's guidance on an emergency
basis, just to get those announcements fixed and flowing again.

Hopefully this information will facilitate a fix for the next update.
Sorry if I treaded on toes here but as I'm sure you understand, "In an
emergency, anything goes."  Please let me know if further information is
needed.

Thanks,
Glen