RE: First look: improved email handling in the datatracker

["Bernie Volz (volz)" <[e-mail]>]

Looks pretty good. Thanks for all the work on this!

While there are pros and cons to doing so (and it may have been debated), I do wonder why when there is a document associated with a WG Milestone, the document authors aren’t included? This might encourage the authors to keep the milestones updated (via the chairs)? In DHC, we’ve had a lot of trouble getting the authors to participant in setting (and asking the chairs to update) the milestones. In some cases, it might also help to get the authors to give their document more attention.

While not directly related to the email issue …

That the milestones are now shown on the datatracker document page is really nice and will hopefully make them more useful going forward. (Though in some cases, that they are old is bad.) Wonder if it would be possible to color code the font to make the status more meaningful – Red is in past due, Yellow is due this (and next) month, black is in the future. I suspect a lot more of us would update the milestones if we saw a lot of red.


-          Bernie

From: WGChairs [mailto:[e-mail]] On Behalf Of Robert Sparks
Sent: Wednesday, September 02, 2015 5:12 PM
To: <[e-mail]>
Subject: First look: improved email handling in the datatracker

WG Chairs :

At this year's IAB/IESG retreat we discussed making the recipients of the email
messages the datatracker sends more configurable, reducing the number of messages
sent for a given action, and making the messages themselves more meaningful.

One of the primary goals was to make it so that the people who needed to
receive each message would receive the message by default, and we would stop
doing things like putting document shepherds in the notify field. The intent
now is for the notify field to be empty most of the time, only containing
addresses that are special cases.

We made a great deal of progress on this front and have some changes ready
for you to look over and test. There are many things here that need feedback.
Please take a few moments to look this over.

Start with <https://dt-test.rjsparks.org/mailtoken/token><https://dt-test.rjsparks.org/mailtoken/token>
(This is a development instance of the tracker - you can log in as anyone using
their datatracker login name and the password "password").
You'll see a list of actions on the left, and recipients on the right.
Mouse over any of them for a short pop-up description.
You can focus on a particular action by going to, for example,
<https://dt-test.rjsparks.org/mailtoken/token/last_call_issued/><https://dt-test.rjsparks.org/mailtoken/token/last_call_issued/>

The recipients listed are table driven - the secretariat can change them.
Note that the actions are configured at the moment to reach more people than
the production system currently does in many cases. One of the most important
things you can do is provide feedback on whether we have the set of recipients
for a given action right. Another is whether we have the right actions listed -
in other words, are there times when we send email now that we shouldn't, and
are there other times where we aren't sending email that we should? (Note that
there are small number of places that the tracker sends email that are not yet
using this system, but if you spot a place that's not covered here, please ask
about it.)

Before brute-forcing your way through the first link above, however, let me
introduce a few other new things. If you go to a specific document, or a group,
there is now a tab on the main page that shows what the email expansions turn
into for that document or group. For example, look at
<https://dt-test.rjsparks.org/doc/draft-ietf-sipcore-refer-clarifications><https://dt-test.rjsparks.org/doc/draft-ietf-sipcore-refer-clarifications>
and note the "Email expansions" tab that takes you to
<https://dt-test.rjsparks.org/doc/draft-ietf-sipcore-refer-clarifications/email/><https://dt-test.rjsparks.org/doc/draft-ietf-sipcore-refer-clarifications/email/>

The way each recipient is computed is shown at
<https://dt-test.rjsparks.org/mailtoken/recipient/><https://dt-test.rjsparks.org/mailtoken/recipient/>
Again, you can hover over a token for a short text description, or focus on
a particular recipient using, for example,
<https://dt-test.rjsparks.org/mailtoken/recipient/doc_shepherd><https://dt-test.rjsparks.org/mailtoken/recipient/doc_shepherd>

Wherever possible, the recipient is expanded using a Django template.  When the
logic for expanding a recipient is too complicated for a template, the work is
done using a short function, as shown on the recipient page.  As we go forward,
we'll be working to simplify this gathering process so that many of the
recipients that require functions now can be moved into templates (but it will
be important to not just bury the details of the truly complicated recipients -
one of the other goals of this project is to make it less of a mystery where
mail is sent)

Now, the IESG will be particularly interested in one special case: Look at the
list of recipients for ballot_saved.  The save-and-send email form will offer
all of the addresses that are expanded from these recipients and allow the
AD to chose which recipient tokens to actually use (by default, all are
selected). When logged in as an AD or the Secretariat, go to
<https://dt-test.rjsparks.org/doc/draft-ietf-sipcore-refer-clarifications/ballot/425017/emailposition/?ad=107190><https://dt-test.rjsparks.org/doc/draft-ietf-sipcore-refer-clarifications/ballot/425017/emailposition/?ad=107190>

Here are a few other highlights from the changes made so far:
* The secretariat has been sending the internal-review and new-work
  messages manually. The Datatracker now assists with those messages.
* The mail sent when issuing ballots has been simplified
* Instead of a generic "state changed" message, recipients get a message
  that says
  - Comment has been added
  - Intended publication status changed
  - Document has been adopted by group
  - IESG is proccessing this document

Finally, there is a script that will run when this is deployed (it has been run
already on this test instance) that will scrub recipients that should be
normally copied out of the Notify field for each document. The leadership associated
with the document will get an email message explaining the change. The IESG will
get a message for all the docs that do not have currently active leadership associated
with them (that message to the IESG will be long). Currently, when that script runs it
reports: Changed 4858 documents. 3775 of those had their notify field emptied

An example of the message that gets sent is at:
<http://www.nostrum.com/~rjsparks/example_notify_email.txt><http://www.nostrum.com/%7Erjsparks/example_notify_email.txt>