Re: header-munging

[Ned Freed <[e-mail]>]

> > Our job in the standards business is to define the ABSTRACT model and
> > the interactions between ABSTRACT model components.

> > Out job is NOT to decide anything about how an implementation might
> > actually make it happen, as long as no ABSTRACT MODEL violations can
> > be detected outside the boundaries of the code that does the work.

> I strongly disagree.

> Our job is to make the net work well, using such tools as we have at
> our disposal.  (Our tools tend to be published documents, but also
> include hallway discussions, technical meetings, private email, and
> other mechanisms for sharing information.)  We don't have to
> artifically limit ourselves to writing standards documents in terms of
> abstract models.

Folks, I think the key point to all this isn't how we define or instantiate the
abstract model, it is in which sorts of fixups are practical in a
split-composition setup and which ones aren't. Specifically, I read Keith's
position as not necessarily in opposition to fixups per se, but as being in
opposition to our attempting to fix things that a separate component from the
user agent isn't likely to get right.

> Case in point: There's nothing wrong with the abstract model we are
> currently using for email.  The problem we have -- that MUAs don't
> generate correctly formatted messages -- isn't due to a failing of the
> abstract model, but due to the practical difficulty of supplying the
> MUA with the correct information that it needs to do the job right.

I agree with this, but in order to understand the issues you have to get down
to the specific fixups that are envisioned. The list in Harald and Randy's
draft is a good start. I'm reproducing it here, with comments of my own after
each item.

     (3)   Add a 'Sender' field to the submitted message, if it knows
           the identity of the sender and this is not given in the
           'From' field.

In some configurations you can infer the proper 'Sender' field from the IP
address of the client and possibly from the use of some sort of authentication
mechanism used under the SMTP connection. However, in many configurations this
simply isn't possible, and as such a submission protocol does not provide a
general solution to this (very real and very vexing) problem.

As such, while I have no problem with implementing facilities along these
lines, I believe the real solution is to provide a means by which user agents
can obtain a definitive form of this information, e.g. as part of the ACAP
protocol. I am also afraid that attempting to solve this problem in a
submission protocol may take energy away from engineering a real solution
to the problem.

     (4)   Add a 'Date' field to the submitted message, if it lacks it.

Adding a 'Date' field when it is missing is better than not having one at all.
But as Keith has already pointed out, the RFC822 'Date' field is the creation
date for the message, not the posting date, and as such this really doesn't
fix the problem properly.

     (5)   Correct the 'Date' field if it does not conform to [RFC-822]
           syntax (as modified by [RFC-1123]).

Again, fixing a broken 'Date' field (e.g. one that lacks a time zone) is better
than leaving it broken, but there is no guarantee that the submission server
will know how to fix a given sort of bust in a 'Date' field. I believe that
at a minimum there needs to be a note added to the effect that this field
wasn't generated by the originating agent, and thus doesn't necessarily
reflect the creation time of the message.

     (6)   Add a 'Message-ID' field to the submitted message, if it
           lacks it.

I see no downside to this.

     (7)   Transfer encode the message according to MIME conventions,
           if desirable and needed.

I see no downside to this either.

     (8)   Resolve aliases (CNAME records) for domain names, in the
           envelope as well as the text, subject to local policy.  For
           example, if www.ab.com and ftp.ab.com are both aliases for
           mail.ab.com, rewriting them could lose useful information.

I have real problems with the use of CNAMEs, period, but modulo those concerns
I see no real downside here. Use of shortform names is also something that can
be dealt with correctly by submission servers in almost every case.

     (9)   Rewrite local parts, according to local policy.  For example,
           a site may prefer to rewrite JRU as J.Random.User in order
           to hide logon names.

I see no downside to this either, but I'm also not sure that this is really
restricted to messages sent to the submission servers. Sites that implement
centralized naming schemes generally want the scheme applied to all messages
all the time.

    (10)   Add a 'Content-MD5' field to the submitted message, if it
           lacks it.

I'm not entirely comfortable with this, as it implies that the message
hasn't been corrupted in the process of getting it to the submission
server. I guess this would be OK if the submission server noted where the
checksum field had been added.

So where does all this leave us? It leaves us with a non-null class of problems
that a submission server can be called upoin to solve. It also leaves us with a 
non-null class of problems a submission server isn't likely to be able to solve
with sufficient degree of generality to be called a real solution to the
underlying problem.

Whether or not the set of problems solvable with a submission server is large
enough and the set of problems not solvable with a submission server small
enough to go to the effort of specifying such a thing is the key decision
we have to make here.

> Splitting the MUA into two abstract components -- one of which
> generates a message on the basis of incomplete or inaccurate
> information and the other of which fixes it up -- isn't going to help,
> UNLESS we somehow significantly increase the liklihood that the
> component that fixes things up has more accurate information than
> present-day MUAs.

Alternately, unless we make reasonably sure that the submission server only
attempts the fixups it is actually in a position to do correctly.

> NONE of the proposals which use SMTP in any way to do "fixup" of
> submitted messages are ANY more likely (certainly not "significantly"
> more likely) to produce correct messages than the status quo.

Hmm. Well, I'm not sure I agree with this. Some fixups do make sense, others
do not, at least not in general. (Note that practically any fixup you
can think of will make sense in a sufficiently limited context and with
sufficiently strong bilateral agreements in place.)

> We can twiddle the abstract model all we want, but if IN PRACTICE, the
> guy doing the fixup still lacks the correct information, we will not
> have improved the reliability of email.  We'll have succeeded only in
> creating a new way to talk about the same old problem (we'll blame a
> different component of the system), and perhaps even in making the
> problem worse than it is now.

Exactly. The key here is what information is available to the various
components. In some cases a submission agent is far less likely to have
the necessary information than the originating user agents, and in these
cases we aren't helping matters by trying to lift a part of the burden
that can only be carried by the originating user agent.

> On the other hand, if we want to improve the reliability of email, we
> need to arrange things so that the component that generates the
> message that goes out on the wire, is likely IN PRACTICE to have all
> of the information that it needs to do so.  That means that the parts
> of the message which are dependent on the sender's environment, need
> to be somehow supplied by the sender's environment.

> The whole approach of transmitting an incomplete 822 message over a
> network connection, and having it be "fixed up" by the entity at the
> other end of that connection, makes it likely that the "fixup guy"
> will be in a different environment than the sender, and thus likely
> that the "fixup guy" will not have the information it needs.  So the
> whole approach is likely to lose.

Again, I think this goes too far. I agree that there is a class of information
that the submission server is less likely to have than the originating agent,
and thus Keith's objection is well founded. But I don't believe that all
information a submission server could provide necessarily falls into this
category.

				Ned