Re: header-munging

[Einar Stefferud <[e-mail]>]

After reviewing this "header-munging" discussion, it is very clear to
me that the new "submit" protocol has long been needed to support our
less capable MUA systems, but also that it is quite independent of,
and need not be embedded in the SMTP standards or in SMTP MTA
implementations.

Internet EMail SUBMIT (lets call it SMSP for Simple Mail Submit
Protocol) should be defined as a separate protocol that specified how
to submit less than completely ready RFC822/821 messages, and makes
them ready for SMTP relaying.

The new protocol should be mounted on a NEW PORT NUMBER.  SMTP and
SMSP need not be implemented as a single body of code, or even appear
to be connected, or even run on the same host.  On the other hand, it
should not matter if some implementations combine all the code into a
single monolith so that one cannot separate them into independent code
modules.

The key is that the behavior on the SMTP PORT 25 must be that of a
simple nonrewriting relay as specified by SMTP, and the behavior on
the new SMSP PORT (n) is that of what the final stages of any proper
MUA would be if those services were implemented in a monolithic MUA.

Thus, those current SMTP MTA implementations that are capable of
doctoring up incomplete and non-conformant RFC822/821 "submissions"
can make a few modifications to distinguish SUBMIT from RELAY using
different port numbers, and meet the new requirements; and other
implementors can simply build free standing SMSP servers using the new
SMSP port number to accept input and use SMTP to relay output
messages.

All this is amazingly simple as soon as we have agreed upon the new
SMSP protocol, and it can all be implemented in various ways using
many tools that already exist, but also opening up the market for
other alternatives designed to meet the specific needs of various
currently messed up EMail enclaves, by permitting specialized
free-standing SMSP servers to be placed where needed, perhaps in
firewalls, or where they make sense, with or without embedded MTA
functions.  Among the functions that should be considered is
authentication of the submitting MUA for each submitted message.

A key to understanding all this is that of keeping our Abstract Models
separate from our MUA and MTA Implementations.  It does not matter how
much you "monolithize" combinations of protocol implementations as
long as you do not allow any violations of the protocol specifications
to be visible outside your implementation (on the wire).

Our new SMSP is actually a part of the Abstract Model concept of the
MUA, which has always been obligated to properly present conformant
messages to SMTP ports for RELAY SERVICE.  It is an MUA responsibility
to submit correct messages for SMTP to relay!  It has never been
otherwise in the specs.

That Sendmail chose to combine some MUA functions in its
implementation is of no consequence as long as "on the wire" we cannot
detect that sendmail is violating the rules of SMTP replay services.

This "header munging" discussion has documented the fact that sendmail
and other MTA implementations are in fact, visibly "on the wire",
performing non-conformant services by "munging message headers" in
ways that are not correct and are also unacceptable.  Thus we have
visible protocol violations occurring with no way to control them.

The reason for all of this is that over time, our Internet-Wide lack
of attention to the requirements of good separation of MUA and MTA
functions has allowed "sendmail" to be mistaken for the "standard".
(We have never written down the "ABSTRACT MODEL" concept for all to
see and understand.  We seem to be more interested in the end product
that the correctness of our embedded abstract models!-)...

What all this is now about is to define a missing bit of EMail
Protocol for the New Message SUBMIT function as part of the MUA
functional responsibility.

This has always been (in our unwritten abstract model) an MUA
responsibility, and in many ways it was a great kindness for sendmail
to gratuitously provide submit services, but in the larger scope of
the global internet, it now also provides some gratuitous services
when it should not, and we need to tidy up the Internet EMail
Submission rules to clean up what is becoming an untenable mess.

In short, SUBMISSION of correctly formatted messages is now and has
always been an MUA responsibility according to the abstract EMail
model.  What sendmail and others have been doing to fix badly
formatted submissions is being done with "MUA Authority", and we are
now talking about defining a module of MUA functionality so we can
split it off from MUA and MTA implementations with a defined protocol
for the handover interface between an MUA and its SUBMIT process.
(The interface for its output is already well defined as SMTP!-)...

This will allow a "partially compete" MUA to hand off its mail
submission responsibilities to a separate process running on a
computer of its choice, which will ensure that its EMAil will be
Internet Conformant when it is injected into the Internet Mail Relay
infrastructure as defined by SMTP.

SMTP does not define or specify what should be done to messages that
are not conformant, and I believe it never should.  If anything, SMTP
implementations should just non-deliver faulty mail if they do not
want to admit that they are performing MUA functions when they fix up
received mail.

All this discussion about how an MTA should know that it is being
asked to behave as a SUBMIT vs RELAY server are, in my view, bogus,
and should be resolved with the use of separate PORT NUMBERS for the
two functions, with no fall back from one to the other.  They are
totally separate functions and one cannot be substituted for the
other.

This is not an SMTP extension!  It is an MUA split!

We are cutting a chunk of MUA functionality out and calling it
"SUBMIT", because this bit of functionality is very difficult for some
MUA "clients" to perform.  This is fine, but separating it out is best
called splitting -- It is not properly called extending.

Looking back, POP was another case of splitting the receiving function
away from some MUA implementations (though it also looks like
splitting delivery away from the MTA), with the "maildrop" on one host
and the mail handling tools and folders and such on another host.
This dates all the way back to some work done by myself and some
students at UCI in 1984 (called MZnet) which framed the "split UA"
abstract model for POP as defined and then implemented by Marshall
Rose.

Ever since then (only 12 years), it has been clear that we also need
the obverse of POP, called SUBMIT, to be defined as another split off
chunk of the abstract MUA model to buffer our less capable MUA
platforms from SMTP for submission purposes.

So, lets just separately define SMSP and insert it into the Internet
Mail Infrastructure and let it all self organize over time to buffer
bad MUAs from our Internet RELAY service.

We just need to define this bit of split of MUA, and stop talking
about Extending SMTP.

Best...\Stef