[spfbis] Document shepherd review of draft-ietf-spfbis-4408bis-14

[S Moonesamy <[e-mail]>]

Hello,

Here's my review as Document Shepherd.  draft-ietf-spfbis-4408bis is 
nearly ready for publication as a Proposed Standard.  I took into 
consideration the restrictions in the SPFBIS Charter and the 
controversial working group discussions in making this assessment.

In the Abstract:

   "This document describes version 1 of the Sender Policy Framework
    (SPF) protocol, whereby an ADMD can explicitly authorize the hosts
    that are allowed to use its domain names, and a receiving host can
    check such authorization."

ADMD should be expanded in the above.  This was pointed out in a 
review posted on August 26, 2012 [1].

In Section 2.1:

   'It is RECOMMENDED that SPF verifiers not only check the "MAIL FROM"
    identity, but also separately check the "HELO" identity by applying
    the check_host() function (Section 4) to the "HELO" identity as the
    <sender>.'

As a note this "RECOMMENDED" is also in RFC 4408.

    'Note that requirements for the domain presented in the EHLO or HELO
     command are not always clear to the sending party, and SPF verifiers
     MUST be prepared for the "HELO" identity to be malformed or an IP
     address literal.'

This RFC 2119 "MUST" is not in RFC 4408.  It is a substantive change 
in my opinion.

   "This SPF check can only be performed when the "HELO" string is a
    valid fully qualified domain."

What is a valid fully qualified domain?

In Section 2.2:

   'SPF verifiers MUST check the ""MAIL FROM" identity if a completed
    "HELO" check has not reached a definitive policy result by applying
    the check_host() function to the "MAIL FROM" identity as the
    <sender>.'

As a note this RFC 2119 MUST is already in RFC 4408.

In Section 2.3:

   "An SPF-compliant domain MUST have valid SPF records as described in
    Section 3."

As a note the RFC 4408 text is:

   "An SPF-compliant domain MUST publish a valid SPF record as described
    in Section 3."

The RFC 2119 MUST is redundant.

   'If domain owners choose to publish SPF records and want to support
    receivers making negative authorization determinations, then they MUST
    publish records that end in "-all", or redirect to other records that do,
    otherwise, no definitive determination of authorization can be made.'

The document uses ADMD while there is "domain owners" in the 
above.  I suggest reviewing that.

The RFC 2119 MUST is a substantive change.  Why is this a MUST?

In Section 2.4:

   'At least the "MAIL FROM" identity MUST be checked, but it
    is RECOMMENDED that the "HELO" identity also be checked beforehand.'

There is already a RFC 2119 MUST in Section 2.2.  There is also a RFC 
2119 RECOMMENDED in Section 2.1.  The above text restates existing 
RFC 2119 requirements or recommendations.

   'Without explicit approval of the domain owner, checking other
    identities against SPF version 1 records is NOT RECOMMENDED because
    there are cases that are known to give incorrect results.'

How should this explicit approval be sought?  This recommendation 
does not make sense.

In Section 2.4:

   'When a mail receiver decides to perform an SPF check, it MUST use a
    correctly-implemented check_host() function (Section 4) evaluated
    with the correct parameters.'

This RFC 2119 requirement states the obvious.  It basically says that 
there is a requirement in draft-ietf-spfbis-4408bis-14 to use a 
correct implementation of draft-ietf-spfbis-4408bis-14.

   'Although invalid, malformed, or non-existent domains cause SPF checks
    to return "none" because no SPF record can be found, it has long been
    the policy of many MTAs to reject email from such domains, especially
    in the case of invalid "MAIL FROM".  Rejecting email will prevent one
    method of circumventing of SPF records.'

This text is already in RFC 4408.

   "Implementations have to take care to correctly extract the <domain>
    from the data given with the SMTP MAIL FROM command as many MTAs will
    still accept such things as source routes ..."

The definition of <domain> is:

   'the domain portion of the "MAIL FROM" or "HELO" identity.'

I am not sure whether it is even a definition.  From an 
implementation perspective the last paragraph of Section 2.4 is unclear.

In Section 2.5:

   "The authorization check SHOULD be performed during the processing of
    the SMTP transaction that sends the mail."

This RFC 2119 SHOULD is already in RFC 4408.

   'Performing the authorization other than using the return-path and
    client address at the time of the MAIL command during the SMTP
    transaction can cause problems, ..'

Shouldn't that be "authorization check"?

   'Generating non-delivery notifications to forged identities that have
    failed the authorization check is a source of backscatter and SHOULD
    be avoided.'

This RFC 2119 SHOULD is not in RFC 4408.  The above does not have 
anything to do with Sender Policy Framework.  It was mentioned during 
WG discussions that "SPF can help the backscatter problem" [2].  This 
text may have been introduced in response to a review [3].  Is this 
an enhancement or a clarification?

In Section 2.6:

   "An SPF verifier implements something semantically identical to the
    function defined there."

John Levine commented about this during the WGLC [4].  I am listing 
this as a reminder for myself.

In Section 2.6.1:

   'A result of "none" means either (a) no syntactically valid DNS domain
    name was extracted from the SMTP session that could be used as the
    one to be authorized, or (b) no TXT records were retrieved from the
    DNS that appeared to be intended for use by SPF verifiers.'

I suggest "DNS TXT records".  What is a syntactically valid DNS 
domain name?  The definition of "none" in RFC 4408 is clear (to me).

In Section 2.6.2:

   'The domain owner has explicitly stated that it is not asserting
    whether the IP address is authorized.  This result MUST be treated
    exactly like the "none" result; the distinction exists only for
    informational purposes.'

As a note, the RFC 2119 MUST is in RFC 4408.  The Introduction 
Section mentions "ADMDs can authorize hosts to use their domain 
names".  The above uses "domain owner".

In Section 2.6.7:

   'A "permerror" result means the domain's published records could not
    be correctly interpreted.  This signals an error condition that
    definitely requires manual intervention to be resolved.'

What is "manual intervention" in the above?

In Section 3:

   'An SPF record is a DNS record that declares which hosts are, and are
    not, authorized to use a domain name for the "HELO" and "MAIL FROM"
    identities.'

How are hosts which are not authorized to use a domain name listed in 
a SPF record?

   "The SPF record is a single string of text."

What is a single string of text?  It may be easier to state this in 
terms of record format.

   'ADMDs publishing SPF records SHOULD try to keep the number of
    "include" mechanisms and chained "redirect" modifiers to a minimum.'

What is the minimum?  Note that this RFC 2119 SHOULD is in RFC 4408.

   'ADMDs SHOULD also try to minimize the amount of other DNS information
    needed to evaluate a record.'

This RFC 2119 SHOULD is also in RFC 4408.  There are two RFC 2119 
SHOULDs in the last paragraph of Section 3.  This usage of RFC 2119 
key words does not help the SPF "publisher".   In my opinion the 
"SHOULD try" in this context uses the RFC 2119 key word in unorthodox ways.

In Section 3.1:

   'SPF records MUST be published as a DNS TXT (type 16) Resource Record
    (RR) [RFC1035] only.'

I would ask why "MUST be published ... only".  By the way, Section 3 
has a reference to Section 4 for record format instead of Section 
3.1.  Is that correct?

In Section 3.2:

   "A domain name MUST NOT have multiple records that would cause an
    authorization check to select more than one record."

This RFC 2119 MUST NOT was in RFC 4408.  Is this to help the SPF 
"publisher" and if so, how does it help?

In Section 3.3:

   "As defined in [RFC1035] sections 3.3.14 and 3.3, a single text DNS
    record can be composed of more than one string."

See previous comment about " a single string".

   "If a published record contains multiple character-strings, then the
    record MUST be treated as if those strings are concatenated together
    without adding spaces."

This RFC 2119 MUST is also in RFC 4408.  I suggest removing the 
"MUST" in the example.

   "TXT records containing multiple strings are useful in constructing
    records that would exceed the 255-byte maximum length of a character-
    string within a single TXT record."

I suggest using "octet" instead of "byte".  I'll point the working 
group to CVE-2008-2469 in case it might wish to consider that issue.

In Section 4:

   "A compliant SPF implementation MUST do something semantically equivalent to
   this description."

It's unusual to see a "do something" RFC 2119 requirement in a 
specification.  Is the SPFBIS WG working on compliance for SPF implementations?

   "Mail receivers that perform this check MUST correctly evaluate the
    check_host() function as described here."

Where does "mail receivers" fit in the Sender Policy Framework?  The 
"MUST correctly evaluate" is stating the obvious.

   "Implementations MAY use a different algorithm than the canonical
    algorithm defined here, so long as the results are the same in all
    cases."

Why is this a RFC 2119 MAY?  I am aware that the text is in RFC 4408.

In Section 4.1:

   '<domain> - the domain that provides the sought-after authorization
               information; initially, the domain portion of the "MAIL
               FROM" or "HELO" identity.'

The above text is different from the text in Section 2.4.

In Section 4.3:

   "If the <domain> is malformed (e.g. label longer than 63 characters,
    zero-length label not at the end, etc.)"

  That should be "octets".

   "Properly formed domains are fully qualified email domains as
    described in [RFC5321] Section 2.3.5."

What are "properly qualified email domains"?

   "Internationalized domain names MUST be encoded as A-labels, as
   described in Section 2.3 of [RFC5890].on 2.3 of [RFC5890]."

I'm listing the above and leave it to Area Director guidance.  Please 
note that it is a new RFC 2119 requirement.

In Section 4.4:

   "In accordance with how the records are published (see Section 3
    above), a DNS query needs to be made for the <domain> name, querying
    for type TXT only."

I don't understand the sentence.

   'Alternatively, for a server failure (RCODE 2) result, check_host()
    MAY track failures and treat multiple failures within 24 hours for
    the same domain as "permerror".'

This text is not in RFC 4408.  I found a note in Issue #1 [5] and in 
a message [6].

In Section 4.6.2:

   "If there are no more mechanisms, the result is specified in
    Section 4.7."

This sentence does not make sense.

In Section 4.6.4:

   "SPF implementations MUST limit the total number of mechanisms and
    modifiers ("terms") that cause any DNS query to at most 10 during SPF
    evaluation."

This was discussed on the mailing list [7].

   "If this number is exceeded during a check, a permerror MUST be returned."

I read this as if an implementation-specific limit is reached a 
permerror is returned.  There are two RFC 2119 MUST in the 
above.  That can be reduced to one MUST.

I read the first paragraph of Section 4.6.4.  I am not sure how the 
absolute requirement will be interpreted by the reader.

   'MTAs or other processors SHOULD impose a limit on the maximum amount
    of elapsed time to evaluate check_host().  Such a limit SHOULD allow
    at least 20 seconds.  If such a limit is exceeded, the result of
    authorization SHOULD be "temperror".'

There are three RFC 2119 SHOULDs in the above.  I suggest rewriting 
the text to reduce that.

   'SPF implementations SHOULD limit "void lookups" to two.'

What are void lookups?

   "In this case, a default of two is RECOMMENDED."

Why is "two" recommended?

   'Note that records SHOULD always use either a "redirect" modifier or
    an "all" mechanism to explicitly terminate processing.'

Why is there a RFC 2119 key word in a note?

In Section 4.8:

   'e.g., "foo..example.com") or overlong labels (more than 63
    characters).'

I suggest octets.

in Section 5:

   'SPF implementations on IPv6 servers need to handle both "AAAA" and "A"
    secords, for clients on IPv4 mapped IPv6 addresses [RFC4291].'

There is a typo, "records".

I'll flag the above for review by people with IPv6 expertise.

In Section 5.3:

   'a   = "a"      [ ":" domain-spec ] [ dual-cidr-length ]'

"dual-cidr-length" is introduced in this sub-section.  I had to 
search through the draft to find out what it means.

In Section 5.4:

   'Note regarding implicit MXs: If the <target-name> has no MX records,
    check_host() MUST NOT pretend the target is its single MX, and MUST
    NOT default to an A or AAAA lookup on the <target-name> directly.'

There are two RFC 2119 MUSTs in the above.  This is over-usage of RFC 
2119 key words.  This text was in RFC 4408.  This is not a divergence 
from RFC 5321, it is a contrary to Section 5 of RFC 5321.

In Section 5.5:

  "This mechanism SHOULD NOT be used."

I suggest providing a reason for this.

   "To prevent DoS attacks, more than 10 PTR names
    MUST NOT be looked up during the evaluation of a "ptr" mechanism
    (see Section 4.6.4)."

The above restates a previous RFC 2119 MUST.

   "If used, proper PTR records MUST be in place for the domain's hosts
    and the "ptr" mechanism SHOULD be one of the last mechanisms checked."

Those RFC 2119 key words are not in RFC 4408.  I don't see how this 
RFC 2119 change qualifies as a clarification or explanation.

   "It is, however, still in use and part of the SPF protocol, so compliant
   check_host() implementations MUST support it."

What is a compliant check_host() implementation?

In Section 5.6:

   "ip6-network      = <as per [RFC 4291], section 2.2>"

I suggest that the above reference be verified for correctness.

In Section 5.7:

   'v=spf1 exists:%{ir}.%{l1r+-}._spf.%{d} -all'

I'll flag this for review by DNS folks.

In Section 6:

   'The modifiers defined in this document ("redirect" and "exp") MAY
    appear anywhere in the record, but SHOULD appear at the end, after
    all mechanisms.'

This text is in RFC 4408.  I would label the RFC 2119 usage as defective.

In Section 6.2:

   "Since the explanation string is intended for an SMTP response and
    [RFC5321] Section 2.4 says that responses are in [US-ASCII], the
    explanation string MUST be limited to US-ASCII.'

It would be easier to defer to RFC 5321 instead of setting a RFC 2119 
requirement in this draft.  I note that this requirement was not in RFC 4408.

   "Software SHOULD make it clear that the explanation string
    comes from a third party."

I could not understand what that means in RFC 2119 terms.  The draft 
uses RFC 2119 key words by example instead of providing an explanation.

In Section 7.1:

   'The "toplabel" construction is subject to the LDH rule plus
    additional top-level domain (TLD) restrictions.'

Where can I find these restrictions?  Please note that I have read 
the Informational RFC.

In Section 7.3:

   "-exists:%(ir).sbl.spamhaus.example.org"

I commented about vendor-neutral previously [8].  The above is a way 
to get around the objection [9].  I suggest cleaning this up.

  "so trailing dots SHOULD NOT be published by domain owners"

Why is it "domain owners"?

   "This macro SHOULD NOT be used.  See Section 5.5 for the discussion
    about why not."

This comes out as a flippant explanation for the RFC 2119 SHOULD.

   "This SHOULD be a fully qualified domain name, but if one does not
    exist (as when the checking is done by a MUA) or if policy restrictions
    dictate otherwise, the word "unknown" SHOULD be substituted."

The RFC 2119 key words are in RFC 2119.  I don't know what to say.

   "When the result of macro expansion is used in a domain name query, if
    the expanded domain name exceeds 253 characters (the maximum length
    of a domain name), the left side is truncated to fit, by removing
    successive domain labels (and their following dots) until the total
    length does not exceed 253 characters."

Where is it specified that the maximum length of a domain name is 253 
characters?

   "Care has to be taken by the sending ADMD so that macro expansion for
    legitimate email does not exceed the 63-character limit on DNS labels."

Where is that 63-character limit specified?

It is odd to have RFC 2119 requirements under a "Notes" heading.

In Section 8.2:

   'A "neutral" result MUST be treated exactly like the "none" result;
    the distinction exists only for informational purposes.'

Why is an existing RFC 2119 restated?

In Section 8.5:

   "The domain owner believes the host is not authorized but is not willing
    to make a strong policy statement."

Why is it domain owner?

In Section 9:

   "An operator could choose to use both to serve different downstream
    agents.  In such cases, care needs to be taken to ensure both fields
    are conveying the same details, or unexpected results can occur."

This has been discussed on the mailing list.  I don't think that it 
encourages interoperability but the working group decided otherwise [10].

In Section 9.1:

   "The Received-SPF header field is a trace field (see [RFC5322] Section
    3.6.7) and SHOULD be prepended to the existing header, above the
    Received: field that is generated by the SMTP receiver."

"prepended to the existing header" does not sound right.

   "It MUST appear above all other Received-SPF fields in the message."

How does this fit into the previous RFC 2119 SHOULD?

in Section 10:

   "This section is not a "how-to" manual, or a "best practices" document,
    and it is not a comprehensive list of what such entities SHOULD do in
    light of this document."

What is the meaning of the RFC 2119 SHOULD in the above?

In Section 10.1.1:

There was a comment from Authur Thisell about the table [11] in this 
sub-section.

In Section 10.1.2:

   "Publishing SPF records for domains that send no mail is
    a well established best practice."

I am not aware of that best practice.  I did a few DNS queries:

;; QUESTION SECTION:
;bing.com.                      IN      TXT

;; ANSWER SECTION:
bing.com.               3600    IN      TXT     "v=msv1 
t=6097A7EA-53F7-4028-BA76-6869CB284C54"

;; QUESTION SECTION:
;com.com.                       IN      TXT

;; ANSWER SECTION:
com.com.                293     IN      TXT 
"google-site-verification:esSnoZdChIkkfCfS9MMgqNhE0yaXfnnZWdZPuBf7e8k"

In Section 10.1.3:

   "SPF functionality is enhanced by administrators ensuring this
   identity is set correctly and has an appropriate SPF record."

How is SPF functionality enhanced?

In Section 10.2:

   "As stated elsewhere in this document, there is no normative requirement
    for specific handling of a message based on any SPF result."

The "as stated elsewhere" is vague.

   'The primary considerations are that SPF might return "pass" for mail
    that is ultimately harmful (e.g., spammers that arrange for SPF to
    pass using nonsense domain names'

What are "nonsense" domain names?

In Section 10.3:

   "The mediator can make the newly-posted message be as similar or as
    different from the original message as they wish."

The sentence seems incomplete, i.e. similar to what.

   "For the operation of SPF, the essential concern is the email
    address in the 5321.MailFrom command for the new message."

What does this mean?

   'Because SPF evaluation is based on the IP Address of the "last"
    sending SMTP server, the address of the mediator will be used, rather
    than the address of the SMTP server that sent the message to the
    mediator.'

I'll note that this is a mix of RFC 5321 and RFC 5598.  The result is 
clear or unclear depending on the background of the reader.

In Section 11.5:

   "An SPF compliant receiver gathers information from the SMTP commands
    it receives and from the published DNS records of the sending domain
    holder"

I suggest explaining the untrusted part.

In Section 11.6:

   "Checking SPF records causes DNS queries to be sent to the domain
    owner."

How are DNS queries sent to domain owners?

Section 12:

I note that Murray S. Kucherawy has contributed significantly to 
draft-ietf-spfbis-4408bis.  If I am not mistaken it is IETF practice 
to acknowledge such contributions.  I don't recall who sent text at 
the moment.  If you did, please send an (off-list) email to the 
SPFBIS WG Chairs.

In Section 13.1:

   "The format of this type is identical to the TXT RR [RFC1035]"

The format is not identical, i.e. it's a different RR.  I suggest 
keeping the IANA Considerations Section simple, i.e. clearly explain 
to IANA what action is requested.

In Section 13.2:

   "Per [RFC3864], the "Received-SPF:" header field is added to the IANA
    Permanent Message Header Field Registry."

Note to self: request removal from the provisional message  header 
field registry.

The "status:" should be "standard"

In Section 13.3:

   "Their status should not be changed."

I suggest "Their status is unchanged".  BTW, it should be "SPF 
Modifier Registry".  I suggest following RFC 6652 Section 5.1.

An "Updates:" for RFC 6652 may be needed.

References:

Why is RFC 1123 a normative reference?

Why is RFC 3864 a normative reference?

Where can I find "Designated Mailers Protocol"?

Where can I find "Domain-Authorized SMTP Mail"?

ABNF:

Did anyone verify the ABNF?

In Appendix C:

In Section E.1:

   'This would cause a lookup on an DNS white list (DNSWL) and cause a
    result of "fail" only for email not either coming from the
    domain's mx host(s) (SPF pass) or white listed sources (SPF
    neutral).'

I didn't find any discussion about this on the SPFBIS mailing 
list.  is there an explanation for this change between RFC 4408 and this draft?

I note that there are 48 pages in RFC 4408.  There are 78 pages in 
this draft.  A significant amount of text has been added in the 
Appendix (over 20 pages).  I doubt that the text has been carefully 
reviewed.  I may be asked for an explanation about all that once the 
draft leaves this working group.

In Appendix I:

Appendix I is about "Protocol Status".  This draft is intended as a 
Proposed Standard. From an IETF perspective that is what it will 
be.  Describing it as something different can be misleading.

   "[RFC4408] was designed to clearly document the protocol defined by
    earlier draft specifications of SPF as used in existing
    implementations.  This updated specification is intended to clarify
    identified ambiguities in [RFC4408], resolve techincal issues
    identified in post-RFC 4408 deplyment experience, and document widely
    deployed extensions to SPF that have been developed since [RFC4408]
    was published."

Extensions to the SPF protocol are clearly mentioned in the charter 
as being out of scope.  The "document widely deployed extensions" is 
problematic.

   "This document updates and replaces RFC 4408 that was part of a group
    of simultaneously published Experimental RFCs (RFC 4405, RFC 4406,
    RFC 4407, and RFC 4408) in 2006.  At that time the IESG requested the
    community observe the success or failure of the two approaches
    documented in these RFCs during the two years following publication,
    in order that a community consensus could be reached in the future."

Why is this needed?  The SPFBIS WG produced RFC 6686 to resolve the experiment.

   "SPF is widely deployed by large and small email providers alike.
    There are multiple, interoperable implementations."

Someone might point out that this is marketing instead of text 
relevant to a protocol.  I'll mention that one or more significant 
issues (e.g. Issue #9) was identified during the WG 
discussions.  Issue #9 was not listed as the errata.  I suggest 
removing the section as it will be less text and there is already two 
informative references to help the reader find background information.

Regards,
S. Moonesamy (as document shepherd)

1. http://www.ietf.org/mail-archive/web/spfbis/current/msg02371.html
2. http://www.ietf.org/mail-archive/web/spfbis/current/msg00928.html
3. http://www.ietf.org/mail-archive/web/spfbis/current/msg01810.html
4. http://www.ietf.org/mail-archive/web/spfbis/current/msg03378.html
5. http://trac.tools.ietf.org/wg/spfbis/trac/ticket/1
6. http://www.ietf.org/mail-archive/web/spfbis/current/msg00630.html
7. http://www.ietf.org/mail-archive/web/spfbis/current/msg00199.html
8. http://www.ietf.org/mail-archive/web/spfbis/current/msg02279.html
9. http://www.ietf.org/mail-archive/web/spfbis/current/msg02308.html
10. http://www.ietf.org/mail-archive/web/spfbis/current/msg02526.html
11. http://www.ietf.org/mail-archive/web/spfbis/current/msg02763.html