Re: [spfbis] Document shepherd review of draft-ietf-spfbis-4408bis-14

[Scott Kitterman <[e-mail]>]

On Thursday, May 16, 2013 06:36:14 PM S Moonesamy wrote:
> Hi Scott,
> 
> Murray Kucherawy, Alessandro Vesely, John Levine and Stuart Gathman
> commented about this review.  I'll cover these comments in my
> response.  By the way if I missed any of the comments please let me know.
> 
> I'll follow the guidance in the message at
> http://www.ietf.org/mail-archive/web/spfbis/current/msg03642.html for
> RFC 2119 key words.
> 
> At 14:45 16-05-2013, Scott Kitterman wrote:
> > >    "This SPF check can only be performed when the "HELO" string is a
> > >    
> > >     valid fully qualified domain."
> > > 
> > > What is a valid fully qualified domain?
> >
> >Would s/domain/domain name/ clear it up for you?
> 
> I'll suggest the following:
> 
>    This SPF check can only be performed when the "HELO" string is a
>    fully-qualified domain name.

Done.

> > > In Section 2.2:
> > >    'SPF verifiers MUST check the ""MAIL FROM" identity if a completed
> > >    
> > >     "HELO" check has not reached a definitive policy result by applying
> > >     the check_host() function to the "MAIL FROM" identity as the
> > >     <sender>.'
> > > 
> > > As a note this RFC 2119 MUST is already in RFC 4408.
> > > 
> > > In Section 2.3:
> > >    "An SPF-compliant domain MUST have valid SPF records as described in
> > >    
> > >     Section 3."
> > > 
> > > As a note the RFC 4408 text is:
> > >    "An SPF-compliant domain MUST publish a valid SPF record as described
> > >    
> > >     in Section 3."
> > > 
> > > The RFC 2119 MUST is redundant.
> >
> >Is there a reason to change it?
> 
>  From a previous comment:
> 
>    'In a recent conversation, Pete described this as "not really wrong, but
>     it's shouting." I guess that means changing "MUST publish" to
> "publishes" or something like that.
> 
>    I am ok with the "publishes".'
> 
> I suggest using that.

Done.

> > >    'If domain owners choose to publish SPF records and want to support
> > >    
> > >     receivers making negative authorization determinations, then they
> > >     MUST
> > >     publish records that end in "-all", or redirect to other records
> > >     that
> > > 
> > > do, otherwise, no definitive determination of authorization can be
> > > made.'
> > > 
> > > The document uses ADMD while there is "domain owners" in the
> > > above.  I suggest reviewing that.
> >
> >Changed
> >
> > > The RFC 2119 MUST is a substantive change.  Why is this a MUST?
> >
> >Because it's the literal truth.  That's what you have to do to achieve that
> >result.  You won't interoperate with receivers in that way effectively if
> >you don't.
> 
> I mentioned this in another message:
> 
> Andrew Sullivan, as an individual, commented about the text (
> http://www.ietf.org/mail-archive/web/spfbis/current/msg02784.html
>   ). I suggest going with what was suggested or else the working
> group will be discussing Issue #33 again.

OK.  I still find it confusing that things that are needed for the protocol to 
function are not considered requirements, but changed.

> > > In Section 2.4:
> > >    'At least the "MAIL FROM" identity MUST be checked, but it
> > >    
> > >     is RECOMMENDED that the "HELO" identity also be checked beforehand.'
> > > 
> > > There is already a RFC 2119 MUST in Section 2.2.  There is also a RFC
> > > 2119 RECOMMENDED in Section 2.1.  The above text restates existing
> > > RFC 2119 requirements or recommendations.
> >
> >Modified to make them non-redundant.
> 
> I suggest including the proposed text for any changes so that the
> working group is aware of the text will be added to the revision of the
> draft.

That's not always easy to do.  In this case, it would be hard to evaluate out 
of context, so I think it's best to wait for the next revision.

> > >    'Without explicit approval of the domain owner, checking other
> > >    
> > >     identities against SPF version 1 records is NOT RECOMMENDED because
> > >     there are cases that are known to give incorrect results.'
> > > 
> > > How should this explicit approval be sought?  This recommendation
> > > does not make sense.
> >
> >What I have now is "Documents that define other identities will have
> >to define
> >the method for explicit approval."
> 
> "out-of-band" or "private" was suggested.  I'll use one of them (I do
> not have any preference):

>      Without out-of-band approval of the ADMD, checking other identities
>      against SPF version 1 records is NOT RECOMMENDED because there are
>      cases that are known to give incorrect results.

I don't think that's correct.  One could get a new modifier added to the SPF 
modifier registry that would indicate a record can be used for other purposes.  
There are no such other purposes defined at the moment, but there's absolutely 
no need for out-of-band/private methods to accomplish this goal.

> > > In Section 2.4:
> > >    'When a mail receiver decides to perform an SPF check, it MUST use a
> > >    
> > >     correctly-implemented check_host() function (Section 4) evaluated
> > >     with the correct parameters.'
> > > 
> > > This RFC 2119 requirement states the obvious.  It basically says that
> > > there is a requirement in draft-ietf-spfbis-4408bis-14 to use a
> > > correct implementation of draft-ietf-spfbis-4408bis-14.
> >
> >This was changed already due to other comments.  I think it makes sense
> >now. Please check when -15 is published.
> 
> The comment I read on the mailing list is:
> 
>    "I agree, that sentence can probably go."
> 
> The other comment was from Alessandro.  Does your comment mean that
> the sentence will be removed or does it mean that text is being proposed?

It means it was changed based on other comments.  Please look at -15 and see 
if you still think it's a problem.

> > >    'Although invalid, malformed, or non-existent domains cause SPF
> > >    checks
> > >    
> > >     to return "none" because no SPF record can be found, it has long
> > >     been
> > >     the policy of many MTAs to reject email from such domains,
> > >     especially
> > >     in the case of invalid "MAIL FROM".  Rejecting email will prevent
> > >     one
> > >     method of circumventing of SPF records.'
> > > 
> > > This text is already in RFC 4408.
> > > 
> > >    "Implementations have to take care to correctly extract the <domain>
> > >    
> > >     from the data given with the SMTP MAIL FROM command as many MTAs
> > >     will
> > >     still accept such things as source routes ..."
> > > 
> > > The definition of <domain> is:
> > >    'the domain portion of the "MAIL FROM" or "HELO" identity.'
> > > 
> > > I am not sure whether it is even a definition.  From an
> > > implementation perspective the last paragraph of Section 2.4 is unclear.
> >
> >It seems clear to me.  Could you expand on why you think it's problematic
> >or provide recommended text.
> 
> I mentioned in a previous comment that "<domain> definitions (not
> sure whether it is a definition) are mentioned in two places.  I
> suggest fixing that and then getting back to the above".

I don't understand what you think is wrong.  Repeating the comment won't help.  
I did read it.  I don't understand it.

> > >    'Generating non-delivery notifications to forged identities that have
> > >    
> > >     failed the authorization check is a source of backscatter and SHOULD
> > >     be avoided.'
> > > 
> > > This RFC 2119 SHOULD is not in RFC 4408.  The above does not have
> > > anything to do with Sender Policy Framework.  It was mentioned during
> > > WG discussions that "SPF can help the backscatter problem" [2].  This
> > > text may have been introduced in response to a review [3].  Is this
> > > an enhancement or a clarification?
> >
> >It's a clarification.
> 
> In a previous comment I mentioned that "my reading of the above is
> that it is neither an enhancement or a clarification.  I suggest
> removing the text".

So you think it's not something that should be avoided?

> > > In Section 2.6.1:
> > >    'A result of "none" means either (a) no syntactically valid DNS
> > >    domain
> > >    
> > >     name was extracted from the SMTP session that could be used as the
> > >     one to be authorized, or (b) no TXT records were retrieved from the
> > >     DNS that appeared to be intended for use by SPF verifiers.'
> > > 
> > > I suggest "DNS TXT records".  What is a syntactically valid DNS
> > > domain name?  The definition of "none" in RFC 4408 is clear (to me).
> >
> >How could one retrieve anything other than DNS TXT records from the
> >DNS?  That
> >seems redundant since it says "retrieved from the DNS".
> 
> I left this DNS angle for Andrew to comment.

OK.

> > > In Section 2.6.7:
> > >    'A "permerror" result means the domain's published records could not
> > >    
> > >     be correctly interpreted.  This signals an error condition that
> > >     definitely requires manual intervention to be resolved.'
> > > 
> > > What is "manual intervention" in the above?
> >
> >Would operator intervention or operator action be better?
> 
> I suggest "operator intervention" (there was a comment about this).

OK.  Changed.

> > > In Section 3:
> > >    'An SPF record is a DNS record that declares which hosts are, and are
> > >    
> > >     not, authorized to use a domain name for the "HELO" and "MAIL FROM"
> > >     identities.'
> > > 
> > > How are hosts which are not authorized to use a domain name listed in
> > > a SPF record?
> >
> >If a record contains something like -a:host.example.com that host is
> >explicitly not authorized.
> 
> Ok.
> 
> > >    "The SPF record is a single string of text."
> > > 
> > > What is a single string of text?  It may be easier to state this in
> > > terms of record format.
> >
> >I think this was addressed in a later comment and it should be improved.
> 
> There was a suggested text:
> 
>    "An SPF policy statement is expressed as a single string of text encoded
> in a DNS TXT record."
> 
> and:
> 
>    "The SPF record is a variable length string of octets encoded in
>     a DSN TXT record."

I split the difference and put in:

The SPF record is expressed as a single string of text encoded in a  DNS TXT 
record.

> > >    'ADMDs publishing SPF records SHOULD try to keep the number of
> > >    
> > >     "include" mechanisms and chained "redirect" modifiers to a minimum.'
> > > 
> > > What is the minimum?  Note that this RFC 2119 SHOULD is in RFC 4408.
> >
> >It's the least an ADMD needs to properly describe the hosts
> >authorized to send
> >mail from their domain.  It's minimum in the sense of no more than needed.
> 
> I commented previously about this "I didn't understand that as there
> wasn't any explanation for the RFC 2119 SHOULD.  I suggest adding an
> explanation or rewriting the text so that the expression (see above)
> is clear".

I'm failing to understand how to clarify that use the minimum means use as 
little as you can.  It's basically just explaining the dictionary to the 
reader, so I think I don't understand what you think needs changing.

> > >    'ADMDs SHOULD also try to minimize the amount of other DNS
> > >    information
> > >    
> > >     needed to evaluate a record.'
> > > 
> > > This RFC 2119 SHOULD is also in RFC 4408.  There are two RFC 2119
> > > SHOULDs in the last paragraph of Section 3.  This usage of RFC 2119
> > > key words does not help the SPF "publisher".   In my opinion the
> > > "SHOULD try" in this context uses the RFC 2119 key word in unorthodox
> > > ways.
> >
> >I took try out of the section.  I think that should do it.
> 
> There was a comment that:
> 
>   "SHOULD minimize" conflicts with one intended use, namely
> 
>     v=spf1 redirect=_spf.example.com
> 
>    Exemplified in Section 6.1, that use does not attain the minimum.

If it makes sense to use the redirect then that's the minimum.  If we truly 
wanted the absolute minimum physically possible, we'd only have ip4, ip4, and 
maybe include mechanisms.  I don't think there's a conflict.

> > > In Section 3.1:
> > >    'SPF records MUST be published as a DNS TXT (type 16) Resource Record
> > >    
> > >     (RR) [RFC1035] only.'
> > > 
> > > I would ask why "MUST be published ... only".  By the way, Section 3
> > > has a reference to Section 4 for record format instead of Section
> > > 3.1.  Is that correct?
> >
> >The only is to make it clear that the Type SPF records that were
> >defined in RFC
> >4408 are no longer part of the protocol.  It is correct.  The ABNF for the
> >record format is in section 4.
> 
> There were two comments:
> 
>    "The reference is indeed incorrect."
> 
> and:
> 
>   'Should we say "is described along with its interpretation in Section
>    4.5"?  How about changing the title of Section 4.5 to "Record Format"?'

Changing the title to Record Format would be wrong because it's not just the 
format.  I changed it to "The record format and the process for selecting  
records is described in ..."

> > > In Section 3.2:
> > >    "A domain name MUST NOT have multiple records that would cause an
> > >    
> > >     authorization check to select more than one record."
> > > 
> > > This RFC 2119 MUST NOT was in RFC 4408.  Is this to help the SPF
> > > "publisher" and if so, how does it help?
> >
> >4408 has an implicit requirement for this.  In section 4.5, it says:
> > > If the resultant record set includes more than one record, check_host()
> > > produces the "permerror" result.
> >
> >The MUST NOT here makes it clear the condition has to be
> >avoided.   It's not a
> >functional change in the protocol.
> 
> Ok.
> 
> > > In Section 3.3:
> > >    "As defined in [RFC1035] sections 3.3.14 and 3.3, a single text DNS
> > >    
> > >     record can be composed of more than one string."
> > > 
> > > See previous comment about " a single string".
> > > 
> > >    "If a published record contains multiple character-strings, then the
> > >    
> > >     record MUST be treated as if those strings are concatenated together
> > >     without adding spaces."
> > > 
> > > This RFC 2119 MUST is also in RFC 4408.  I suggest removing the
> > > "MUST" in the example.
> >
> >Why?  If you don't do it that way, the result is not interoperable.
> 
> I'll take some text from the Responsible Area Director:
> 
>    "MUSTs are normally instructions for protocol actions or
> implementation coding".
> 
> I suggest not using it in examples.

OK.  I changed it to "is equivalent to:"

> > > In Section 4:
> > >    "A compliant SPF implementation MUST do something semantically
> > 
> > equivalent
> > 
> > > to this description."
> > > 
> > > It's unusual to see a "do something" RFC 2119 requirement in a
> > > specification.  Is the SPFBIS WG working on compliance for SPF
> > > implementations?
> >
> >It was added as part of making check_host() not a formal API/functional
> >specification.  You need an equivalent result, but needn't follow
> >all the steps
> >exactly.  Please recommend an alternative.
> 
> I suggest fixing the "do something"

OK.  I changed it to produce results.

> > >    "Implementations MAY use a different algorithm than the canonical
> > >    
> > >     algorithm defined here, so long as the results are the same in all
> > >     cases."
> > > 
> > > Why is this a RFC 2119 MAY?  I am aware that the text is in RFC 4408.
> >
> >trying to make it clear there's flexibility for the implementer.  Do
> >you think
> >it should be changed?
> 
> I'll leave this as it is, i.e. no change suggested.
> 
> > > In Section 4.1:
> > >    '<domain> - the domain that provides the sought-after authorization
> > >    
> > >                information; initially, the domain portion of the "MAIL
> > >                FROM" or "HELO" identity.'
> > > 
> > > The above text is different from the text in Section 2.4.
> >
> >Yes, but not inconsistent with it.
> 
> I suggest having the definition in one place for consistency.  What I
> am looking at is consistency for the reader instead of where I
> believe that there is inconsistency.

They are talking about different things relative to domains, so given the 
organization, I don't see an easy way to do that.  I'm open to suggestions.

> > > In Section 4.3:
> > >    "If the <domain> is malformed (e.g. label longer than 63 characters,
> > >    
> > >     zero-length label not at the end, etc.)"
> > >   
> > >   That should be "octets".
> >
> >I recall further discussion about this in a later message in the
> >thread.  I'll
> >address it there.
> 
> Ok.
> 
> > >    "Properly formed domains are fully qualified email domains as
> > >    
> > >     described in [RFC5321] Section 2.3.5."
> > > 
> > > What are "properly qualified email domains"?
> >
> >What it says they are in the reference.
> 
> There was the following comment:
> 
>    "Properly formed domains are described in [RFC5321] Section 2.3.5.
> 
>     The wording is not that good but I could not think of anything better."
> 
> There was another comment:
> 
>    'The term "email domain" lacks a formal definition, and is used
> improperly in Section 4.3 since HELO needs not be an email domain in some
> sense. I'd s/fully qualified email domains/fully-qualified domain names/.'

OK.  I took email out.

> > >    "Internationalized domain names MUST be encoded as A-labels, as
> > >    described in Section 2.3 of [RFC5890].on 2.3 of [RFC5890]."
> > > 
> > > I'm listing the above and leave it to Area Director guidance.  Please
> > > note that it is a new RFC 2119 requirement.
> >
> >It's a new requirement because IDN wasn't addressed in 4408.  It's
> >necessary to address it now and the WG agreed this was the best way to do
> >it.
> Ok.  I am leaving this to the Responsible Area Director.
> 
> > > In Section 4.4:
> > >    "In accordance with how the records are published (see Section 3
> > >    
> > >     above), a DNS query needs to be made for the <domain> name, querying
> > >     for type TXT only."
> > > 
> > > I don't understand the sentence.
> >
> >What don't you understand?
> 
> I commented previously about this: "I read the first paragraph of
> Section 4.4 as meaning that the DNS query is for Type TXT only. I
> suggest explaining that the record lookup is done by query for the
> SPF record described in Section 3. By the way, the text you suggested
> for Section 3 fits in nicely".

Sorry, still confused.   "I read the first paragraph of Section 4.4 as meaning 
that the DNS query is for Type TXT only."  That's correct, so I think you do 
understand the sentence.

> > >    'Alternatively, for a server failure (RCODE 2) result, check_host()
> > >    
> > >     MAY track failures and treat multiple failures within 24 hours for
> > >     the same domain as "permerror".'
> > > 
> > > This text is not in RFC 4408.  I found a note in Issue #1 [5] and in
> > > a message [6].
> >
> >Yes.  This is a change to address an operational issue found during SPF
> >deployment.
> 
> Stuart mentioned that: "This also solves the TIMEOUT for type 99
> problem. Reporting a PermError for a broken name server would be very
> appropriate (and switching to checking TXT first while said failure
> is cached would also be permissible)".
> 
> John commented that:  "Not that I'm aware of.  That looks to me like
> an optimization to do in the DNS cache, not in the client".
> 
> In a previous comment I mentioned that: "Are there any
> implementations that do this? If so, that's the justification; it's
> practical experience. If not, we should consider dropping it".

There was an issue.  We discussed possible solutions (my initial hope was to 
make all SERVFAIL responses permerror) and this was the best solution we came 
up with.  Because we want consistent results across implementations it's not 
just a local cache optimization.  I think we should leave it since I don't 
want to redo the entire process of solving the issue again.

> > > In Section 4.6.2:
> > >    "If there are no more mechanisms, the result is specified in
> > >    
> > >     Section 4.7."
> > > 
> > > This sentence does not make sense.
> >
> >If I changed it to "If there are no more mechanisms, the result is
> >the default
> >result as described in section 4.7" would it be clearer?
> 
> The follow text was proposed in a comment:
> 
>    "After all mechanisms have been processed and no matches are found, the
>     result is determined according to Section 4.7."

OK, but you didn't answer the question.  Would the text I proposed clarify 
things for you?  I like it better than the comment you brought up because I 
think it's clearer, so I'd prefer to use it if you think it's OK.

> > > In Section 4.6.4:
> > >    "SPF implementations MUST limit the total number of mechanisms and
> > >    
> > >     modifiers ("terms") that cause any DNS query to at most 10 during
> > >     SPF
> > >     evaluation."
> > > 
> > > This was discussed on the mailing list [7].
> > > 
> > >    "If this number is exceeded during a check, a permerror MUST be
> > > 
> > > returned."
> > > 
> > > I read this as if an implementation-specific limit is reached a
> > > permerror is returned.  There are two RFC 2119 MUST in the
> > > above.  That can be reduced to one MUST.
> >
> >Is there a problem with it the way it is?
> 
> Here is my previous comment:
> 
> For context we are at Section 4.6.4.  I'll quote some text from it:
> 
>   "If this number is exceeded during a check, a permerror MUST be returned."
> 
>   'If this limit is exceeded, the "mx" mechanism MUST produce a
>    "permerror" result.'
> 
>   'If this limit is exceeded, all records other than the first 10
>    MUST be ignored.'
> 
> I prefer not to discuss about the last one (see quoted text) again.
> In terms of coding it is easier if I am told to return a "permerror"
> results when DNS limit X is exceeded. That can be specified with one
> RFC 2119 MUST.  I am taking in consideration the SecDir
> recommendation and interoperability.  Speaking of interoperability
> the following text:
> 
>    'SPF implementations MUST limit the total number of mechanisms and
>     modifiers ("terms") that cause any DNS query to at most 10 during SPF
>     evaluation.'
> 
> does not encourage it because of the "at most". Section 4.6.4 does
> not make things easier for DNS operations as the reader cannot
> identify the limits easily due to the exceptions. I am ok with the
> exceptions. What I am suggesting is not to use two RFC 2119 MUST if
> it there is a way for it to be done with one RFC 2119 MUST.

They are different limits that you have to code differently, so I think 
combining them would reduce clarity for implementers.

> > > I read the first paragraph of Section 4.6.4.  I am not sure how the
> > > absolute requirement will be interpreted by the reader.
> >
> >We worked on making it clearer.  If you have suggested improvements, please
> >send them.
> 
> Could you please provide the proposed text?

No, I don't have any more proposed text.  I was saying the WG has already 
worked over trying to clarify this section.  I think we've done our best,  to 
clearly express something that is complicated.  I don't think "do it better" 
is going to result in further improvements, so if you have an idea how to 
improve it, please provide it.

> > >    'MTAs or other processors SHOULD impose a limit on the maximum amount
> > >    
> > >     of elapsed time to evaluate check_host().  Such a limit SHOULD allow
> > >     at least 20 seconds.  If such a limit is exceeded, the result of
> > >     authorization SHOULD be "temperror".'
> > > 
> > > There are three RFC 2119 SHOULDs in the above.  I suggest rewriting
> > > the text to reduce that.
> >
> >They are three different things.  Why would rewriting it help?
> 
> Here is my previous comment:
> 
> Throughout the document the words "MTA", "other processors", "mail
> receivers", "SPF implementations" and "SPF verifiers" are used. That
> looks inconsistent to me.
> 
> Here's an example of a rewrite:
> 
>    MTA or other processors SHOULD:
> 
>    (a) impose a limit on the maximum amount of elapsed time to
> evaluate check_host().
> 
>    (b) allow for at least 20 seconds.
> 
>    (c) return a result of authorization of "temperror".

I don't find that any clearer, probably less so than what's in the draft now.
> 
> I don't see what Points (a) or (b) has to do with interoperability.
> Trying to common factor the RFC 2119 SHOULD reduces occurrences of
> the word instead of addressing the issue. I suggest understanding the
> intent first before working on the text to be added.

They are about defining some consistent limits/expectations in order to avoid 
both impedance between senders and receivers and the potential security 
implications of not implementing proper limits.

> Alessandro and Murray did not agree with the above.

OK.  It's from RFC 4408, so I don't think it should be changed:

>    MTAs or other processors MAY also impose a limit on the maximum
>    amount of elapsed time to evaluate check_host().  Such a limit SHOULD
>    allow at least 20 seconds.  If such a limit is exceeded, the result
>    of authorization SHOULD be "TempError".

> > >    'SPF implementations SHOULD limit "void lookups" to two.'
> > > 
> > > What are void lookups?
> >
> >The term is defined in the two immediately preceding sentences.
> 
> Ok.
> 
> > >    "In this case, a default of two is RECOMMENDED."
> > > 
> > > Why is "two" recommended?
> >
> >Because that's the limit that is currently widely deployed (in the perl
> >Mail::SPF implementation) with which there has been good experience.
> 
> Ok.
> 
> > >    'Note that records SHOULD always use either a "redirect" modifier or
> > >    
> > >     an "all" mechanism to explicitly terminate processing.'
> > > 
> > > Why is there a RFC 2119 key word in a note?
> >
> >Took out the 2119 word.
> 
> Ok.
> 
> > > In Section 4.8:
> > >    'e.g., "foo..example.com") or overlong labels (more than 63
> > >    
> > >     characters).'
> > > 
> > > I suggest octets.
> > > 
> > > in Section 5:
> > >    'SPF implementations on IPv6 servers need to handle both "AAAA" and
> > >    "A"
> > >    
> > >     secords, for clients on IPv4 mapped IPv6 addresses [RFC4291].'
> > > 
> > > There is a typo, "records".
> >
> >Already fixed.
> 
> Ok.
> 
> > > I'll flag the above for review by people with IPv6 expertise.
> >
> >What's in -15 is the result of the review and WG feedback.
> 
> As a note for the working group, Simon Perreault proposed text about this.

There were several suggestions, some of them inconsistent with actual practice 
experienced by some WG members with real operating systems.  I chose to go 
with the text that supported getting interoperable results regardless of OS.

> > > In Section 5.4:
> > >    'Note regarding implicit MXs: If the <target-name> has no MX records,
> > >    
> > >     check_host() MUST NOT pretend the target is its single MX, and MUST
> > >     NOT default to an A or AAAA lookup on the <target-name> directly.'
> > > 
> > > There are two RFC 2119 MUSTs in the above.  This is over-usage of RFC
> > > 2119 key words.  This text was in RFC 4408.  This is not a divergence
> > > from RFC 5321, it is a contrary to Section 5 of RFC 5321.
> >
> >It's not contrary.  It's saying that aspect of 5321 is not relevant to SPF.
> >In any case, changing it would be a substantiative change to the
> >protocol that
> >would require all implementations to be updated.
> 
> There are changes listed in Appendix C.  Implementations will have to
> be updated anyway in my opinion.  I'll discuss this matter with the
> Responsible Area Director.

What is there to discuss?  It's NOT a change from 4408?  Changing it from 
what's there now would be out of scope for our charter.

> > > In Section 5.5:
> > >   "This mechanism SHOULD NOT be used."
> > > 
> > > I suggest providing a reason for this.
> >
> >The reason is given in the note at the end of the section.
> 
> The following text was suggested:
> 
>    "This mechanism SHOULD NOT be used it is not as reliable as other
> mechanisms in cases of DNS errors, and it places a large burden on the
> .arpa name servers."

In -14, it already says:

>    Note: This mechanism is slow, it is not as reliable as other
>    mechanisms in cases of DNS errors, and it places a large burden on
>    the .arpa name servers.  

What should I change?

> > >    "To prevent DoS attacks, more than 10 PTR names
> > >    
> > >     MUST NOT be looked up during the evaluation of a "ptr" mechanism
> > >     (see Section 4.6.4)."
> > > 
> > > The above restates a previous RFC 2119 MUST.
> > > 
> > >    "If used, proper PTR records MUST be in place for the domain's hosts
> > >    
> > >     and the "ptr" mechanism SHOULD be one of the last mechanisms
> > >     checked."
> > > 
> > > Those RFC 2119 key words are not in RFC 4408.  I don't see how this
> > > RFC 2119 change qualifies as a clarification or explanation.
> >
> >For the MUST, it flat out won't work if you don't have those (no
> >interoperation).  For the SHOULD, it's an optimization.  We can change it
> >if you want.
> 
> My previous comment was that: "It's a change. I suggest reverting it.
> Telling people who are using the "ptr" mechanism that they must have
> PTR records is stating the obvious".

Obvious to you.  Not obvious to everyone.

> > >    "It is, however, still in use and part of the SPF protocol, so
> > >    compliant
> > >    check_host() implementations MUST support it."
> > > 
> > > What is a compliant check_host() implementation?
> >
> >One that works in accordance with 4408bis.  Please suggest an alternative
> >if you don't like that.
> 
> I am ok with what the working group wants.
> 
> > > In Section 5.6:
> > >    "ip6-network      = <as per [RFC 4291], section 2.2>"
> > > 
> > > I suggest that the above reference be verified for correctness.
> > > 
> > > In Section 5.7:
> > >    'v=spf1 exists:%{ir}.%{l1r+-}._spf.%{d} -all'
> > > 
> > > I'll flag this for review by DNS folks.
> > > 
> > > In Section 6:
> > >    'The modifiers defined in this document ("redirect" and "exp") MAY
> > >    
> > >     appear anywhere in the record, but SHOULD appear at the end, after
> > >     all mechanisms.'
> > > 
> > > This text is in RFC 4408.  I would label the RFC 2119 usage as
> > > defective.
> >
> >Why?
> 
> My previous comment was: "The text tells me that it is ok if the
> modifiers appear anywhere but they should appear at the end after all
> mechanisms. It would be clearer to say that it should appear after
> all mechanisms".

That is defective use of 2119 language?  I disagree that that would be 
clearer.

> > > In Section 6.2:
> > >    "Since the explanation string is intended for an SMTP response and
> > >     [RFC5321] Section 2.4 says that responses are in [US-ASCII], the
> > >     explanation string MUST be limited to US-ASCII.'
> > > 
> > > It would be easier to defer to RFC 5321 instead of setting a RFC 2119
> > > requirement in this draft.  I note that this requirement was not in RFC
> > > 4408.
> >
> >It's not a new requirement.  If you couldn't find it in 4408, then that
> >means we've succeeded in making the actual requirement more clear.
> 
> None of the WG participants have pointed out where the actual
> requirement (RFC "MUST") is in RFC 4408.  If the text was in RFC 4408
> it would be possible for a WG participant to point me to it.   There
> was a comment about 'this could be expressed non-normatively (e.g.
> "needs to be")'.

If it's not ASCII, it won't interoperate (at all).  Isn't that exactly where 
2119 language is supposed to be used?  Regardless of how it's worded in 4408, 
putting non-ASCII characters in explanation strings does not and has never 
worked.  At most it's making something explicit that was implicit before, 
which is a good clarification I'd think.

> > >    "Software SHOULD make it clear that the explanation string
> > >     comes from a third party."
> > > 
> > > I could not understand what that means in RFC 2119 terms.  The draft
> > > uses RFC 2119 key words by example instead of providing an explanation.
> >
> >I don't understand the comment.
> 
> There was a comment from Murray:  "we're talking about
> interoperability between humans here, and not between implementations".

We're talking about avoiding security issues caused by the humans getting 
confused.

> > > In Section 7.1:
> > >    'The "toplabel" construction is subject to the LDH rule plus
> > >    
> > >     additional top-level domain (TLD) restrictions.'
> > > 
> > > Where can I find these restrictions?  Please note that I have read
> > > the Informational RFC.
> >
> >LDH + non-numeric as the RFC says.
> 
> I'll get back to this issue.
> 
> > > In Section 7.3:
> > >    "-exists:%(ir).sbl.spamhaus.example.org"
> > > 
> > > I commented about vendor-neutral previously [8].  The above is a way
> > > to get around the objection [9].  I suggest cleaning this up.
> >
> >Took spamhaus out.
> 
> Ok.
> 
> > >   "so trailing dots SHOULD NOT be published by domain owners"
> > > 
> > > Why is it "domain owners"?
> >
> >Took out everything after published.
> 
> Ok.
> 
> > >    "This macro SHOULD NOT be used.  See Section 5.5 for the discussion
> > >    
> > >     about why not."
> > > 
> > > This comes out as a flippant explanation for the RFC 2119 SHOULD.
> >
> >Please recommend non-flippant text then.  It doesn't read that way to me.
> 
>    "This macro SHOULD NOT be used (See Section 5.5 for discussion).

Done.

> > >    "This SHOULD be a fully qualified domain name, but if one does not
> > >    
> > >     exist (as when the checking is done by a MUA) or if policy
> > >     restrictions
> > >     dictate otherwise, the word "unknown" SHOULD be substituted."
> > > 
> > > The RFC 2119 key words are in RFC 2119.  I don't know what to say.
> >
> >OK.
> 
> Murray comemnted: "I think that looks fine as-is, though we could
> change the first SHOULD to "ought to" because it's a transformation
> of other input and not an implementation choice.  (And I assume you
> mean they were in RFC4408.)

Since the values end up in log files and Received-SPF/authentication results 
header fields that are provided to be processed on other systems, it is an 
interoperability issue.  Having consistency helps interoperability.  I'd 
prefer to leave it.

> > >    "When the result of macro expansion is used in a domain name query,
> > >    if
> > >    
> > >     the expanded domain name exceeds 253 characters (the maximum length
> > >     of a domain name), the left side is truncated to fit, by removing
> > >     successive domain labels (and their following dots) until the total
> > >     length does not exceed 253 characters."
> > > 
> > > Where is it specified that the maximum length of a domain name is 253
> > > characters?
> >
> >This was covered in someone else's response.
> 
> Ok.
> 
> > >    "Care has to be taken by the sending ADMD so that macro expansion for
> > >    
> > >     legitimate email does not exceed the 63-character limit on DNS
> > >     labels."
> > > 
> > > Where is that 63-character limit specified?
> >
> >This was covered in someone else's response.
> 
> Ok.
> 
> > > It is odd to have RFC 2119 requirements under a "Notes" heading.
> >
> >They aren't really notes in that sense.  If you have a better name for the
> >section, please suggest.
> 
> I suggest removing all the RFC 2119 requirements in Section 7.3 if
> the working group cannot find a way to fix this.

Why?  How about if we call it "Details" instead of Notes?

> > > In Section 8.2:
> > >    'A "neutral" result MUST be treated exactly like the "none" result;
> > >     the distinction exists only for informational purposes.'
> > > 
> > > Why is an existing RFC 2119 restated?
> >
> >Because of the structure of the document that the WG came up with.  I don't
> >find a good way to avoid it and not make it less readable.
> 
> I'll leave this issue open as it was mentioned in the AppsDir review.
> 
> > > In Section 8.5:
> > >    "The domain owner believes the host is not authorized but is not
> > >    willing
> > >    
> > >     to make a strong policy statement."
> > > 
> > > Why is it domain owner?
> >  
> >  Because conversion to the newer terms was incomplete.  I clean up this
> >  and
> >
> >some others as well.
> 
> Ok.
> 
> > > In Section 9:
> > >    "An operator could choose to use both to serve different downstream
> > >    
> > >     agents.  In such cases, care needs to be taken to ensure both fields
> > >     are conveying the same details, or unexpected results can occur."
> > > 
> > > This has been discussed on the mailing list.  I don't think that it
> > > encourages interoperability but the working group decided otherwise
> > > [10].
> > > 
> > > In Section 9.1:
> > >    "The Received-SPF header field is a trace field (see [RFC5322]
> > >    Section
> > >    
> > >     3.6.7) and SHOULD be prepended to the existing header, above the
> > >     Received: field that is generated by the SMTP receiver."
> > > 
> > > "prepended to the existing header" does not sound right.
> >
> >I believe it is.
> 
> Murray explained this in a comment about the review.
> 
> > >    "It MUST appear above all other Received-SPF fields in the message."
> > > 
> > > How does this fit into the previous RFC 2119 SHOULD?
> >
> >It will generally happen as a natural result of following the previous one,
> >but is there in case more than one is added by a single MTA.
> 
> Murray commented on this.
> 
> > > in Section 10:
> > >    "This section is not a "how-to" manual, or a "best practices"
> > >    document,
> > >    
> > >     and it is not a comprehensive list of what such entities SHOULD do
> > >     in
> > >     light of this document."
> > > 
> > > What is the meaning of the RFC 2119 SHOULD in the above?
> >
> >Got rid of it.
> 
> Ok.
> 
> > > In Section 10.1.1:
> > > 
> > > There was a comment from Authur Thisell about the table [11] in this
> > > sub-section.
> >
> >There didn't seem to be support for changing it though.
> 
> I read RFC 4408 and I did not find any table similar to the one in
> Section 10.1.1 of draft-ietf-spfbis-4408bis-14.  My understanding of
> "support for changing text" is that the text is proposed on the
> working group mailing list and it is added if there is agreement.  I
> didn't find any messages about that in the mailing list
> archives.  What I found was a message disagreeing with a change which
> was made in the draft which was posted together with an explanation
> about why there was disagreement.  In my opinion there wasn't any
> support for making that change in the draft.

The text came from Alessandro and was discussed on the list.  Arthur's comment 
came later.  The table is new, but is only a new expression of information 
that was in 4408.  It does not create any new requirements.

> > > In Section 10.1.2:
> > >    "Publishing SPF records for domains that send no mail is
> > >    
> > >     a well established best practice."
> > > 
> > > I am not aware of that best practice.  I did a few DNS queries:
> > > 
> > > ;; QUESTION SECTION:
> > > ;bing.com.                      IN      TXT
> > > 
> > > ;; ANSWER SECTION:
> > > bing.com.               3600    IN      TXT     "v=msv1
> > > t=6097A7EA-53F7-4028-BA76-6869CB284C54"
> > > 
> > > ;; QUESTION SECTION:
> > > ;com.com.                       IN      TXT
> > > 
> > > ;; ANSWER SECTION:
> > > com.com.                293     IN      TXT
> > > "google-site-verification:esSnoZdChIkkfCfS9MMgqNhE0yaXfnnZWdZPuBf7e8k"
> >
> >OK.  See
> >http://www.bits.org/publications/security/BITSEmailAuthenticationFeb2013.pd
> >f
> I suggest "good practice" as mentioned by Alessandro.

What's wrong with the way it is?  I think the current text is accurate.

> > > In Section 10.1.3:
> > >    "SPF functionality is enhanced by administrators ensuring this
> > >    identity is set correctly and has an appropriate SPF record."
> > > 
> > > How is SPF functionality enhanced?
> >
> >If there's an SPF record for HELO as well as Mail From, then a check can be
> >performed on bounces.
> 
> Ok.
> 
> > > In Section 10.2:
> > >    "As stated elsewhere in this document, there is no normative
> > >    requirement
> > >    
> > >     for specific handling of a message based on any SPF result."
> > > 
> > > The "as stated elsewhere" is vague.
> >
> >It comes up multiple places.  Adding several specific reference won't
> >improve the draft.
> 
> I suggest adding a reference to Section 8 (subject to the AppsDir
> review being addressed).

The next sentence has a reference to Section 8.

> > >    'The primary considerations are that SPF might return "pass" for mail
> > >    
> > >     that is ultimately harmful (e.g., spammers that arrange for SPF to
> > >     pass using nonsense domain names'
> > > 
> > > What are "nonsense" domain names?
> >
> >There was follow-on discussion about this.
> 
> "discardable" was suggested.

I went with disposable, since discardable is a specific term of art in other 
email authentication related documents and I don't want to create potential 
for confusion.

> > > In Section 10.3:
> > >    "The mediator can make the newly-posted message be as similar or as
> > >    
> > >     different from the original message as they wish."
> > > 
> > > The sentence seems incomplete, i.e. similar to what.
> >
> >The original message.
> 
> Ok.
> 
> > >    "For the operation of SPF, the essential concern is the email
> > >    
> > >     address in the 5321.MailFrom command for the new message."
> > > 
> > > What does this mean?
> >
> >It means that's the thing that's the most important.
> 
> Ok if the working group agrees.
> 
> > >    'Because SPF evaluation is based on the IP Address of the "last"
> > >    
> > >     sending SMTP server, the address of the mediator will be used,
> > >     rather
> > >     than the address of the SMTP server that sent the message to the
> > >     mediator.'
> > > 
> > > I'll note that this is a mix of RFC 5321 and RFC 5598.  The result is
> > > clear or unclear depending on the background of the reader.
> >
> >I guess I have the background for it to be clear.  Please suggest an
> >improvement.
> 
> I suggest leaving it as it if there isn't any disagreement.
> 
> > > In Section 11.5:
> > >    "An SPF compliant receiver gathers information from the SMTP commands
> > >    
> > >     it receives and from the published DNS records of the sending domain
> > >     holder"
> > > 
> > > I suggest explaining the untrusted part.
> >
> >That's what the section does.
> 
> There was a comment:  "A second sentence indicating most of that
> input is unverified is probably in order, with references if possible".

I think the section explains exactly that, so some text would help.

> > > In Section 11.6:
> > >    "Checking SPF records causes DNS queries to be sent to the domain
> > >     owner."
> > > 
> > > How are DNS queries sent to domain owners?
> >
> >Fixed it.
> 
> Ok.
> 
> > > Section 12:
> > > 
> > > I note that Murray S. Kucherawy has contributed significantly to
> > > draft-ietf-spfbis-4408bis.  If I am not mistaken it is IETF practice
> > > to acknowledge such contributions.  I don't recall who sent text at
> > > the moment.  If you did, please send an (off-list) email to the
> > > SPFBIS WG Chairs.
> >
> >There are many, many people who contributed to both 4408 and 4408bis that
> >aren't listed.  If someone wants to make a list of people who made a
> >significant contribution, I'll be glad to include it.
> 
> My understanding is that it is the task of the document editor to
> keep track of contributions and contributors.  There is a message at
> http://www.ietf.org/mail-archive/web/spfbis/current/msg03059.html
> where it is mentioned that:
> 
>    "2.  Scott, in collaboration with Murray Kucherawy (who proposed
>        the original reorganization), undertakes the reorganization and
>        creates the next WG document."

I think to single out one person in this way would be unfair to other people 
that made significant contributions.  The original 4408 acknowledgements 
weren't done this way and so I think it's literally impossible to properly 
develop an acknowledgements section that would be fairly done.  This is not to 
knock down Murray's contribution, which I agree has been significant.

There was a thread about this on the IETF main list in late March.  Working 
groups vary significantly.  What's in the draft for acknowledgements has been 
there, virtually unchanged since last summer.  If you really want a reasonable 
list of contributors, we'll need to discuss a new schedule because this won't 
finish in May.

> > > In Section 13.2:
> > >    "Per [RFC3864], the "Received-SPF:" header field is added to the IANA
> > >    
> > >     Permanent Message Header Field Registry."
> > > 
> > > Note to self: request removal from the provisional message  header
> > > field registry.
> > > 
> > > The "status:" should be "standard"
> >
> >Not Standards Track?
> 
> I verified this again; it's "standard".

Fixed.

> > > In Section 13.3:
> > >    "Their status should not be changed."
> > > 
> > > I suggest "Their status is unchanged".  BTW, it should be "SPF
> > > Modifier Registry".
> >
> >Fixed
> 
> Ok.
> 
> > > I suggest following RFC 6652 Section 5.1.
> > > 
> > > An "Updates:" for RFC 6652 may be needed.
> >
> >Why?  It's the registry that gets updated, not 6552.
> 
> I'll ask the Area Director.
> 
> > > References:
> > > 
> > > Why is RFC 1123 a normative reference?
> >
> >See section 2.4.
> 
> Ok.
> 
> > > Why is RFC 3864 a normative reference?
> >
> >Because it defines how the Received-SPF registration is done.
> 
> Ok.
> 
> > > Where can I find "Designated Mailers Protocol"?
> >
> >See draft-fecyk-dsprotocol
> 
> Ok.  As a note for the working group, I could not find the draft.

First Google hit for draft-fecyk-dsprotocol is:

http://tools.ietf.org/html/draft-fecyk-dsprotocol-04

> > > Where can I find "Domain-Authorized SMTP Mail"?
> >
> >I don't have a copy of this.
> 
> I suggest dropping the reference.

SPF did not spring from nothing.  It gathered a number of concepts, added a 
few things, and managed to be successful.  I think it would be more 
appropriate to leave it as it shows part of the trace back to the origin of 
the protocol.

> > > In Appendix C:
> > > 
> > > In Section E.1:
> > >    'This would cause a lookup on an DNS white list (DNSWL) and cause a
> > >    
> > >     result of "fail" only for email not either coming from the
> > >     domain's mx host(s) (SPF pass) or white listed sources (SPF
> > >     neutral).'
> > > 
> > > I didn't find any discussion about this on the SPFBIS mailing
> > > list.  is there an explanation for this change between RFC 4408 and this
> > > draft?
> >
> >It was pointed out that this is a white list, not a black list.
> 
> I suggest dropping this change as it was not discussed within the
> working group.

Yes.  It was.

> > > I note that there are 48 pages in RFC 4408.  There are 78 pages in
> > > this draft.  A significant amount of text has been added in the
> > > Appendix (over 20 pages).  I doubt that the text has been carefully
> > > reviewed.  I may be asked for an explanation about all that once the
> > > draft leaves this working group.
> >
> >Why wait?
> 
> I am not the person doing the IESG evaluation.

OK.  I misunderstood your comment.  I think you're wrong about the lack of 
review though.

> > > In Appendix I:
> > > 
> > > Appendix I is about "Protocol Status".  This draft is intended as a
> > > Proposed Standard. From an IETF perspective that is what it will
> > > be.  Describing it as something different can be misleading.
> > > 
> > >    "[RFC4408] was designed to clearly document the protocol defined by
> > >    
> > >     earlier draft specifications of SPF as used in existing
> > >     implementations.  This updated specification is intended to clarify
> > >     identified ambiguities in [RFC4408], resolve techincal issues
> > >     identified in post-RFC 4408 deplyment experience, and document
> > >     widely
> > >     deployed extensions to SPF that have been developed since [RFC4408]
> > >     was published."
> > > 
> > > Extensions to the SPF protocol are clearly mentioned in the charter
> > > as being out of scope.  The "document widely deployed extensions" is
> > > problematic.
> >
> >The charter specifically allows for "addition of any enhancements
> >that have already gained widespread support".
> 
> I read the discussion about the charter again.  It is my
> understanding that extensions to the SPF protocol is
> out-of-scope.  The working group did not add a work item during the
> chartering discussion as it was considered as an extension to the protocol.

It was also not widely deployed.  Should I take authentication-results out of 
the draft then?

> > >    "This document updates and replaces RFC 4408 that was part of a group
> > >    
> > >     of simultaneously published Experimental RFCs (RFC 4405, RFC 4406,
> > >     RFC 4407, and RFC 4408) in 2006.  At that time the IESG requested
> > >     the
> > >     community observe the success or failure of the two approaches
> > >     documented in these RFCs during the two years following publication,
> > >     in order that a community consensus could be reached in the future."
> > > 
> > > Why is this needed?  The SPFBIS WG produced RFC 6686 to resolve the
> > > experiment.
> >
> >Since this is what obsoletes 4408, not everyone will read 6686.
> 
> There is an informative reference to RFC 6686.  I don't see why the
> text mentioned above is needed.

I think the section is incomplete without it.

> > >    "SPF is widely deployed by large and small email providers alike.
> > >    
> > >     There are multiple, interoperable implementations."
> > > 
> > > Someone might point out that this is marketing instead of text
> > > relevant to a protocol.  I'll mention that one or more significant
> > > issues (e.g. Issue #9) was identified during the WG
> > > discussions.  Issue #9 was not listed as the errata.  I suggest
> > > removing the section as it will be less text and there is already two
> > > informative references to help the reader find background information.
> >
> >#9 has been known since before 4408 was published.  In 2005 it was the
> >price of admission to the IETF.
> 
> My suggestion as document shepherd is to avoid that discussion.  The
> draft will face a very difficult Last Call as it is and more
> controversies won't make matters any better.

I'm not the one claiming it was discovered by the WG.  I'm not sure what 
controversy you think I'm adding.

Scott K