[dane] TLSA lookup impedance mismatch with bare-bones DNS servers

[Viktor Dukhovni <[e-mail]>]

This week I received the first report of SMTP deliveries consistently
deferred due to TLSA RRset lookup problems.  The problem site is
nist.gov which has a DNSSEC validate MX RRset:

    $ secdig() { dig +adflag +noall +comments +ans "$@" @8.8.8.8; }

    $ secdig -t mx nist.gov.
    ;; Got answer:
    ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 20926
    ;; flags: qr rd ra ad; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0

    ;; ANSWER SECTION:
    nist.gov.	IN	MX	0 nist-gov.mail.protection.outlook.com.

with the MX host an unsigned zone:

    $ secdig -t a nist-gov.mail.protection.outlook.com.
    ;; Got answer:
    ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 3142
    ;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 0

    ;; ANSWER SECTION:
    nist-gov.mail.protection.outlook.com. 9	IN A	207.46.163.247
    nist-gov.mail.protection.outlook.com. 9	IN A	207.46.163.138
    nist-gov.mail.protection.outlook.com. 9	IN A	207.46.163.215

So far, so good, but if one tries to obtain TLSA RRs for the SMTP service
at this host:

    $ secdig -t TYPE52 _25._tcp.nist-gov.mail.protection.outlook.com.
    ;; Got answer:
    ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 63234
    ;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0

the remote DNS server responds with SERVFAIL, not NXDOMAIN, even though:

    $ secdig -t NS _25._tcp.nist-gov.mail.protection.outlook.com.
    ;; Got answer:
    ;; ->>HEADER<<- opcode: QUERY, status: NXDOMAIN, id: 30308
    ;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0

so clearly whatever DNS load-balancing kit is responsible for
mail.protection.outlook.com (the problem happens for all names
from this domain down) has a rather incomplete DNS implementation.

What this means for DANE is that one should try to avoid sending
TLSA queries when the TLSA base domain (the desired TCP endpoint)
has been discovered to lie in an "insecure" zone.

It is extremely unlikely that when "mx.example" is insecure, somehow
through the magic of DLV "_port._tcp.mx.example" is secure.

Postfix had code to avoid such queries, but it was applied too
narrowly (only when sending to non-MX destinations), the next
snapshot will have code to suppress TLSA lookups also with insecure
MX hosts.

Barring a contrary consensus from the group I will add suitable
language to the SMTP and the OPS drafts.  (The Postfix code will
need to implement the work-around regardless, my question is whether
the work-around can be stated in the SMTP and/or OPs drafts).

Without the work-around, there will be undesirable interoperability
issues when TLSA queries are sent to DNS servers that are neither
DNSSEC not DANE TLSA aware and may misbehave when presented with
unexpected queries.

FWIW, the server in question even fails with CNAME or PTR lookups:

    $ secdig -t CNAME _25._tcp.nist-gov.mail.protection.outlook.com.
    ;; Got answer:
    ;; ->>HEADER<<- opcode: QUERY, status: SERVFAIL, id: 38672
    ;; flags: qr rd ra; QUERY: 1, ANSWER: 0, AUTHORITY: 0, ADDITIONAL: 0

-- 
	Viktor.