[TLS] FYI: Ubuntu 26.04.01 GnuTLS ML-DSA sigalg friction

[Viktor Dukhovni <[e-mail]>]

An Exim MTA SMTP client on Ubuntu 26.04.01 with default settings is
unable to negotiate a TLS connection to my SMTP server because its
GnuTLS library advertises ML-DSA-{44,65,87} sigalgs even though the
implementation of the corresponding verification algorithms in
libnettle is stubbed out and always fails.

My server has both RSA and ML-DSA keys and selects the latter when
both are offered by the client.

For the specific case reported, Exim falls back to cleartext (the sending
system has not enabled client-side DANE) so I am still able to receive
email from that system.  Other Ubuntu 26.04.01 configurations may in
fact be unable to send me mail without explicitly overriding the sigalg
list. :-(

I hope this combination will not become entrenched.

-- 
    Viktor.  🇺🇦 Слава Україні!

    (gdb) fr 0
    #0  ml_dsa_verify (algo=GNUTLS_PK_MLDSA65, signature=0x7fff41dbfab0, message=0x7fff41dbfac0, raw_pub=0x55c364b1a430)
        at nettle/../../../lib/nettle/pk.c:1849
    1849            return gnutls_assert_val(GNUTLS_E_UNSUPPORTED_SIGNATURE_ALGORITHM);
    (gdb) l -
    1844    static int ml_dsa_verify(gnutls_pk_algorithm_t algo MAYBE_UNUSED,
    1845                             const gnutls_datum_t *signature MAYBE_UNUSED,
    1846                             const gnutls_datum_t *message MAYBE_UNUSED,
    1847                             const gnutls_datum_t *raw_pub MAYBE_UNUSED)
    1848    {
    1849            return gnutls_assert_val(GNUTLS_E_UNSUPPORTED_SIGNATURE_ALGORITHM);
    1850    }

The full backtrace is:

    (gdb) bt
    #0  ml_dsa_verify (algo=GNUTLS_PK_MLDSA65, signature=0x7fff41dbfab0, message=0x7fff41dbfac0, raw_pub=0x55c364b1a430)
        at nettle/../../../lib/nettle/pk.c:1849
    #1  _wrap_nettle_pk_verify (algo=GNUTLS_PK_MLDSA65, vdata=0x7fff41dbfac0, signature=0x7fff41dbfab0, pk_params=0x55c364b1a398,
        sign_params=0x7fff41dbfa00) at nettle/../../../lib/nettle/pk.c:2719
    #2  0x00007fb19bbb0639 in pubkey_verify_data (se=se@entry=0x7fb19bd2d830 <sign_algorithms.lto_priv+2800>,
        me=me@entry=0x7fb19bd2cbf0 <hash_algorithms.lto_priv+1456>, data=data@entry=0x7fff41dbfac0,
        signature=signature@entry=0x7fff41dbfab0, params=params@entry=0x55c364b1a398, sign_params=sign_params@entry=0x7fff41dbfa00,
        flags=33619968) at ../../lib/pubkey.c:2695
    #3  0x00007fb19bbb32af in gnutls_pubkey_verify_data2 (pubkey=0x55c364b1a390, algo=<optimized out>, flags=33619968,
        data=0x7fff41dbfac0, signature=0x7fff41dbfab0) at ../../lib/pubkey.c:2209
    #4  0x00007fb19bbc7e42 in _gnutls13_handshake_verify_data (session=0x55c364b20b90, verify_flags=33619968, cert=0x7fff41dbfad0,
        context=0x7fb19bd277c0 <srv_ctx>, signature=0x7fff41dbfab0, se=<optimized out>) at ../../lib/tls13-sig.c:132
    #5  _gnutls13_recv_certificate_verify (session=0x55c364b20b90) at ../../lib/tls13/certificate_verify.c:128
    #6  0x00007fb19bb5f0f8 in _gnutls13_handshake_client (session=0x55c364b20b90) at ../../lib/handshake-tls13.c:131
    #7  0x00007fb19bb77bcb in handshake_client (session=0x55c364b20b90) at ../../lib/handshake.c:3094
    #8  gnutls_handshake (session=0x55c364b20b90) at ../../lib/handshake.c:2914
    #9  0x000055c35a22ea0c in do_handshake (socket=socket@entry=0x7fff41dc5200) at ../../src/cli.c:1856
    #10 0x000055c35a22f97c in socket_open_int (hd=hd@entry=0x7fff41dc5200, hostname=0x7fff41dc7826 "mx1.imrryr.org",
        service=service@entry=0x55c35a2b6ea0 <service> "25", app_proto=0x7fff41dc77ef "smtp", app_hostname=0x0,
        flags=flags@entry=68, rdata=0x0, edata=0x0, server_trace=0x0, client_trace=0x0, msg=0x55c35a238750 "Connecting to")
        at ../../src/socket.c:627
    #11 0x000055c35a228e4e in main (argc=<optimized out>, argv=<optimized out>) at ../../src/cli.c:1366