Re: [ietf-smtp] parsing SMTP replies

[Hector Santos <[e-mail]>]

>>
>> Our system has no limit out of the box and its system wide (global 
>> registry value). No current out of the box logic per user.  There 
>> might be a 3rd party RCPT command override p-code script 
>> (smtpcmd-rcpt.wcx) that may place a limit.  Can't a typical system 
>> handle 1000, 10K, 100K+ RCPTs?  How does a big list send mail 1 
>> million subscribers?
>
> Most of the very big senders, the ones sending to hundreds or 
> millions of people at a time, use VERP. Depending on what is known 
> about the domain MXs they open a set of connections and send the 
> mail. Most of the bulk mail is sent with one RCPT TO:. This is true 
> for marketing mail but even more true for transactional mail.

Not sure how VERP applies in the Queuing strategies other than to get 
a better feel for tracking, sorting bounces, i.e. how was the user and 
where should the bounce be directed.

Outbound Queuing Strategies (OQS) can be complex.  Long established 
strategies have been developed, fined tuned with variable retry 
tables, per domains, per reply code, including socket vs smtp error 
codes, etc.  Our OQS is pretty stable.  I leave it alone.  It works. 
Just let the engine run and eventually the mail gets out or bounced. :-)

So in lieu of a pre-established authorization, contract for sending, 
as we know receivers have connection load (balancing) limits.  One 
growing concern is with the growth of hosting domains using the same 
MXs.  We see the same MX behavior across different (trade) business 
domains.  It makes the queuing strategies a bit more complex as the 
sorting is more MX-dependent, not domain independent.  For example, I 
can add a rule that says private domain "MyCustomer.com" should get 
priority over customers using free ESP domains, but it is possible, 
the ESP is hosting the private domains.  I don't know if the ESP has a 
different profiles for the hosted domains, but currently, they appear 
to fall under the same umbrella as their regular free ESP mail service 
for the general public.

I do think that the big free ESP domains are very bias towards 
legitimate mail senders with private customer mailing list who happen 
to have members from these bigger ESP domains.   I am not talking 
about the big 1 million or even 1000 list, just maybe 100 and it can 
cause problems.  They place a big burden on the sender to the point I 
have informed one big domain if they continue to slow down my machines 
delaying deliveries with erroneous reply code & response text, despite 
all things checked off to comply with their requirements, I will add 
new logic where these free ESP domains could be restricted from 
mailing list subscriptions, i.e. a new (silly) option to offer mail 
operators. The big free ESP domains make it far too easy for one user 
to stop other users from getting mail and hurt a business reputation. 
Worst, they make it extremely difficult for support to resolve the 
problem.  This is what puts pressure on the small guys (like myself) 
to just totally block the usage of the free ESP domains.

>
>>  When our MLS is going thru a submission distribution, it has a 
>> transport to SMTP or create UUCP-ready files option.  The former 
>> method gets to learn from the SMTP receiver RCPT responses where a 
>> permanent 5yz result could  unsubscribe the user after a number of 
>> consecutive different message times.  Any permanent negative 
>> results with the intention of just being a limit could be 
>> interpreted as a user does no longer exist.
>
> I may have agreed with you a decade ago. But that’s not true for 
> modern bulk senders. Bounces are classified and sorted. We are long 
> past the days where any permanent negative result is interpreted as 
> a user no longer exists. 
> (https://wordtothewise.com/2019/11/theres-something-about-bounces/)
>

Nice graph.  In other words, we need to get more into parsing the TEXT 
responses which is now an optional concept with the extended codes?  
To know the reasons for the bounce?

The problem I found is that most of the time, the reply text are 
erroneous and/or vague and there is no standard here.  Among the 
problem, we are seeing more domains claim "DMARC" support, throwing 
out untrue signals it is a standard and now required for mail 
delivery.  For example, you get a 5yz with extended information with a 
link to some policy and the among the list of reasons, it is DMARC, 
SPF and bulk related stuff, maybe even a PTR requirement. Even with 
all those things met, most of the time, it just happen to be some user 
accidentally marked you as spam. Its not hard to do via some of the 
ESP's primitive UI. Even if unintentionally, its very hard to almost 
impossible to undo.   The problem is that one user will affect all the 
other users getting the same message or same mail from the same 
domain.   That's a problem also when Receiver X uses a set of 
verification methods that differs from Receiver Y.  No consistency in 
receiver expectations.

Anyway, regarding outbound mail strategies, there is the draft 
proposal for "retry=" time hint to help with minimizing retry delays 
and this has worked beautifully among conforming servers and clients.

It is obviously doable with updating SMTP software.  It believe we 
should develop a formal language for automated response text 
processing in order to resolve many of these "advanced" issues, like a 
RCPT limit.   Of course, it has to consider backward compatibility. We 
currently would express a RCPT LIMIT idea with a 4yz response.  I 
believe the RCPT LIMIT proposal attempts to reduce the RCPT commands 
when no longer accepted but there is no harm if the client was 
ignorant of this proposal.

Thanks

-- 
Hector Santos,
https://secure.santronics.com
https://twitter.com/hectorsantos