Re: [ietf-smtp] SMTP Reply code 1yz Positive Preliminary reply

Hector Santos <hsantos@isdg.net> Fri, 06 March 2020 15:22 UTC

Return-Path: <hsantos@isdg.net>
X-Original-To: ietf-smtp@ietfa.amsl.com
Delivered-To: ietf-smtp@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id D86493A0915 for <ietf-smtp@ietfa.amsl.com>; Fri, 6 Mar 2020 07:22:41 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.1
X-Spam-Level:
X-Spam-Status: No, score=-2.1 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (1024-bit key) header.d=isdg.net header.b=Qs0A+V/x; dkim=pass (1024-bit key) header.d=beta.winserver.com header.b=pmtpx3hE
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id kG75vC5cCqmW for <ietf-smtp@ietfa.amsl.com>; Fri, 6 Mar 2020 07:22:39 -0800 (PST)
Received: from mail.winserver.com (ftp.catinthebox.net [76.245.57.69]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 72C9F3A0913 for <ietf-smtp@ietf.org>; Fri, 6 Mar 2020 07:22:39 -0800 (PST)
DKIM-Signature: v=1; d=isdg.net; s=tms1; a=rsa-sha1; c=simple/relaxed; l=5627; t=1583508148; atps=ietf.org; atpsh=sha1; h=Received:Received:Received:Received:Message-ID:Date:From: Organization:To:Subject:List-ID; bh=Y1nSWijCJDrMRvpq/WYah5ITxhE=; b=Qs0A+V/xZ2wDLwwBuS266gioiJ3eP2hw35Qnl2NyGwKwfN4bCDgu14OctwUYL0 62oQg14OILGEfrH6jkg9xkRj4k38LWdDm0tehjXUKyDHxV0QMoXdvARN900OZy4Z QHIqbMDsoq6LYtppdWl5UaVOY8+9FoueiZIX/yNJiN6BI=
Received: by winserver.com (Wildcat! SMTP Router v8.0.454.9) for ietf-smtp@ietf.org; Fri, 06 Mar 2020 10:22:28 -0500
Authentication-Results: dkim.winserver.com; dkim=pass header.d=beta.winserver.com header.s=tms1 header.i=beta.winserver.com; dmarc=pass policy=reject author.d=isdg.net signer.d=beta.winserver.com (atps signer);
Received: from beta.winserver.com ([76.245.57.74]) by winserver.com (Wildcat! SMTP v8.0.454.9) with ESMTP id 2825485235.16278.5248; Fri, 06 Mar 2020 10:22:27 -0500
DKIM-Signature: v=1; d=beta.winserver.com; s=tms1; a=rsa-sha256; c=simple/relaxed; l=5627; t=1583507878; h=Received:Received: Message-ID:Date:From:Organization:To:Subject:List-ID; bh=LjymJnh W4ha0CcXBgXbLaOhT6zhrR8ejvlBX1+U2Mbs=; b=pmtpx3hEazyWQ4dPMsS8d7B +dqVaP7zyw9KzdQXBjTWDqBSEYizek9npqejzk+XReaCnbyc93NmlGz+pBYgxMaU cOg5qVkxBm4nhzZfs1BwxKQnuZiNNjOomNaCJLh9Wpg/7/43dKnoBSSZO5Ea976Y T7SW48Yt1gCHocrV7884=
Received: by beta.winserver.com (Wildcat! SMTP Router v8.0.454.9) for ietf-smtp@ietf.org; Fri, 06 Mar 2020 10:17:57 -0500
Received: from [192.168.1.68] ([75.26.216.248]) by beta.winserver.com (Wildcat! SMTP v8.0.454.9) with ESMTP id 2673518171.4.14784; Fri, 06 Mar 2020 10:17:57 -0500
Message-ID: <5E626AB7.60201@isdg.net>
Date: Fri, 06 Mar 2020 10:22:31 -0500
From: Hector Santos <hsantos@isdg.net>
Reply-To: hsantos@isdg.net
Organization: Santronics Software, Inc.
User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:24.0) Gecko/20100101 Thunderbird/24.8.1
MIME-Version: 1.0
To: Paul Smith <paul@pscs.co.uk>, ietf-smtp@ietf.org
References: <1583290845.3368.15.camel@gmail.com> <aedd19df-c406-2513-934e-4498ae159964@pscs.co.uk> <5E6128C8.7070001@isdg.net> <5E613D31.70301@isdg.net> <CFEDA025D86BD13BB8D15A56@PSB> <5E61B94D.9020104@isdg.net> <466b05c4-8db2-b0d8-8e87-8e8034aea97b@pscs.co.uk>
In-Reply-To: <466b05c4-8db2-b0d8-8e87-8e8034aea97b@pscs.co.uk>
Content-Type: text/plain; charset="UTF-8"; format="flowed"
Content-Transfer-Encoding: 7bit
Archived-At: <https://mailarchive.ietf.org/arch/msg/ietf-smtp/urf_WjVuZedyWJpKEmBSDHxheqo>
Subject: Re: [ietf-smtp] SMTP Reply code 1yz Positive Preliminary reply
X-BeenThere: ietf-smtp@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: "Discussion of issues related to Simple Mail Transfer Protocol \(SMTP\) \[RFC 821, RFC 2821, RFC 5321\]" <ietf-smtp.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ietf-smtp>, <mailto:ietf-smtp-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/ietf-smtp/>
List-Post: <mailto:ietf-smtp@ietf.org>
List-Help: <mailto:ietf-smtp-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ietf-smtp>, <mailto:ietf-smtp-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 06 Mar 2020 15:22:42 -0000

On 3/6/2020 4:02 AM, Paul Smith wrote:> On 06/03/2020 02:45, Hector 
Santos wrote:
>> John K, regarding the Timeouts, I was more facetious than serious
>> here. However, there are situations we already talked about in the
>> past. We could highlight some of them, but I believe the main one
>> was the 5 mins after the transaction was completed and a RSET/MAIL
>> can start a new transaction.
>
> I thought the main one was with delays after the the terminating '.',
> where the client is meant to wait 10 minutes for the server
> acknowledgement, but usually gives up a lot sooner than that, leading
> to duplicate messages.

There are basically two:

At all the states EHLO, MAIL, RCPT and DATA, etc. Each state can spawn 
scripts, shims, hooks, mfilters, etc. Once upon a time, a concern was 
scripts, current or future, can potentially take longer than the 
allotted idle timeout period per state. So there is technically 
runtime limits & restrictions at all states.  10 mins for DATA, 5 for 
the rest.

But the one that was real and significant, are with clients who do 
complete the DATA state, get a 2yz, 4yz or 5yz response to complete a 
transaction. Then instead of issuing a QUIT, it keeps the socket 
connection open and waits for a new outbound message to be queued to 
begin a new RSET/MAIL command to start a new transaction.  This may 
help scale clients but not servers because of wasteful session 
residence time. Depending on your server load settings, the server 
threads/processes/ports could be consumed honoring the 5 minute wait. 
  However, most of the time, no new transaction begins.  So it is very 
wasteful and wcSMPT provides an operator option to shorten it.

I think since the last review of this, over a decade ago, some 
operators reading the threads got some operational, practical input on 
this, agreed it was wasteful for the server and probably shorten the 
"new transaction" wait time so that the client does not chew up server 
resources.  I recall one big domain waiting the 5 minutes but today, 
looking at the logs, I see they have cut that down to 5 seconds!  So 
if no new mail is queued on the sender side within 5 seconds, it 
issues a QUIT.

Here is one example where the client is waiting and my particular 
wcSMTP server setup disconnects after 35 seconds.  It is set in the 
registry "PostDataTimeoutSecs" DWORD 35  and the default is 0 (the 
SMTP 5 mins limit to wait for a new transaction).

**************************************************************************
Wildcat! ESMTP Server v8.0.454.9
SMTP log started at Thu, 05 Mar 2020  20:27:49
Connection Time: 20200305 20:27:49  cid: 00003F96 tid: 00001390
SSL-Enabled=YES No-Quit-Cancel=OFF Receiver-Bin=ON
Client IP: 143.228.181.88:25678 (unknown) Host IP: 76.245.57.69:25
20:27:49.763 ** WCX Process: smtpcmd-connect  ret: -1
20:27:49.763 S: 220-winserver.com Wildcat! ESMTP Server v8.0.454.9 ready
20:27:49.763 S: 220-************** WARNING:  FOR AUTHORIZED USE ONLY! 
**********************
20:27:49.763 S: 220-* THIS SYSTEM DO NOT AUTHORIZE THE USE OF ITS 
PROPRIETARY COMPUTERS    *
20:27:49.763 S: 220-* AND COMPUTER NETWORKS TO ACCEPT, TRANSMIT, OR 
DISTRIBUTE UNSOLICITED *
20:27:49.763 S: 220-* BULK E-MAIL SENT FROM THE INTERNET. THIS SYSTEM 
WILL RESTRICT ACCESS *
20:27:49.763 S: 220-* TO CAN-SPAM (US S. 877) COMPLIANT CLIENTS ONLY. 
                      *
20:27:49.763 S: 220 
************************************************************************
20:27:50.120 C: EHLO s-bulk2-p.house.gov
20:27:50.126 ** WCX Process: smtpcmd-check-ehlo  ret: -1
20:27:50.126 S: 250-winserver.com, Pleased to meet you.
20:27:50.126 S: 250-SIZE 102400000
20:27:50.126 S: 250-8BITMIME
20:27:50.126 S: 250-SUBMITTER
20:27:50.126 S: 250-ETRN
20:27:50.126 S: 250-AUTH CRAM-MD5 DIGEST-MD5 LOGIN PLAIN PLAIN-MD5 SHA-1
20:27:50.126 S: 250-AUTH=LOGIN
20:27:50.126 S: 250-HELP
20:27:50.126 S: 250 STARTTLS
20:27:50.535 C: STARTTLS
20:27:50.536 S: 220 Ready to start TLS
20:27:50.536 ** Begin SSL negotiation
20:27:50.807 ** SSL negotiation successful
20:27:50.863 C: EHLO s-bulk2-p.house.gov
20:27:50.869 ** WCX Process: smtpcmd-check-ehlo  ret: -1
20:27:50.869 S: 250-winserver.com, Pleased to meet you.
20:27:50.869 S: 250-SIZE 102400000
20:27:50.869 S: 250-8BITMIME
20:27:50.869 S: 250-SUBMITTER
20:27:50.869 S: 250-ETRN
20:27:50.869 S: 250-AUTH CRAM-MD5 DIGEST-MD5 LOGIN PLAIN PLAIN-MD5 SHA-1
20:27:50.869 S: 250-AUTH=LOGIN
20:27:50.869 S: 250 HELP
20:27:51.022 C: MAIL From:<NJ09BPIMA@mail.house.gov> SIZE=30462
20:27:51.023 S: 250 <NJ09BPIMA@mail.house.gov>... Sender validation 
pending. Continue.
20:27:51.081 C: RCPT To:<andrea.santos@santronics.com>
20:27:56.555 ** WCX Process: wcsap ret: -1 (5473 msecs)
20:27:56.555 S: 250 <andrea.santos@santronics.com>... Recipient ok
20:27:57.502 C: DATA
20:27:57.502 S: 354 Start mail input; end with <CRLF>.<CRLF>
20:27:59.441 ** end of data received. (bytes: 31708) (15.9 K/s)
20:28:00.539 ** WCX Process: SmtpFilterHookLoader  ret: 1 (1098 msecs)
20:28:00.539 ** Authentication-Results: dkim.winserver.com;
20:28:00.539 **      dkim=pass header.d=mail.house.gov 
header.s=November2012-msg-mhg header.i=mail.house.gov;
20:28:00.539 **      dkim=pass header.d=mail.house.gov 
header.s=November2012-msg-mhg header.i=mail.house.gov;
20:28:00.540 ** message copied to router prespool
20:28:00.540 ** Router signaled to begin handling new messages
20:28:00.540 S: 250 Message accepted for delivery. (bytes: 31708)
20:28:37.045 S: 421 Idle Timeout - Server closing transmission channel.
20:28:37.045 ** Completed. Elapsed Time: 47300 msecs

-- 
HLS