[tcpm] More motivating scenarios for tcpm-ack-pull
Bob Briscoe <ietf@bobbriscoe.net> Tue, 05 November 2019 23:14 UTC
Return-Path: <ietf@bobbriscoe.net>
X-Original-To: tcpm@ietfa.amsl.com
Delivered-To: tcpm@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id B81E812008F for <tcpm@ietfa.amsl.com>; Tue, 5 Nov 2019 15:14:00 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -1.998
X-Spam-Level:
X-Spam-Status: No, score=-1.998 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, URIBL_BLOCKED=0.001] autolearn=ham autolearn_force=no
Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=bobbriscoe.net
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id QgLkrqSYMP0c for <tcpm@ietfa.amsl.com>; Tue, 5 Nov 2019 15:13:58 -0800 (PST)
Received: from server.dnsblock1.com (server.dnsblock1.com [85.13.236.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 72FF8120072 for <tcpm@ietf.org>; Tue, 5 Nov 2019 15:13:57 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=bobbriscoe.net; s=default; h=Content-Type:MIME-Version:Date:Message-ID:Cc: To:Subject:From:Sender:Reply-To:Content-Transfer-Encoding:Content-ID: Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc :Resent-Message-ID:In-Reply-To:References:List-Id:List-Help:List-Unsubscribe: List-Subscribe:List-Post:List-Owner:List-Archive; bh=SZY6yZDTVSu3Ks3r5GOM38sadrv6fmEpDWhMh4Q+8aE=; b=MkUerPa2O5ZIGK0HNJB5Lx7t5k CJ2hye2B1QczuSps9eD1gDxeGv6iUU1AC1yHRT5g/DltdfosSPLs6PVgkclFwSgGSDLYq/RgUy+X8 O7jlGdbb7nh9d/x4p6DIMfN32ITQQBvuKtQBD77wMFCTj/Yo5els6F01o5Iz3zgfRJcyJDr6VvSmv px+MiUOAlHQuZ8I2K6IeKdEz+/gC0edEe1KjN3qaih0hj6ob3D0Phu7JL4TLuOKHkXgywceikZI7D fjmmgczBvqEzTsKxZKk8zi31cJs6rXy66sVafbFMX39scIWv12SzTsOfGw0HxsZNWWrw0A5yYxh9a jttpJycg==;
Received: from [31.185.128.31] (port=36342 helo=[192.168.0.5]) by server.dnsblock1.com with esmtpsa (TLSv1.2:ECDHE-RSA-AES128-GCM-SHA256:128) (Exim 4.92) (envelope-from <ietf@bobbriscoe.net>) id 1iS81L-0004rG-M4; Tue, 05 Nov 2019 23:13:56 +0000
From: Bob Briscoe <ietf@bobbriscoe.net>
To: carlesgo@entel.upc.edu, "CROWCROFT, Jon" <Jon.Crowcroft@cl.cam.ac.uk>
Cc: tcpm IETF list <tcpm@ietf.org>
Message-ID: <3326ed99-b077-592b-7913-aeb2286912c4@bobbriscoe.net>
Date: Tue, 05 Nov 2019 23:13:54 +0000
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="------------0C20B222F5F222CA80011BBF"
Content-Language: en-GB
X-AntiAbuse: This header was added to track abuse, please include it with any abuse report
X-AntiAbuse: Primary Hostname - server.dnsblock1.com
X-AntiAbuse: Original Domain - ietf.org
X-AntiAbuse: Originator/Caller UID/GID - [47 12] / [47 12]
X-AntiAbuse: Sender Address Domain - bobbriscoe.net
X-Get-Message-Sender-Via: server.dnsblock1.com: authenticated_id: in@bobbriscoe.net
X-Authenticated-Sender: server.dnsblock1.com: in@bobbriscoe.net
Archived-At: <https://mailarchive.ietf.org/arch/msg/tcpm/6Ai1c2OrMo596gJlEknLfENKPBc>
Subject: [tcpm] More motivating scenarios for tcpm-ack-pull
X-BeenThere: tcpm@ietf.org
X-Mailman-Version: 2.1.29
Precedence: list
List-Id: TCP Maintenance and Minor Extensions Working Group <tcpm.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/tcpm>, <mailto:tcpm-request@ietf.org?subject=unsubscribe>
List-Archive: <https://mailarchive.ietf.org/arch/browse/tcpm/>
List-Post: <mailto:tcpm@ietf.org>
List-Help: <mailto:tcpm-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/tcpm>, <mailto:tcpm-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 05 Nov 2019 23:14:01 -0000
Carles, Jon, (re-sending, this time without accidentally omitting tcpm, also see couple of addenda tagged [Bob adds:]) I'm generally supportive of a mechanism to suppress delayed ACKs from the receiver (in any transport protocol including TCP). So thank you. You might want to borrow at least some of the text for additional motivating scenarios for such a facility from the first two posts on a thread for a similar facility in QuiC: https://github.com/quicwg/base-drafts/issues/1978 Also pasted at the end. Some is not relevant to your specific ack-pull scheme, 'cos it's proposing a more general facility in QUIC for the sender to the receiver alters its ACK ratio. (theoretically there are more bits available in QUIC, but this particular request has been put on hold while someone goes and finds them ;) Another motivating example (rather niche) Coincidentally, Just this morning I published a tech report that had to use the hack of overlapping a byte from the previous segment to force a quick ack: TCP Prague Fall-back on Detection of a Classic ECN AQM <https://arxiv.org/pdf/1911.00710.pdf#page9> (the link takes you to Fig 2, which illustrates the hack) I'd rather not have to do these sorts of hacks - it really messes with the segmentation code. Middlebox traversal of bit 6 I recall that someone did a study of traversal of each of the reserved flags (Marcelo Bagnulo maybe?). I don't think it was that good [Bob adds: I mean traversal - not the study!]. A good search engine might find it ;) More alternative approaches: 1/ At one stage, we tried to include a bit for Delayed Ack control in another protocol called Accurate ECN. We were persuaded to take it out, because it was mission creep (outside the scope of the protocol we were working on): https://tools.ietf.org/html/draft-kuehlewind-tcpm-accurate-ecn-03#appendix-B.4 2/ Have you considered using the Urgent Pointer in a novel way? I.e. a non-zero Urgent Pointer field, even tho the URG flag is zero. [Bob adds:] From memory, traversal was pretty good - much better than bits 4-6. But I think Windows treated it as a potential attack. I believe Fernando Gont did a study on how 'invalid' TCP header fields are handled. You could assign, say, 3 bits of the urgent pointer for a variable ack_exp that is log base 2 of the ack ratio the sender would like. Then the sender can request the receiver uses ack_ratio` = 2^ack_exp, as in the QUIC proposal below, With ack_exp = 0, you get ack_ratio = 2^0 = 1, which has the same effect as ack-pull. But you also have a more general facility for high speed machines to widen out the ack ratio. If this works, you could open up a registry for the other 13 bits. So instead of using up a precious bit, you generate 13 more ;) See: https://tools.ietf.org/agenda/90/slides/slides-90-tcpm-10.pdf#page=5 I think there were problems with some implementations and middleboxes (bound to be). I've checked the minutes of the IETF meeting where that slide was presented <https://www.ietf.org/proceedings/90/minutes/minutes-90-tcpm>, but there's no push-back in there. I suggest you delve back into the archives of the tcpm mailing list around the time of that meeting to find out if there were any show-stoppers. Cheers Bob Paste from QUIC thread: /\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\/\ Problem/goals: 1. Getting up to speed fast without inducing much queue is one of the main areas where latency reductions are needed (for short and long flows). The sender needs frequent ACKing during this phase in many approaches (hystart, etc). A Linux TCP receiver starts with ack_ratio 1 and uses heuristics to identify the end of the sender's slow-start (or a re-start after idle). This has ossified a certain slow-start behaviour into all TCP senders (whether Linux or not). We are trying to new sender behaviours (paced chirping being an example), but we need a way for the sender to suppress delayed ACKs. 2. At the other end of the scale, for long-running flows, many current CC approaches (e.g. BBR) use pacing not ACK-clocking so they can use very few ACKs per RTT. And fewer means less cache ejections for GRO. But the receiver doesn't know what CC the sender is using, so it doesn't know how many or few is too many or too few. 3. Some middleboxes and the majority of link technologies (e.g. DOCSIS, LTE, Satellite) thin TCP ACKs when they detect the upstream is filled with TCP ACK stream(s), which are unresponsive. Otherwise the ACKs constrain downstream throughput (and any upstream data either in the flow itself, or in others). We don't want these links to attempt to guess which are the QUIC ACKs and try to thin them. QUIC can and should do this itself. QUIC has all the machinery for the sender CC to detect ACK congestion, but not the protocol to tell the receiver to do the thinning. This protocol addition would provide a sufficient hook for hosts to unilaterally add this to their CC behaviour. Some scenarios where the sender's preferred ack_ratio changes through the connection: 1. A sender CC that wants the receiver to turn off DelAcks during flow-start (e.g. it's using hybrid slow-start as in Cubic and wants to get delay measurements more frequently) sets ack_exp=0 during flow-start (ack_ratio=1), then increases ack_exp during congestion avoidance. If it goes idle, then re-starts, it would set ack_exp=0 again. Note on heuristics: A Linux TCP receiver currently uses a heuristic to determine when the sender has exited slow-start. However, heuristics -> ossification. A Linux receiver's heuristic only works with the current pattern of slow-start. In TCP, when we tried to improve the pattern on the sender (paced chirping), the heuristic on the receiver killed us. 2. Imagine a paced sender has hardware generic receive offload (GRO), so for a long-running flow it doesn't want a high rate of QUIC ACKs that are opaque to GRO. Let's say it would prefer at least 8 ACKs per RTT. Again, it starts with ack_exp=0, but in congestion avoidance it would use: ack_ratio <= cwnd_in_packets/8 Upshot: By remote controlling the receiver, the server offloads nearly all the ACKs from large downloads, but still focuses its ACK-receiving resources on getting each client up to speed. Note that calculation of ack_ratio lends itself to fast integer arithmetic. Bob -- ________________________________________________________________ Bob Briscoehttp://bobbriscoe.net/
- [tcpm] More motivating scenarios for tcpm-ack-pull Bob Briscoe
- Re: [tcpm] More motivating scenarios for tcpm-ack… Jon Crowcroft
- Re: [tcpm] More motivating scenarios for tcpm-ack… Bob Briscoe
- Re: [tcpm] More motivating scenarios for tcpm-ack… Carles Gomez Montenegro
- Re: [tcpm] More motivating scenarios for tcpm-ack… Jon Crowcroft
- Re: [tcpm] More motivating scenarios for tcpm-ack… Scheffenegger, Richard
- Re: [tcpm] More motivating scenarios for tcpm-ack… Carles Gomez Montenegro
- Re: [tcpm] More motivating scenarios for tcpm-ack… Jeremy Harris
- Re: [tcpm] More motivating scenarios for tcpm-ack… Gorry Fairhurst
- Re: [tcpm] More motivating scenarios for tcpm-ack… Jonathan Morton
- Re: [tcpm] More motivating scenarios for tcpm-ack… Scheffenegger, Richard
- Re: [tcpm] More motivating scenarios for tcpm-ack… Bob Briscoe
- Re: [tcpm] More motivating scenarios for tcpm-ack… Bob Briscoe
- Re: [tcpm] More motivating scenarios for tcpm-ack… Carles Gomez Montenegro
- Re: [tcpm] More motivating scenarios for tcpm-ack… Carles Gomez Montenegro
- Re: [tcpm] More motivating scenarios for tcpm-ack… Bob Briscoe
- Re: [tcpm] More motivating scenarios for tcpm-ack… Jonathan Morton
- [tcpm] Sender control of Delayed ACKs (was Re: Mo… Carles Gomez Montenegro
- Re: [tcpm] More motivating scenarios for tcpm-ack… Carles Gomez Montenegro
- Re: [tcpm] Sender control of Delayed ACKs (was Re… Bob Briscoe
- Re: [tcpm] Sender control of Delayed ACKs (was Re… Rahul Arvind Jadhav
- Re: [tcpm] Sender control of Delayed ACKs (was Re… Bob Briscoe
- Re: [tcpm] Sender control of Delayed ACKs (was Re… Carles Gomez Montenegro