Re: [re-ECN] CONEX Principles & Constraints

[John Leslie <[e-mail]>]

[e-mail] <[e-mail]> wrote:
>> [e-mail]
>> 
>> 1. the purpose is to standardise the conex protocol...
> 
> I might even go so far as saying "Potential uses for such information
> are explicitly beyond the scope of this initial
> {WG/BoF/protocol/whatever}"

   "the planned WG".

>> 2. The protocol should be open about the responses it allows to the
>> conex information. For example, it should not make assumptions about
>> the behaviour of a particular application, and it should allow a
>> diversity of potential traffic management practices (of networks and
>> end-hosts).

   Diversity of traffic management practices is critical: I'm not sure
whether the rest of this paragraph helps or tends to confuse.

>> 3. the conex information needs to be easily visible to all network
>> elements, and be application-independent (including encrypted apps).
>> Reason: this ensures widest possible applicability -- it is visible
>> to both end-systems (to help them control their sending rate) and to
>> network managers (to help them manage a bottleneck, whether traffic
>> originates from their own users or from other networks).
> 
> I would either merge this with the later points about IP or at least
> re-order the list so they occur in a row.

   I wouldn't recommend "merging" this with anything else.

>> 4. the conex information needs to be sufficiently up-to-date, for
>> example so that it can usefully assist congestion control by end
>> systems. Reason: stale information is little use. [Question:
>> instead of "sufficiently up-to-date", would "real-time" or "near
>> real-time" be better?]
> 
> How about "Sufficiently up-to-date to enable near-real-time traffic
> management". 

   I think Toby captures the essential point: that this needs to be
used for real-time traffic management. I don't think "up-to-date"
helps.

> Incidentally this is nothing to do with congestion control at
> end-systems - the end-system already sees congestion! So the only
> link to congestion control is if this in turn allows the network
> to be more lenient towards end-systems.

   "Usefully assist congestion control by end systems" isn't really
the point. While we envision a system where ECN marking is "re-inserted
into the stream being sent," this is mostly unrelated to what the end
user application and/or stack does to control congestion.

   I'm not sure what we can say about how this benefits end users.
I do imagine some overlap with the Multipath TCP WG, but it's way too
early for any specifics.

   Our point really is that ConEx marking can enable network operators
to distinguish "heavy" traffic from "congestion-causing" traffic, thus
enabling experimentation with transport algorithms that do a better
job of filling the pipes without harming other uses.

>> 5. the conex information needs to be sufficiently granular, to allow
>> identification of those sending traffic to the bottleneck [Comment:
>> this is trying to get a point that John made about 'granular enough
>> for cost allocation' but in a more generic way.

   I guess I should comment on that.

   In hindsight, I was already too "generic" when I wrote that phrase.
I meant allocation within the accounting system of the network operator
that might perform the upgrade; but I wished to allow for policing
which either charged for congestion from a particular forwarder or
dropped / delayed / whatever packets that would "cause" more congestion
than "normal".

   The point I wanted to make is that upgrading a bottleneck link needs
cost justification, and we need to collect information that can justify
the cost.

>> "Identification" is a bad word as, for some readers, it might be
>> imply more than intended

   Agreed.

>> - but I can't think of a better one at the moment.

   Better to leave out anything that even resembles "identification"...
The point is strictly related to who _forwards_ packets, not who _sends_
them; and how that is determined is pretty strictly outside the scope
of ConEx.

   The point of "granularity" is that we can't afford to try to aggregate
to "flows" and then de-aggregate to portions of those "flows" that cause
congestion: we need to operate packet-by-packet, determining a probability
that the packet in question will "cause" congestion.

>> I hope that the following bullet about conex information in the IP
>> header solves how to do it.]
> 
> The information needs to granular enough to hold the originator of the
> traffic accountable?

   I hope not!

>> 6. the conex information is carried in the IP header. Note: it is in
>> scope to work out which IP header bits to use (and the implications
>> for other stuff that may already be using those bits).

   This is really weak on justification for the use of scarce bits...

> Better: "This may require defining new codepoints in the IP header or
> possibly even the re-definition of existing codepoints. We will be
> careful to consider the impact of our choices both on existing protocols
> and on deployability.

   I would try to avoid trespassing on that ground.

   While there is space in the IPv6 header, we want to experiment in
IPv4-land. Talking of "defining new codepoints" is likely to get our
scope limited to IPv6. Better to experiment in IPv4, and perhaps _then_
define IPv6 codepoints.

   BTW, I firmly believe the "granularity" issue is essential to justify
experimenting with IPv4 codepoints: if we don't get buy-in to the need
for granularity, the whole experiment is endangered.

>> 7. a protocol will be developed for both IPv4 and IPv6. Reason: both
>> are needed for full applicability. (There was push-back on an earlier
>> suggestion of narrowing the scope to IPv4.)

   I agree, but justification is weak. Bob Briscoe should clarify how
we'd experiment in IPv6 space: I believe that would be very good;
however, I suspect we'll eventually find a better way to do congestion
exposure in IPv6, and we shouldn't limit ourselves to designing the
final IPv6 solution too early.

> I might consider phrasing this "although on the face of it an IPv6
> protocol may be easier, we will also develop solutions for IPv4 as it is
> still the dominant network-layer protocol.."

   As I've already said, we need to _experiment_ in IPv4 in order to
collect sufficient data and observe how congestion exposure information
is actually used. I would avoid calling the IPv4 stuff "solutions"

>> 8. TCP options will be used to inform the sender about congestion
>> received at the receiver. (Note, current proposals require this
>> information transfer.) Reason: narrow the scope by looking only at
>> TCP, which is used by the vast majority of the traffic (~85-90%).

   I thorouhly disagree!

   Limiting the scope to TCP would be fatal. We need to collect data
on congestion caused by UDP packets; and any limitation to particular
transport(s) incents the use of other transports to "end-run" ConEx.

> Do we want to specify using TCP options at the moment? Option space is
> limited and already used for lots of other things + MPTCP will be using
> even more (and there is a very high chance of eventual overlap between
> conex and MPTCP). I would prefer to leave this as "we will concentrate
> on TCP as this provides a feedback path..."

   TCP option "space" is limited in that packets are limited to 44 bytes
of options per packet. We _would_ get burned by that limit. :^(

   More importantly, we _really_ don't want to wander into the weeds of
how to juggle TCP options when we're dealing with the forwarding of IP
packets. (Yes, I know we do have folks clever enough to pull it off
99.999% of the time; but the rest of us will quickly lose track of the
details, and sooner or later the folks that are clever enough will find
themselves on the "rough" end of "rough consensus.)

>> 9. a baseline assumption is that routers are unmodified. Reason:
>> deployment viability
> 
> "This MUST work with unmodified routers, even if they don't do ECN...
> although modifying routers may allow additional refinements...

   A reasonable way to say it...

>> 10. a baseline assumption is that conex information is signalled
>> explicitly (through the IP header) and not implicitly signalled
>> through packet loss, delay or jitter. Reason: explicit signalling
>> allows a "better" reaction, for example faster, without impairment

   Pretty good... but the justification needs to be stronger.

   First of all, explicit signaling can carry more-than-one bit,
instead of less-than-one (due to the multitude of other things that
can lead to packet loss); and can reach the actor who needs to react
significantly faster.

   Second, explicit signaling carries consistent information to every
router along the path, meaning that incentives to slow down a sending
rate can be applied "close enough" to the sender to be effective; and
if a packet actually does need to be dropped, it could be dropped
before it causes other congestion earlier in the forwarding path.

> In theory implicit signalling already exists - if you observe a TCP
> source halving its rate you can assume it was (probably) in response to
> a loss event...

   Hmm... that's getting "too clever" for this bear-of-small-brain...

>> 11. but we will also develop a solution where the conex information
>> is signalled through packet loss. Reason: ECN is not enabled in most
>> routers today; incremental deployment is critical

   Am I missing how to "re-insert" that signal?

> No! - we may develop a solution that informs the network of congestion
> the sender has learnt about both as a result of explicit notification
> (ECN) and implicit notification (loss). However we DON'T use loss to
> signal ConEx information (which is surely impossible anyway?)! 

   I strongly recommend against any statement along those lines.

>> 12. (possibly) we will study an incremental deployment step where only
>> the sender is conex-enabled [Step suggested by Matt]
> 
> We certainly need to leave this open as a possible step

   I don't see any reason to mention this. Clearly, a ConEx-aware sender
can re-insert a signal whether or not the end-receiver is ConEx aware.
And that signal can enable all the experimenting we need along the
forwarding path.

   We need only specify the _meaning_ of the ConEx signal the sender
re-inserts -- not any particular algorithm for generating it.

>> 13. (possibly) another incremental deployment step?

   Way premature, IMHO.

>> 14. we will describe threats, especially from falsifying or
>> suppressing the conex information - whether by ISPs or end-hosts.
>> The protocol needs to allow such threats to be dealt with (in as
>> simple a manner as possible), but the actual mechanisms to tackle
>> those threats (using policer boxes, for instance) are out of scope.
>> Reason: such mechanisms are about uses of conex, rather than exposing
>> the information.

   I agree.

>> 15. anything else?

   Not at this time.

--
John Leslie <[e-mail]>