Mime-Version: 1.0 (Apple Message framework v1251.1)
Content-Type: text/plain; charset=windows-1252
From: Toby Moncaster <toby.moncaster@cl.cam.ac.uk>
In-Reply-To: <20111205230153.GC39149@verdi>
Date: Tue, 6 Dec 2011 09:50:27 +0000
Content-Transfer-Encoding: quoted-printable
Message-Id: <9BD81879-81A2-4EF0-A60B-F541D0BA418B@cl.cam.ac.uk>
References: <CAH56bmD2fh3sm4mozh17K2C+K0Pxyw7vRvykCo9Xt-jeEP36ZQ@mail.gmail.com>
 <201111201956.pAKJuJSQ007421@bagheera.jungle.bt.co.uk>
 <20111120214012.GE22465@verdi>
 <201111202327.pAKNRJPT008060@bagheera.jungle.bt.co.uk>
 <20111121203356.GG22465@verdi>
 <201111212314.pALNEhvZ013554@bagheera.jungle.bt.co.uk>
 <20111122001928.GH22465@verdi>
 <82AB329A76E2484D934BBCA77E9F524924B97E81@Polydeuces.office.hd>
 <20111202232051.GH31463@verdi>
 <Prayer.1.3.4.1112031010520.17047@hermes-2.csi.cam.ac.uk>
 <20111205230153.GC39149@verdi>
To: John Leslie <john@jlc.net>
Sender: "T. Moncaster" <tm444@hermes.cam.ac.uk>
Cc: "T. Moncaster" <tm444@cam.ac.uk>, ConEx IETF list <conex@ietf.org>
Subject: Re: [conex] byte vs packet counting
Precedence: list

On 5 Dec 2011, at 23:01, John Leslie wrote:

> T. Moncaster <tm444@cam.ac.uk> wrote:
>>=20
>> I have been purposely not contributing to this thread over the past =
few=20
>> weeks as I wasn't sure that more voices in the argument would =
actually
>> help things.
>=20
>   IMHO, _any_ additional voices would help. (Of course, I may be =
biased:
> if nobody takes my ideas seriously it's easier to declare me "in the
> rough"...)
>=20
>> However I feel things are now at a point where I can contribute
>> something (hopefully) useful.
>=20
>   Toby has _always_ had something useful to contribute!!
>=20
>> On Dec 2 2011, John Leslie wrote:
>>> Dirk Kutscher <Dirk.Kutscher@neclab.eu> wrote:
>>>> [John Leslie <john@jlc.net> wrote:]
>>>>=20
>>>> Auditing is really a fundamental piece of the basic Conex =
mechanism,
>>>=20
>>> I'd be happy to discuss that. I see no reason to bother auditing
>>> if the entire path is known to be free of congestion. Auditing might
>>> make sense if you expect traffic you send to encounter congestion
>>> later in the path, but only if you have some contractual arrangement
>>> calling for it. (Otherwise, you can't know what the effect of
>>> ConEx marks may be.)
>>=20
>> I think there is a subtle difference between stating that auditing
>> is a fundamental part of ConEx and saying that auditing can never
>> be turned off.=20
>=20
>   Agreed... but it's not obvious which of these Dirk meant. :^(
>=20
>> I would definitely agree that auditing is required in any network
>> where there are potential trust issues and where you can't guarantee
>> the whole-path network conditions.
>=20
>   In other words, any real-world network=85

indeed

>=20
>> But I do think there are cases where it could be turned off (for
>> instance in private corporate networks using conex as a means to
>> control their monthly network bills).
>=20
>   Is that really so different from real-world networks?

The main difference is that they may have an additional "big stick" that =
they can beat their users with if they are found to have cheated. In =
other words you can make the trust  separate issue.

>=20
>   (An aside: I talked with Matt today, and we're making progress on
> understanding each other on what the WG should mean by "auditing".
> Stay tuned=85)

That sounds like an interesting chat - looking forward to seeing the =
output of it.

>=20
>>> Auditing every packet is plausible near the sender and receiver,
>>> but not in the backbone. I don't believe we have agreement among
>>> ourselves what a receiver's uplink will do if their auditing shows
>>> "abuse" of ConEx marking: so I don't see what auditing at the
>>> sender's uplink can do and be confident it's the right thing.
>>=20
>> I agree, it seems hard to see how auditing something when you are
>> missing half the information is going to be hard.
>=20
>   (I'm having trouble parsing that sentence.)

I think we are hitting the issue of "what is audit?" If audit is taken =
to mean accounting for ConEx marks, then that can happen anywhere. =
However I was taking auditing to mean accounting for the balance of =
ConEx marks with actual congestion. That function can only work =
somewhere where you can see both the congestion and the ConEx marks. The =
only way you can achieve that at the sender side is by inspecting the =
ACKs on the return path (which may be impossible).

>=20
>> There are possible arguments along the lines of an ISP might make
>> a decision that all users of a given access node (B-RAS or whatever)
>> are likely to be experiencing roughly similar amounts of congestion.
>=20
>   (Seems unlikely to me=85)

Oh, patently unlikely, but it is the sort of thing I can (sadly) imagine =
some ISPs doing.

>=20
>> So any customer that declares no congestion is probably not being
>> honest.
>=20
>   This sort of assumption bothers me. What if that customer is using
> a particularly good LBE protocol?

Well quite - that is the argument against blanket rate limiting of all =
P2P, even if it is running on LBE transport.

>=20
>>>> Having said that, it is certainly implementation-specific how
>>>> auditing really works -- there may be tradeoffs between efficiency
>>>> and effectiveness -- but for the sake of the discussion here, I'd
>>>> say it's best to assume that, yes, auditing evaluates every packet.
>>>=20
>>> IMHO, it's _always_ best to assume otherwise.
>>=20
>> My view is that we're arguing the wrong point - auditing every packet
>> seems trivial if we are assuming that most routers can mark a packet
>> (e.g. can do ECN). In that case they are already inspecting 2 bits of
>> the IP header, so asking them to inspect a 3rd bit seems reasonable.
>=20
>   (Talking with Matt today, I am beginning to understand he may be
> assuming that "auditing" measures only ConEx marks versus ECN marks
> for a particular packet; and that the action taken on that basis
> will be simply and only dropping the packet where the imbalance is
> detected, keeping no state. That much could be implemented anywhere;
> but it falls far short of what we are led to believe is promised by
> his "auditing function". I'm hoping the next revision will assign
> more intuitive names to whatever his current "auditing function"
> proposes=85)

That has been sort of my assumption, but with the proviso that you can =
only really tell the imbalance over a set of packets (see below)

>=20
>> However, on a per-packet basis audit is meaningless.
>=20
>   I think I agree with Toby.
>=20
>> Audit only has any meaning over a period of time across a given flow
>> (or set of flows).
>=20
>   There's a pretty big difference between auditing a particular
> flow versus auditing the particular set of flows represented by all
> traffic an ISP receives from a peer at a peering point... I'm not
> sure exactly what Toby refers to here.

Audit has two granularities - either you are auditing a given user to =
ensure his flow (or flows if he is using several) are correctly =
reporting the expected level of congestion, or you are auditing a whole =
peer network to ensure they aren't allowing excess congestion to enter =
your network without it being declared. These require (subtly?) =
different audit mechanisms

>=20
>> There is a separate accounting mechanism where you can measure bulk
>> flows of congestion, but that has to be based on a belief in the
>> accuracy of the marking of the underlying individual flows.
>=20
>   I don't see how that follows.

Measuring the bulk is very easy, but if all the flows making up the bulk =
are understating their congestion then the bulk will be understated as =
well. Bob always had in mind a clever mechanism to do spot checking at =
any border to identify persistent under (or over) declaration, but I =
never entirely understood how it was meant to work.

>=20
>>> Your life will be easier if you don't depend on assumptions which
>>> are likely to be false.
>>>=20
>>> We'd do better, when discussing this, to get specific about how
>>> auditing might improve the use-cases we're pushing.
>>>=20
>>> Unless I misread draft-ietf-conex-concepts-uses, our principle
>>> use case is Informing Traffic Management. I have posted before that
>>> the most important feature of ConEx marking (IMHO) is informing
>>> any node along the way that the sender is knowingly sending packets
>>> despite having reason to doubt that congestion has cleared.
>>>=20
>>> Auditing does nothing to help here. The abusive thing to do is
>>> to _clear_ ConEx marks -- and auditing can't detect that.
>>=20
>> No. ConEx is about a balance of markings. To cheat you fail to send
>> forward into the network sufficient ConEx markings to cover the
>> congestion your flow is causing (or has caused).
>=20
>   Not quite... an ISP can cheat by clearing marks which one sender
> placed in his outgoing traffic -- that is what I meant to refer to.

OK, yes. That is a separate requirement for audit.=20

>=20
>> So the function of auditing is to check that the two numbers
>> roughly match.
>=20
>   (I note that Toby says "roughly match". Others seem to be assuming
> a much stricter match. I'm afraid we don't have a clear understanding
> here.)

Clearly, given the min 1RTT delay between getting congestion info and =
responding to it, there can never be a completely accurate match. I did =
have numerous debates with Bob about this topic. Basically you run the =
risk of a user just terminating his flow rather than making up his =
losses. You also have to be cautious that the actual process of dropping =
packets for audit doesn't simply distort the apparent congestion =
information (because the sender will respond by retransmitting and =
hopefully adding more ConEx marks)

>=20
>> And yes, this is a very difficult problem to solve if you can't
>> assume ECN is being used.
>=20
>   I absolutely agree with Toby here.
>=20
>>> Auditing, to tell truth, may or may not prove useful after we
>>> have done enough experiments. IMHO, if it proves useful we'll find
>>> that it's useful when done statistically, rather than on every
>>> packet.
>>=20
>> I think this may point to the heart of the issue. Yes, you can sample
>> to do audit, but only if you assume a relatively high density of
>> markings.
>=20
>   I don't follow why that is true. If auditing is not strict, sampling
> should work for a wide range of density; if auditing is strict, it
> will fail pretty much regardless of the density=85

OK, this comes down to the point below

>=20
>> If ConEx info is unary encoded into packets then you can only measure
>> it by measuring long streams of the packets.
>=20
>   I agree with what I think Toby intends to say: that the congestion
> metric of ConEx is a fraction, not a binary function per packet.

Which follows very naturally from ECN - ECN (RED) randomly decides =
whether or not to mark a packet, with the aim that the overall number of =
marks (very roughly) reflects the congestion level experienced at that =
queue. Because this is a random unary scheme, any ConEx mechanism based =
on ECN (and that is trying to audit against current network conditions) =
will need to track this over a few packets to see how many ConEx marks =
it should expect. OR the sender has to have sent sufficient credit up =
front.

Even in a scheme that is trying to identify either gaps in sequence =
space (dangerous - especially in an MPTCP world) or is trying to =
identify retransmissions (again, dangerous in MPTCP), then you have to =
have some sort of memory of what went before (a counter of congestion =
seen against ConEx credit seen for instance).

>=20
>> If ConEx is a specific number carried in each packet then sparse
>> sampling may well give you enough to be able to achieve the aims of
>> audit.
>=20
>   Carrying what I call "congestion-expected" as a multi-bit fraction
> is an interesting idea, in which I see potential benefits. I'd be
> happy to discuss it, but my impression is that we're not heading in
> that direction.

That was certainly how I understood a re-ECN like scheme working=85 Te =
whole core of this argument (should you account for bytes or packets) =
has always been a major bone of contention between me and Bob. Not least =
because it can be impossible for a sender to comply (it is very easy to =
construct pathological sequences of packets where the sender is forced =
to either over or under declare if they are doing per-byte auditing). I =
think this is a case where the IETF view has to diverge from the =
researcher's view. In the IETF we have to make prosaic engineering =
decisions. Personally I would like ConEx to account on a per-packet =
basis, BUT with an allowance for it to be more accurate if an operator =
so wishes (in other words MUST as a minimum account per packet, but MAY =
account per byte)

>=20
>> Which brings me on to the aims of audit. To my mind there are two
>> possible aims.
>> 1) to guarantee that every single packet, flow, aggregation of =
traffic
>>   in the network has accurately declared their congestion; or
>> 2) to act as a "speed camera" that catches a sufficiently high
>>   proportion of infringements to ensure that senders don't risk
>>   under-declaring their congestion.
>=20
>   I don't see auditing as either of these; but that may be more a
> function of my tendency to think of auditing as gathering information,
> not enforcing disciplinary measures based on this information.

Perhaps auditing is an unhelpful word to use. To my mind an audit =
implies an accountancy audit that tries to balance the books. I agree =
that carries no implication of policing or enforcement, but I think that =
a lot of this debate has assumed the two concepts are identical

>=20
>   IMHO, "guaranteeing" much of anything isn't worth the effort over
> organizational boundaries. At significant expense, we could use
> cryptography to give arbitrarily high confidence to a particular
> "guarantee" having come from a particular entity, but without also
> signing the data "guaranteed", the effort is entirely wasted.
>=20
>   (And processing all that cryptography in backbone routers seems
> a non-starter to me.)

Agreed, this has to be a lightweight mechanism. All the guarantees have =
to come from the idea that operators have to have some level of trust =
among one another (or at least, the realisation that an operator found =
to have cheated at the expense of others runs  risk of getting sat on =
heavily by the others),

>=20
>> My own view is that the second is more feasible in the real world,
>> although the first might well be more desirable.
>=20
>   I at least partly agree here. However, my approach to thinking
> about such problems simply stops descending into details when I reach
> a feasibility blockage.

And therein lies the lesson for ConEx. We HAVE to fix on a SIMPLE subset =
of ConEx that we can all agree on and that seems at least feasible in =
the real world. Arguments like the recent ones make ConEx appear to be =
still firmly stuck in the land of research (IRTF). Too many people (on =
all sides of all arguments) seem to want perfection and that is not =
sensible in the real world. That is just a recipe for ratholing =
ourselves to the point where the IESG declares our WG dysfunctional=85=20=


Toby


>=20
> --
> John Leslie <john@jlc.net>