Re: [mpls] [PWE3] Pseudowire Sequence Number in draft-ietf-mpls-forwarding: a minor comment
Alexander Vainshtein <Alexander.Vainshtein@ecitele.com> Thu, 12 December 2013 05:49 UTC
Return-Path: <Alexander.Vainshtein@ecitele.com>
X-Original-To: mpls@ietfa.amsl.com
Delivered-To: mpls@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 518F51AE006; Wed, 11 Dec 2013 21:49:40 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -2.601
X-Spam-Level:
X-Spam-Status: No, score=-2.601 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id VxtZXleP4YZZ; Wed, 11 Dec 2013 21:49:35 -0800 (PST)
Received: from emea01-am1-obe.outbound.protection.outlook.com (mail-am1lp0012.outbound.protection.outlook.com [213.199.154.12]) by ietfa.amsl.com (Postfix) with ESMTP id 6ACBE1A802D; Wed, 11 Dec 2013 21:49:32 -0800 (PST)
Received: from AM3PR03MB529.eurprd03.prod.outlook.com (10.242.109.153) by AM3PR03MB401.eurprd03.prod.outlook.com (10.242.110.14) with Microsoft SMTP Server (TLS) id 15.0.837.10; Thu, 12 Dec 2013 05:49:26 +0000
Received: from AM3PR03MB532.eurprd03.prod.outlook.com (10.242.109.156) by AM3PR03MB529.eurprd03.prod.outlook.com (10.242.109.153) with Microsoft SMTP Server (TLS) id 15.0.842.7; Thu, 12 Dec 2013 05:49:24 +0000
Received: from AM3PR03MB532.eurprd03.prod.outlook.com ([10.242.109.156]) by AM3PR03MB532.eurprd03.prod.outlook.com ([10.242.109.156]) with mapi id 15.00.0842.003; Thu, 12 Dec 2013 05:49:23 +0000
From: Alexander Vainshtein <Alexander.Vainshtein@ecitele.com>
To: "curtis@ipv6.occnc.com" <curtis@ipv6.occnc.com>
Thread-Topic: [PWE3] Pseudowire Sequence Number in draft-ietf-mpls-forwarding: a minor comment
Thread-Index: AQHO9vYcSq0tCZZscUehMpHq+OAdtZpQDFrZ
Date: Thu, 12 Dec 2013 05:49:22 +0000
Message-ID: <4946c79f4554497d84cd676684739124@AM3PR03MB532.eurprd03.prod.outlook.com>
References: Your message of "Wed, 11 Dec 2013 06:05:20 +0000." <774d74926b684ffba7f4f89ace758547@AM3PR03MB532.eurprd03.prod.outlook.com>, <201312120453.rBC4rYp4065163@gateway1.ipv6.occnc.com>
In-Reply-To: <201312120453.rBC4rYp4065163@gateway1.ipv6.occnc.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
x-originating-ip: [5.153.9.202]
x-forefront-prvs: 0058ABBBC7
x-forefront-antispam-report: SFV:NSPM; SFS:(10009001)(13464003)(53754006)(189002)(199002)(43784003)(377454003)(52604005)(51704005)(76482001)(47736001)(50986001)(47976001)(81342001)(4396001)(81816001)(76576001)(74662001)(47446002)(74502001)(76786001)(31966008)(74316001)(49866001)(71446004)(74366001)(56816005)(90146001)(66066001)(74706001)(63696002)(76796001)(85306002)(65816001)(80022001)(87266001)(87936001)(19580405001)(19580395003)(80976001)(2656002)(46102001)(83322001)(56776001)(79102001)(74876001)(77982001)(59766001)(83072002)(81542001)(51856001)(85852003)(33646001)(69226001)(54356001)(81686001)(54316002)(53806001)(427584002)(24736002); DIR:OUT; SFP:1101; SCL:1; SRVR:AM3PR03MB529; H:AM3PR03MB532.eurprd03.prod.outlook.com; CLIP:5.153.9.202; FPR:; RD:InfoNoRecords; MX:1; A:1; LANG:en;
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: ecitele.com
Cc: "samante@apple.com" <samante@apple.com>, "kireeti@juniper.net" <kireeti@juniper.net>, "mpls@ietf.org" <mpls@ietf.org>, "curtis@occnc.com" <curtis@occnc.com>, "pwe3 (pwe3@ietf.org)" <pwe3@ietf.org>
Subject: Re: [mpls] [PWE3] Pseudowire Sequence Number in draft-ietf-mpls-forwarding: a minor comment
X-BeenThere: mpls@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: Multi-Protocol Label Switching WG <mpls.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/mpls>, <mailto:mpls-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/mpls/>
List-Post: <mailto:mpls@ietf.org>
List-Help: <mailto:mpls-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/mpls>, <mailto:mpls-request@ietf.org?subject=subscribe>
X-List-Received-Date: Thu, 12 Dec 2013 05:49:40 -0000
Curtis, Lots of hanks for a long and detailed explanation. It appears that your original comment about the need of jitter buffer refered to TDM (or TDM-like) PWs (bit-stream PWs in the Architecture RFC). If this was indeed the case, there is nothing to argue about, because I agree completely that jitter buffer is absolutely necessary for these. My impression from reading the original text has been that it referred to ALL kinds of PWs (bit-stream, celland packet) alike. Apparently this has been my mis-interptetation; if this is indeed so, then maybe some clarification (placing the jiter buffer discussion in the proper context) would hopefully help. Regards, Sasha ________________________________________ From: Curtis Villamizar <curtis@ipv6.occnc.com> Sent: Thursday, December 12, 2013 6:53 AM To: Alexander Vainshtein Cc: curtis@occnc.com; curtis@ipv6.occnc.com; kireeti@juniper.net; samante@apple.com; agmalis@gmail.com; cpignata@cisco.com; mpls@ietf.org; Yaakov Stein (yaakov_s@rad.com); pwe3 (pwe3@ietf.org) Subject: Re: [PWE3] Pseudowire Sequence Number in draft-ietf-mpls-forwarding: a minor comment In message <774d74926b684ffba7f4f89ace758547@AM3PR03MB532.eurprd03.prod.outlook.com> Alexander Vainshtein writes: > Curtis, > Lots of thanks for sending out the new text. > > I still have a couple of issues with this version of the text: one > technical and one editorial. > > The technical issue is about the fragment "A small jitter buffer is > always necessary". > > I am not sure this is correct, since I am aware of several PW > implementations that pass the payload of a received PW packet directly > for transmission via the corresdponding attachment circuit. Of course > this payload would be stored in some Tx queue associated with the > attachment circuit and controlled by the appropriate egress scheduler, > but IMO this does not constitue a PW jitter bufer, since the PW > interworking function does not exersize any control over these > queues. Well if its a technical argument you want, then here goes ... Even in pure TDM systems some jitter exist due to the longer time it takes if multiple passes of the FEC algorithm are needed. In that case there is a very small jitter buffer that can be set to the maximum needed for a worst case FEC, or set lower to decrease delay with the potential to make a greater number of errors uncorrectable. For IP, even if IP EF service is used and there is a very small amount of EF traffic, some jitter exists. For most systems there is some jitter crossing the fabric and internal jitter elsewhere. The egress buffer in most chips is very small but needed for that jitter. For TDM, the egress buffer can never run dry or there will be a timing slip. Consider what happens when packets arrive late in a system with only a non-zero egress buffer. For example, assume 1% of packets arrive about one packet time late (one other EF packet in queue somewhere on the path), 0.1% arrive two packets late. Note that this would be an extremely light EF load. Generally most packets arrive at least zero to one minus epsilon packet times late due to a lower priority packet partially transmited when the EF packet is moved to the front of the queue. If there is no egress buffer at all or considerably less than a packet time, lots of data would be dropped. If the egress buffer could hold just a few packets data drop would be very low. If a packet arrives late and there is no standing queue the egress runs dry, there is a timing slip. When the packet does arrive a little late the packet is sent, albiet late. Every packet to follow is now late by that amount of time (if the egress buffer has not overflowed on a later packet). At that point there is an standing queue in the egress buffer which acts as a jitter buffer. Most packets arrive early. A few arrive a little later but as long as the queue doesn't run dry again, the jitter is compensated for. If the egress buffer is large, then it will tend to grow to the worst case packet delay and keep a large standing queue. This is with no reorder and no drop. If the egress buffer is small (or configured to be small) then when a packet arrives very late followed by a set of packets in close succession, the buffer runs dry until that first late packet arrives. This packet is transmitted but very late. Subsequent packets arrive and some of them have to be dropped due to the limited queue capacity. This in effect puts a cap on the jitter buffer. With only an egress buffer there are multiple strategies for dealing with dropped packets. 1. Just ignore it and send the next one. 2. Put one packet time worth of all zeros in the queue. Now consider what happens with a dropped packet and a standing queue. The first strategy creates a hidden timing slip. The second strategy avoids the timing slip. So my argument is that the egress buffer is in effect the jitter buffer. When jitter occurs a standing buffer forms that reflects the lesser of the limit of the buffer size or the worst case jitter. There is always a jitter buffer. There always needs to be a jitter buffer of at least one large packet time, otherwise high loss of data will occur. There should be a jitter buffer that can be made larger so that jitter of multiple packet times doesn't turn in to loss. Therefore I assert that a jitter buffer is always present and is always needed because jitter is always non-zero. Therefore this statement is true: A small jitter buffer is always necessary and the jitter buffer is needed regardless of whether reordering is done. > In particular, these queues could not be used for re-ordering > misordered PW packets. That does not mean that they can't be used to compensate for jitter which is by definition what a jitter buffer is for. When reordering is done, the egress buffer is made small and a buffer preceding that is used which has some ability to reorder packets. In hardware a circular queue makes sense with packets place according to their sequence number. If you prescribe to the "just insert zeros" idea when a packet is dropped, then after moving a packet to the egress queue, zero the buffer or mark it as unoccupied and more the circular buffer a bit. If the packet that is later supposed to occupy that slot never arrives, then it is already zerod (or marked as missing in which case a packet of all zeros is put on the egress queue). So ... reording within a small jitter buffer is not hard to do. Reordering can be supported with a small jitter buffer but adding reordering to a small buffer may be ineffective. If reordering is due to a path change, then the buffer of a small number of packet times may yield no improvement. To be effective in that situation, the buffer may need to be larger. OTOH if moving EF traffic from one queue with no standing EF queue to another with no standing EF queue and the path change is just moving to another wave on the WDM, reordering with a small jitter buffer may help. > IMHO and FWIW this understanding matches RFC 3985 which mentions (in > Section 5.2.1.1 "Frame Ordering") that reodering as the strategy for > handling mis-ordered packets introduces additional delay (i.e., > requires a ddicated jitter buffer), while the alternative strategy of > discarding mis-ordered packets does not introduce such additional > delay. You quoted the framework and not a TDM document. Nowhere in that text is there mention of a jitter buffer because that text is not specific to TDM. For packet traffic you have no idea if a prior packet is dropped or will arrive out of order and so when a drop occurs packets have to be held for a non-zero amount of time rather than transmitted as soon as they arrive. This applies to all cell or packet AC, such as FR, ATM, Ethernet, etc. You might want to look at section 5.2.2.1. Clock Recovery which does apply to TDM. It points to RFC3550 (RTP) which assumes a jitter buffer is always needed. > The editorial issue is about the entire sentence that contains the > above-mentioned fragment, i.e., "A small jitter buffer is always > necessary and the jitter buffer is needed regardless of whether > reordering is done" - to me this looks like an unnecessary repetition > if if it were techically correct. The second part of the sentence was to emphasis that the need for a jitter buffer is not contingent on implementation or reordering. Perhaps it would be more clear if the two points were separated. How about this OLD new sentence A small jitter buffer is always necessary and the jitter buffer is needed regardless of whether reordering is done. NEW new sentence A small jitter buffer is always necessary. A jitter buffer is needed regardless of whether reordering is done. > Hence I would like to suggest discrading the problematic sentence as > the simplest way to resolve both issues. I think the sentence provides an important clarification. I think the need to have this discussion proves that the clarification is needed. If we can agree on keeping this, I can submit the updated draft. > Hopefully these notes will e helpful. > > Regards, > Sasha Thanks again for the review and good comments. Regards, Curtis > ________________________________________ > From: Curtis Villamizar <curtis@occnc.com> > Sent: Tuesday, December 10, 2013 5:47 PM > To: Alexander Vainshtein > Cc: curtis@ipv6.occnc.com; curtis@occnc.com; kireeti@juniper.net; samante@apple.com; agmalis@gmail.com; cpignata@cisco.com; mpls@ietf.org; Yaakov Stein (yaakov_s@rad.com); pwe3 (pwe3@ietf.org) > Subject: Re: [PWE3] Pseudowire Sequence Number in draft-ietf-mpls-forwarding: a minor comment > > In message <F9336571731ADE42A5397FC831CEAA025AC56C9D@ILPTWPVEXMB02.ecitele.com> > Alexander Vainshtein writes: > > > > Curtis, > > > > Lots of thanks for a prompt and very detailed response. > > May I suggest an alternative version of the following text fragment? > > > > <Curtis> > > > Identifying the position of any lost packets is important > > > for PW services which are attempting to reconstruct a bit stream > > > which maintains bit timing, such as time division multiplexing (TDM) > > > services. TDM and other PW services which require strict ordering > > > also require that misordered packets be either dropped or reordered. > > Invalid XML - you didn't close this element. :-) > > > <Sasha> > > > > Identifying lost PW packets and exact amount of lost payload is > > critical for PW services which maintain bit timing, such as Time > > Division Multiplexing (TDM) services since these services MUST > > compensate lost payload on a bit-for-bit basis. > > > > With these services PW packets that have been received out of order > > also MUST also be identified and may be either re-ordered or > > dropped. Reordering requires, in addition to sequence numbering, a > > "de-jitter buffer" in the egress PE, and ability to reorder is > > limited by the depth of this buffer. The down side of maintaining a > > de-jitter buffer is added end-to-end service delay. > > > > </Sasha> > > Accepted with a small change. > > After the words "down side" > s/de-jitter buffer/large jitter buffer/ > Elsewhere > s/de-jitter buffer/jitter buffer/g > Then add the sentence. > > A small jitter buffer is always necessary and the jitter buffer > is needed regardless of whether reordering is done. > > This yields a slightly changed second paragraph. > > With these services PW packets that have been received out of order > also MUST also be identified and may be either re-ordered or > dropped. Reordering requires, in addition to sequence numbering, a > "jitter buffer" in the egress PE, and ability to reorder is limited > by the depth of this buffer. The down side of maintaining a large > jitter buffer is added end-to-end service delay. A small jitter > buffer is always necessary and the jitter buffer is needed > regardless of whether reordering is done. > > Is this wording OK with you? > > Curtis > > > Hopefully this will be useful. > > > > > > Regards, > > Sasha > > > > > -----Original Message----- > > > From: Curtis Villamizar [mailto:curtis@ipv6.occnc.com] > > > Sent: Monday, November 25, 2013 8:30 PM > > > To: Alexander Vainshtein > > > Cc: curtis@occnc.com; kireeti@juniper.net; samante@apple.com; > > > agmalis@gmail.com; cpignata@cisco.com; mpls@ietf.org; Yaakov Stein > > > (yaakov_s@rad.com); pwe3 (pwe3@ietf.org) > > > Subject: Re: [PWE3] Pseudowire Sequence Number in draft-ietf-mpls- > > > forwarding: a minor comment > > > > > > > > > In message > > > <F9336571731ADE42A5397FC831CEAA025392F92F@ILPTWPVEXMB01.ecitele.c > > > om> > > > Alexander Vainshtein writes: > > > > > > > > Hi all, > > > > > > > > I would like to comment on the text in Section 2.1.8.1 "Pseudowire > > > > Sequence Number" in draft-ietf-mpls-forwarding-03 - MPLS Forwarding > > > > Compliance and Performance Requirements > > > > <http://tools.ietf.org/html/draft-ietf-mpls-forwarding-03>. > > > > > > > > This section states, in short, that the main drive for using the > > > > sequence number in the PW Control Word is handling of packet > > > > reordering events. TDM PWs are presented as a major example, with CBR > > > > ATM services as another example. > > > > > > > > In fact, this statement is not accurate. > > > > > > Yes. You are correct in pointing this out as inaccurate by omitting > > > the other important function of identifying drops for the purpose of > > > recunstructing TDM bit streams. > > > > > > Andy brought up resequencing being a strong provider request. > > > Reordering is far more common than loss particularly for high priority > > > services on provider networks and without resequencing reorder results > > > in PW loss in what could otherwise be lossless service. > > > > > > So we should get the base requirements right but still reflect this, > > > but strictly as advice with no normative wording. We had used the > > > phrase "is beneficial" and will retain that. > > > > > > Andy brought up the topic. The incorrect wording in the existing > > > draft is my fault. This text went in fairly early with a lot of other > > > changes and it appears that no one had since given it a careful enough > > > read and review until you came along. > > > > > > > The main drive for mandating the use of sequence number in TM PWs is > > > > the need to detect and count lost packets because the egress PE MUST > > > > compensate the lost payload bit for bit. > > > > > > > > Ability to compensate reordering of PW packets at egress is a side > > > > effect of (a) sequence number usage and (b) usage of the de-jitter > > > > buffer in the egress PW. It is not mandatory and in any case is > > > > limited by the depth of the de-jitter buffer: re-ordered packets that > > > > cannot be accommodated within this buffer are treated as lost. > > > > > > > > Additional details can be found, e.g., in section 6.2.2 of RFC > > > > 4553<http://tools.ietf.org/html/rfc4553>. Ability to re-order > > > > mis-ordered PW packets is defined there as OPTIONAL, while replacement > > > > of the payload of lost PW packets is defined as MANDATORY. > > > > > > The change below at the top of the section more accurately reflects > > > requirements in the RFCs but still states that resequencing can be > > > beneficial. The comments about EPD and PPD are dropped and therefore > > > also the informative reference. > > > > > > Context: > > > > > > 2.1.8.1. Pseudowire Sequence Number > > > > > > OLD > > > > > > Pseudowire (PW) sequence number support is most important for PW > > > payload types with a high expectation of in-order delivery. > > > Resequencing support, rather than dropping at egress on out of order > > > arrival, is most important for PW payload types with a high > > > expectation of lossless delivery. For example, TDM payloads require > > > sequence number support and require resequencing support. The same > > > is true of ATM CBR service. ATM VBR or ABR may have somewhat relaxed > > > requirements, but generally require ATM Early Packet Discard (EPD) or > > > ATM Partial Packet Discard (PPD) [ATM-EPD-and-PPD]. Though sequence > > > number support and resequencing support are beneficial to PW packet > > > oriented payloads such as FR and Ethernet, they are highly desirable > > > but not as strongly required. > > > > > > NEW > > > > > > Pseudowire (PW) sequence number support is most important for PW > > > payload types with a high expectation of lossless and/or in-order > > > delivery. Identifying the position of any lost packets is important > > > for PW services which are attempting to reconstruct a bit stream > > > which maintains bit timing, such as time division multiplexing (TDM) > > > services. TDM and other PW services which require strict ordering > > > also require that misordered packets be either dropped or reordered. > > > > > > PW services which are not timing critical bit streams in nature are > > > cell oriented or frame oriented. Though resequencing support is > > > beneficial to PW cell and frame oriented payloads such as ATM, FR and > > > Ethernet, they are highly desirable but not required. > > > > > > NEW (end of subsection after list of possible reording causes) > > > > > > In provider networks which use multipath techniques and which may > > > occassionally rebalance traffic or which may change PW paths > > > occasionally for other reasons, reordering may be far more common > > > than loss. Where reordering is more common than loss, resequencing > > > packets is beneficial, rather than dropping packets at egress when > > > out of order arrival occus. Resequencing is most important for PW > > > payload types with a high expectation of lossless delivery since in > > > such cases out of order delivery within the network results in PW > > > loss. > > > > > > The final paragraph sums up the motivation for highlighting > > > resequencing as beneficial and desirable. It does so without any > > > normative wording. > > > > > > > Hopefully these notes will be useful. > > > > > > > > Regards, > > > > Sasha > > > > > > These notes are very useful. Thank you. > > > > > > Please let us know if you (and the WG) are OK with the rewording > > > proposed above. > > > > > > Curtis
- [mpls] Pseudowire Sequence Number in draft-ietf-m… Alexander Vainshtein
- Re: [mpls] Pseudowire Sequence Number in draft-ie… Curtis Villamizar
- Re: [mpls] [PWE3] Pseudowire Sequence Number in d… Alexander Vainshtein
- Re: [mpls] [PWE3] Pseudowire Sequence Number in d… Alexander Vainshtein
- Re: [mpls] [PWE3] Pseudowire Sequence Number in d… Alexander Vainshtein
- Re: [mpls] [PWE3] Pseudowire Sequence Number in d… Alexander Vainshtein
- Re: [mpls] [PWE3] Pseudowire Sequence Number in d… Alexander Vainshtein
- Re: [mpls] [PWE3] Pseudowire Sequence Number in d… Alexander Vainshtein
- Re: [mpls] [PWE3] Pseudowire Sequence Number in d… Alexander Vainshtein