RE: [Sipping] [PMOL] SIP End-to-End Performance Metrics
"Daryl Malas" <D.Malas@cablelabs.com> Thu, 24 January 2008 18:15 UTC
Return-path: <pmol-bounces@ietf.org>
Received: from [127.0.0.1] (helo=stiedprmman1.va.neustar.com) by megatron.ietf.org with esmtp (Exim 4.43) id 1JI6be-0000bK-52; Thu, 24 Jan 2008 13:15:18 -0500
Received: from pmol by megatron.ietf.org with local (Exim 4.43) id 1JI6bc-0000aG-UG for pmol-confirm+ok@megatron.ietf.org; Thu, 24 Jan 2008 13:15:16 -0500
Received: from [10.91.34.44] (helo=ietf-mx.ietf.org) by megatron.ietf.org with esmtp (Exim 4.43) id 1JI6bc-0000Zw-5w; Thu, 24 Jan 2008 13:15:16 -0500
Received: from ondar.cablelabs.com ([192.160.73.61]) by ietf-mx.ietf.org with esmtp (Exim 4.43) id 1JI6bb-0004u0-4u; Thu, 24 Jan 2008 13:15:16 -0500
Received: from kyzyl.cablelabs.com (kyzyl.cablelabs.com [10.253.0.7]) by ondar.cablelabs.com (8.13.8/8.13.8) with ESMTP id m0OIFB69027679; Thu, 24 Jan 2008 11:15:11 -0700
Received: from srvxchg3.cablelabs.com (10.5.0.25) by kyzyl.cablelabs.com (F-Secure/fsigk_smtp/511/kyzyl.cablelabs.com); Thu, 24 Jan 2008 11:15:10 -0700 (MST)
X-Virus-Status: clean(F-Secure/fsigk_smtp/511/kyzyl.cablelabs.com)
X-Mimeole: Produced By Microsoft Exchange V6.5
Content-class: urn:content-classes:message
MIME-Version: 1.0
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
Subject: RE: [Sipping] [PMOL] SIP End-to-End Performance Metrics
Date: Thu, 24 Jan 2008 11:15:11 -0700
Message-ID: <160DE07A1C4F8E4AA2715DEC577DA4917B0C39@srvxchg3.cablelabs.com>
In-Reply-To: <8E3A1C85FE049E4BB5B4C96B207020B101199F68@esealmw106.eemea.ericsson.se>
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
Thread-Topic: [Sipping] [PMOL] SIP End-to-End Performance Metrics
Thread-Index: AchZ7ZVdC+psk80ASTW3x0Ddfzi+cwCJukqwAAre+OAAjaUCkAAPmpxQ
References: <1197672127.7806.53.camel@montag.eng.level3.com><E6C2E8958BA59A4FB960963D475F7AC306E4ED4626@mail.acmepacket.com><19f9b0170801180816p4bcc4f30kd962ae29a9b4c845@mail.gmail.com> <8E3A1C85FE049E4BB5B4C96B207020B10111A2F1@esealmw106.eemea.ericsson.se> <160DE07A1C4F8E4AA2715DEC577DA4917B0C0E@srvxchg3.cablelabs.com> <8E3A1C85FE049E4BB5B4C96B207020B101199F68@esealmw106.eemea.ericsson.se>
From: Daryl Malas <D.Malas@cablelabs.com>
To: Marian Delkinov <marian.delkinov@ericsson.com>, Hadriel Kaplan <HKaplan@acmepacket.com>
X-Approved: ondar
X-Spam-Score: 0.0 (/)
X-Scan-Signature: f0b5a4216bfa030ed8a6f68d1833f8ae
Cc: sipping@ietf.org, pmol@ietf.org
X-BeenThere: pmol@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: Performance Metrics at Other Layers <pmol.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/pmol>, <mailto:pmol-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/pmol>
List-Post: <mailto:pmol@ietf.org>
List-Help: <mailto:pmol-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/pmol>, <mailto:pmol-request@ietf.org?subject=subscribe>
Errors-To: pmol-bounces@ietf.org
Mario, At this point, I agree with your assertion to keep SEER as is, unless there is a strong argument to change it. Thanks... Daryl -----Original Message----- From: Marian Delkinov [mailto:marian.delkinov@ericsson.com] Sent: Thursday, January 24, 2008 4:01 AM To: Daryl Malas; Hadriel Kaplan Cc: sipping@ietf.org; pmol@ietf.org Subject: RE: [Sipping] [PMOL] SIP End-to-End Performance Metrics Daryl, Regarding adding 401/407 to the numerator of SEER, my opinion is that it depends on the idea behind SEER definition. 1) If SEER should complement SER, but exclude the called party behaviour, then I support keeping SEER as it is. This case matches the NER definition in PSTN. 2) If SEER should represent the ability of the infrastructure to establish sessions, then I would support adding 401/407 codes to the numerator. (If you remember during our discussion over the last year, I proposed adding all 4xx codes except 408 in the numerator of SEER, but we then agreed that it would cover much larger scope than just excluding the customer behaviour. Perhaps we have to define another metric to cover those cases, if the WG supports that.) Best regards! Mario. -----Original Message----- From: Daryl Malas [mailto:D.Malas@cablelabs.com] Sent: Monday, 21 January, 2008 16:19 To: Marian Delkinov; Hadriel Kaplan Cc: sipping@ietf.org; pmol@ietf.org Subject: RE: [Sipping] [PMOL] SIP End-to-End Performance Metrics Mario, I agree. This is the traditional difference between SER and SEER, much like the difference between ASR and NER (ITU-T e.411). Now, one thing that might make sense is to add 401/407 to the numerator of the SEER metric algorithm. --Daryl -----Original Message----- From: Marian Delkinov [mailto:marian.delkinov@ericsson.com] Sent: Monday, January 21, 2008 3:45 AM To: Daryl Malas; Hadriel Kaplan Cc: sipping@ietf.org; pmol@ietf.org Subject: RE: [Sipping] [PMOL] SIP End-to-End Performance Metrics Daryl and Hadriel, I have some comments regarding SER and SEER: > SER - need to specify that a 401/407 would also be subtracted in that > denominator, or better yet the Invite would not be double-counted > period. Also, need to point out that "# of INVITE" is for > new-sessions only (ie, ones without to-tags) - not re-Invites. (I know > it's obvious, but ya never know) If we go back to the analogy with PSTN and the ASR defined by ITU, I would suggest leaving SER as it is. Thus the metric would really be useful for estimating the %revenue compared to the total volume of invites, regardless of the reason for failure (as it is in PSTN). For analysis of the different aspects of the infrastructure efficiency I suggest using complementary to SER metrics, as SEER (already defined), and why not defining some more. SEER: Section 3.7 defines SEER, however the formula still uses SER=... I suggest using "SEER" in the formula in order to avoid confusion with SER. Best regards! Mario. -----Original Message----- From: Daryl Malas [mailto:dmalas@gmail.com] Sent: Friday, 18 January, 2008 17:17 To: Hadriel Kaplan Cc: sipping@ietf.org; pmol@ietf.org Subject: Re: [Sipping] [PMOL] SIP End-to-End Performance Metrics Hadriel, Comments in-line... On 1/17/08, Hadriel Kaplan <HKaplan@acmepacket.com> wrote: > > Hey Daryl, > Comments: > > Failed RRD - I'm a little confused by the wording of how we decide what constitutes a failed registration. Clearly a 401/407 challenge round does not, but you state a second round would. Why? It's totally possible it's not a failure. > I'm also confused what the value/purpose of measuring this delay is. > The section says measuring failed RRD is useful for detecting problems with reaching the intended Registrar - that would be true if you counted transaction timeouts, not request/response delays for auth failures, no? [DM] I agree this could create some false indications of failures. I can update the metric to only determine a failed condition based on a timeout scenario. Also, I'm going to put together a suggested list of metrics from this draft. The topic has come up several times on how many metrics should be included. Since, no one has suggested a list, I will throw one out. > > Failed SRD - need to specify a 401/407 is not a failed SRD (you say any 4xx). > [DM] I agree and will update the draft. > Successful SDD - you say at the end "In these two examples, TB and TS are the same even if the UAC/UAS receives the indicated messages instead of sending them." I'm pretty sure the delay in receiving a Bye and sending the 200 ok for that is not really useful. :) Or at least a very different issue than that measured from sending a Bye and receiving a 200. > [DM] I agree. At one point, someone requested this to be exhaustive in all conditions. I can remove this to indicate relevance only in the situation of initiating the Bye from either the UAC or UAS. > Failed SDD - what do we count the time of a Bye to a 4xx (but not 401/407) or 5xx as? Since those are the failure conditions for the other cases, this seems a bit odd to no longer count it as one here. [DM] I'm not sure I understand this question. > > Failed SDT - this seems to be at odds with the definition of successful SDT. Successful SDT is stopped on sending the Bye, but this one is stopped on timing out that sent Bye. (in other words, every call will thus count as a successful SDT, and some will also count as a failed one) Also, what good is this failed SDT info? You already have Failed SDD. It seems to me an SDT can only be of type "successful", and the timer should be stopped at send/receipt of Bye. [DM] I agree. I will update this to only be between the 200 and Bye. It should be the Bye regardless of whether or not it times out based on the intiator. > > AHR - how does the UAC or UAS know this info? I.e., how does the UAS know what max-forwards the UAC started with, and how does the UAC know what max-forwards the UAS received? [DM] The thought is the clients (or some other in-line monitoring device) would have to capture this information at both ends and hold it in order to correlate and make the determination. This metric is a tricky one and has come up much scrutiny. It will work and could be very useful, but will require some capture outside of SIP in all likelihood. > > SER - need to specify that a 401/407 would also be subtracted in that > denominator, or better yet the Invite would not be double-counted > period. Also, need to point out that "# of INVITE" is for > new-sessions only (ie, ones without to-tags) - not re-Invites. (I know > it's obvious, but ya never know) [DM] Are you saying 401/407 should be included with this sentence "...to the total number of attempted INVITE requests less INVITE requests resulting in a 3XX response..."? > > -hadriel > > > -----Original Message----- > > From: Daryl Malas [mailto:daryl@level3.net] > > Sent: Friday, December 14, 2007 5:42 PM > > To: pmol@ietf.org > > Cc: sipping@ietf.org > > Subject: [Sipping] [PMOL] SIP End-to-End Performance Metrics > > > > At the conference, I introduced the SIP End-to-End Performance > > Metrics draft (draft-malas-performance-metrics-08) to the PMOL working group. > > Although this draft is referenced in the PMOL WG charter, I wanted > > to ask everyone to review the draft and provide feedback of whether > > or not you feel the current version is ready for the WG to accept as > > a working group item. A couple of questions came up, which I think > > should be answered regarding this consideration: > > > > Is the SIPPING WG content with the current set of metrics? > > > > Are there too many? > > > > Do these metrics capture the relevant concerns of performance > > regarding the SIP protocol? > > > > Are these metrics depicted accurately from a SIP protocol perspective? > > > > In addition to these questions, I have a couple of tasks from the > > PMOL group to include in the next revision. > > > > Here is a link to the draft: > > http://www.ietf.org/internet-drafts/draft-malas-performance-metrics- > > 08.txt > > > > Thanks... > > > > Daryl > > > > > > > > _______________________________________________ > > Sipping mailing list https://www1.ietf.org/mailman/listinfo/sipping > > This list is for NEW development of the application of SIP Use > > sip-implementors@cs.columbia.edu for questions on current sip Use > > sip@ietf.org for new developments of core SIP > > ______________________________________________________________________ > This email has been scanned by the MessageLabs Email Security System. > For more information please visit http://www.messagelabs.com/email > ______________________________________________________________________ > _______________________________________________ Sipping mailing list https://www1.ietf.org/mailman/listinfo/sipping This list is for NEW development of the application of SIP Use sip-implementors@cs.columbia.edu for questions on current sip Use sip@ietf.org for new developments of core SIP _______________________________________________ Sipping mailing list https://www1.ietf.org/mailman/listinfo/sipping This list is for NEW development of the application of SIP Use sip-implementors@cs.columbia.edu for questions on current sip Use sip@ietf.org for new developments of core SIP _______________________________________________ PMOL mailing list PMOL@ietf.org https://www1.ietf.org/mailman/listinfo/pmol
- [PMOL] SIP End-to-End Performance Metrics Daryl Malas
- Re: [Sipping] [PMOL] SIP End-to-End Performance M… Daryl Malas
- RE: [Sipping] [PMOL] SIP End-to-End Performance M… Marian Delkinov
- RE: [Sipping] [PMOL] SIP End-to-End Performance M… Daryl Malas
- RE: [Sipping] [PMOL] SIP End-to-End Performance M… Marian Delkinov
- RE: [Sipping] [PMOL] SIP End-to-End Performance M… Daryl Malas