FW: [bmwg] WG Last Call: draft-ietf-bmwg-dsmterm-05

[Tony DeLaRosa <[e-mail]>]

Jerry,

The last time I checked, the deadline for dsmterm's last call was March 25,
and not the date of the IETF meeting.  Also, "no response" at the IETF
meeting does not mean consensus was achieved.

Concerns that were either discounted or ignored
Date	   Author			Subject
Notes
03/03/06 Barbara Denny 		Forwarding Congestion
No response yet to her last set of questions and no resolution to her
issue(s) raised
03/03/06 Mark E. Robinson	Forwarding Congestion
No response yet
03/03/14 Mark E. Robinson 	Forwarding Congestion
First question answered but dismissed the remaining questions.  
03/03/21 Al	Morton 		Latency
Responded 3/27/2003
03/03/25 Sunil Kalidindi	Latency
No response yet
03/03/25 Sunil Kalidindi	Delay
No response yet

Section 3.1.3 Forwarding Congestion
<<Overloading versus Forwarding Congestion>>
My objection to this section was that the justification used to define
forwarding congestion was weak.  Your original premise for forwarding
congestion was that existing definitions were only applicable to ingress
interfaces which are not correct.  Your clarifying explanation in your
recent email is an improvement, however unfortunately, not documented in
this section.  As David Newman stated on a 3/14/2003 email, no methodology
should assume that "the test person ...understand[s] what he/she is doing."
Hence, the section should be rewritten to clear up any ambiguities.   

<<Packet Loss Indicates Congestion>>
Yes, I agree with you that packet loss due to physical layer errors is
outside the scope of BMWG and that packet loss due to switching fabric
cannot be externally observed.  However, the RFCs you mentioned measured
packet loss for a specific configuration or condition.  In contrast, the
discussion in your section is very broad.  You are stating that ANY packet
loss indicates congestion.  This is where you open yourself up to comments
about all the reasons packet loss may occur.  This is why I suggest you
place a disclaimer if you wish to make that assertion.  Otherwise, I would
suggest providing scenarios and examples in the discussion on when exactly
packet loss is an appropriate symptom to indicate congestion.  Also, I would
avoid generic statements such as:

"Packet Loss, not increased delay, is the metric to indicate condition of
forwarding congestion", or 
"...any device that cannot forward packets on one or more egress interfaces
is under forwarding congestion"  

My proposal for rewriting this section (in this order)
1. Beef up the paragraph on why the overload definition is inappropriate
when enabling traffic control mechanisms 
2. Add more description on the types of measurements used to indicate
Forwarding Congestion (delay and packet loss) 
3. Add a more detailed explanation on why packet loss is preferred over
delay (BTW, RA99 is the wrong reference to use) 
4. Expand on when packet loss is appropriate to indicate Forwarding
Congestion.  (i.e. If I send 5 packets and one or more gets dropped, does it
mean the egress interface is congested?) 

Section 3.2.1 Channel Capacity
Can you point me to the appropriate thread archive/documentation?  I am not
entirely convinced that John Dawson's comments have been interpreted
correctly.

Section 3.2.4 Delay
<<Delay Usage>>
rfc956
rfc955
rfc813
rfc795
rfc793
rfc3449
rfc3439
rfc3096
rfc3095
rfc3019
rfc3002
rfc2682
rfc2626
rfc2508
rfc2461
rfc2330
rfc2213
rfc2205
rfc1850
rfc1801
rfc1633
rfc1479
rfc1470
rfc1323
rfc1257
rfc1165
rfc1123
rfc1077
rfc1045
rfc1017
rfc1005
rfc3414
rfc3272
rfc3246
rfc3148
rfc3134
rfc3125
rfc2757
rfc2214
rfc2194
rfc2021
rfc1821

<<does the BMWG wish to restrict Delay to this definition?>>
Sorry Jerry, this question was not meant to you but to the reflector.  I
know your position.

<<Delay is delay>>
I did not say that the implementer defines the arbitrary goalposts.  I was
just offering the replacement text for "Delay is delay".  You may use my
recommendation and then follow it with a statement saying "For this
definition, the goalposts defined will be the last bit... and the first
bit..."

<<How Delay measures the IP datagram>>
Generally the term "IP Packet" refers only to the IP header +
datagram/payload portion exclusive of the da/sa/crc portion of the frame.
Is this what you mean?  Can you give me an example how this would be
measured?

<<Latency is measured only at the throughput level>> 
[BR99] is a procedure that describes measuring latency at throughput.  It
does not say that latency is invalid at any other offered rates.  [BR91]
does not define latency at a particular offered rate.  Let's take it to the
reflector and solicit a vote on how many persons associate latency with
throughput as a MUST criteria.

<<Bullet number 5>>
Thanks

<<Jitter>>
I agree that we postpone this discussion until the Delay discussion is
resolved.  

Regards,

Tony
-----Original Message-----
From: Perser, Jerry [mailto:[e-mail]] 
Sent: Wednesday, March 26, 2003 11:56 AM
To: 'Tony DeLaRosa'; 'Kevin Dubray'; 'Al Morton'
Cc: '[e-mail]'
Subject: RE: [bmwg] WG Last Call: draft-ietf-bmwg-dsmterm-05


Tony,

You were at the IETF meeting last week.  I got up and presented the changes
and action items to dsmterm-05.  I asked if there were and other comments or
issues.  Do you remember the response?  None.  The general consensus was all
concerns were being addressed.

Could you list all the "concerns voiced on the reflector" that "were either
discounted or ignored"?  To my knowledge, each and every one has been
addressed.  Please state the year, month, and author of the concern that you
feel were not addressed and I will review them with my co-authors.

For you new comments, I address them inline.  I will cut a lot of the text
out so we can get to the heart of the issues:

> <<<Section 3.1.3 Forwarding Congestion>>>
>  
> <<ISSUE>>
> This definition argues that overloading is too limited for this 
> application. Unfortunately, the cited references used to make this 
> assertion have been taken out of context.

Please read RFC2285 section 3.5.4 Overloading.  It limits the scope to
"maximum rate of transmission allowed by the medium".  What happens when the
traffic control mechanisms polices at a level below the medium?  I can have
Forwarding Congestion with OR without Overloading.
  
> <<ISSUE>>
> Draft asserts that any packet loss above throughput is due to 
> forwarding congestion (AKA overloading).  As Barbara Denny, John 
> Dawson and Mark Robinson pointed out, it is presumptuous to assume 
> that the only reason packets are being dropped above throughput is 
> because of forwarding congestion.  One can not assume that a DUT 
> forwarding switching fabric is 100 percent bug free.  Packet loss can 
> be caused by any errors from the
> physical layer to state machine implementation.

Packet loss due to physical layer has been outside the scope of the BMWG.
RFC1944, RFC2544, RFC2889, and mcastm-11 assume no physical layer errors.

Packet loss due to switching fabric bugs can not be external observed.  To
ask the DUT if it meant to forward a packet is considered white box testing.
dsmterm proposes no metrics based on white box testing.
  
> <<Recommendation>>
> Insert a caution or disclaimer stating that packet loss will be used 
> as an indication of forwarding congestion although it may not be the 
> sole reason. Additional clarification may be required on why packet 
> loss alone is enough for indicating forwarding congestion.  The 
> reference to [RA99] pertains to
> only packet loss on best-effort data carried by TCP.

I will propose the disclaimer in mcastm-11 to the other co-authors.  I am
having a hard time finding it in section 4.3.  Can you point it out to me?

> Section 3.2.1 Channel Capacity
> <<ISSUE>>
> The definition of "Channel Capacity" is the same as for throughput.

This is a John Dawson concept back in 2000.  Throughput measure the rate on
the ingress.  The DUT/SUT internal buffering will artificially raise the
throughput level.  John proposed measuring the rate at the egress
(Forwarding rate).  He also proposed the term "Channel Capacity".  Ingress
is not the same as egress.
  
> Section 3.2.4 Delay
> <<ISSUE>>
> Delay is a common term used pervasively throughout many RFCs to 
> indicate a time measurement between two specified endpoints.  I agree 
> with Debby Stopp that the draft should a add modifier in front of 
> Delay to make it specific for the particular measurement.  I can 
> imagine the confusion in methodology
> document when trying to distinguish the difference between 
> delay and Delay.

Can you state the RFCs?

> So does the BMWG wish to restrict Delay to this definition?    

Yes.  Look at the charter.  It states "... present the requirements for the
common, unambiguous reporting of benchmarking results."

> <<ISSUE>>
> Bullet number 1, unclear on the following statement "Delay is delay,
> regardless of the technology being measured"

I understood it when a co-author wrote it.  I see Debby's point that it may
not be clear.  I will ping the co-author to see if he can come up with
better wording.
  
> <<RECOMMENDATION>>
> Replace with "Delay measures the elapsed time between two arbitrary
> goalposts, regardless of the technology being measured."

Disagree.  Are you proposing that each implementation pick "two arbitrary
goalposts"?  What if two implementations pick different goalposts?

> <<ISSUE>>
> Bullet number 3, I am unclear on how Delay measures the IP datagram 
> only.

Does this sound better:

3. Delay is referenced to the end of IP packet only, unlike [Br91], which
also includes link layer overhead.

> <<ISSUE>>
> Bullet number 4.  Latency is measured only at the throughput level. 
> Yes, I would agree that [BR99] measures latency at throughput but 
> neither [BR91] nor [BR99] precludes measuring latency at any other 
> offered loads.

When you say "either [BR91] nor [BR99] precludes measuring latency at any
other offered loads", you meant that it is outside their scope.

Read [BR99] section 26.2, the procedure.  It is very specific about "at the
determined throughput rate".  Other rate can be measured outside of [BR99].
They are not precluded, they just to comply with RFC2544.

> <<ISSUE>>
> Bullet number 5.  Should not be considered a bullet since this 
> paragraph is more of a note.

I will ask the other co-author for a consensus on this.  It is not a
technical issue.  If the majority wants to change it to a paragraph, you
will see it on dsmterm-06.

> Section 3.2.5 Jitter
> <<ISSUE>>
> See discussion on using common terms as a definition in Section 3.2.4 
> Delay.

Only after Delay discussion is resolved.  No consensus yet.

Jerry.

___________________________________________________________ 
Jerry Perser                         Spirent Communications
Director of Test Methodology         SmartBits Division 
Phone: [phone]                  26750 Agoura Road 
Lab:   818.676.2337                  Calabasas, CA, 91302 
Cell:  818.292.6457                  Corp:  818.676.2300

  
_______________________________________________
bmwg mailing list
[e-mail]
https://www1.ietf.org/mailman/listinfo/bmwg