Re: [PMOL] [tsvwg] Fwd: Re: Fwd: I-D Action: draft-sun-tsvwg-flowbased-pm-00.txt

[Yufang <[e-mail]>]

Hi Morton and Andreas, 

Thanks for your comments. I have read the RFCs and give comments as follow.
We called the method we proposed FPM.

1. The differences between TWAMP and FPM:
(1) Technically, although FPM is an active measurement protocol, there exist much difference between TWAMP and FPM. First, TWAMP and FPM both support the on-the-spot measurement, it means that they can perform measurement online 
when traffic of applications is running. But TWAMP is probe based, in TWAMP, extra probe measurement packets are injected to simulate real traffic, to carry out measurements and sample the performance of the network. The frequency 
of injected packets has great impact on the accuracy of measurements. For example, if TWAMP wants to measure an application that transmits packets frequently, it should also injected probe measurement packets (TWAMP-test packets) frequently, then it may make negative impact on the real traffic of applications, otherwise it can't reflect the actual service performance if the TWAMP-test packets are sparsely inserted.
FPM is based on the running traffic of applications, it collects the statistics of real traffic flow. FPM needs additional OAM packets participate in measurement. The OAM packets are used to carry statistics of the path/flow/application, they can be small and the inserted frequency can be lower.

(2) In addition, in some cases, it needs to monitor the various time-varying performance indexes of the IP network, the performance measurement should be based on real traffic  and reflect the real performance of the network. 
IP Performance Monitoring based on flow/applications is needed in many cases. For example, in mobile operator's backhaul network, there must be performance monitoring mechanism to check the traffic status in the network, and the applications/traffic are divided into multiple bearers with proper mobile QoS parameters (e.g. QCI). If the mobile network would manage bearers as QoS and applications, then the performance of backhaul is more like to be based on applications or QoS. With the status information, some strategies can be implemented, for example, eNB can implement online congestion control and bandwidth adjustment strategy based on the performance monitoring result.
TWAMP may be able to do flow-based measurement by specifying DSCP for the TWAMP-test packets. But it can't well support the online measurement and length of test packets is changeless and not varying as the real traffic packets. The average performance indexes measured by this method may not be suitable in these cases.
 
(3) For further detailed analysis:
   1) Measurement Control 
TWAMP actually consists of two parts: TWAMP-Control and TWAMP-Test. TWAMP-Control is used to initiate, start, and stop test sessions, whereas TWAMP-Test is used to exchange test packets between two TWAMP entities.
FPM also consists of two parts: connection control and measurement process. The Connection control consists of two parts: Connection Activation and Connection Deactivation. The Connection Activation is to setup measurements, whereas the Connection Deactivation is to stop the measurements.
Actually, similar to the session initiation of TWAMP, during the Connection Activation process of FPM, it supports the negotiation of the measurement flow, such as the negotiation of the sender and receiver addresses and port numbers, protocol type, periods of the measurement, DSCP in some cases. These parameters are used to extract the packets of a specific flow while there are several types of flow between the sender and the receiver.   
However, the TWAMP-Control also supports the per-session encryption and authentication for the control and test traffic, while the connection control of FPM doesn't do any security mechanism in its current release (more information about security will be provided later.).
For FPM doesn't use a specific control plane, and there are great differences between FPM and TWAMP in security mechanism, FPM seems simpler.

  2) Packets used for measurement
There are eight types of packets in TWAMP besides TWAMP-Test packet. The server Greeting message, Set-Up- Response message and Server-Start message are used for connection setup. Request-TW-Session message and Accept-Sessions message are used for creating test sessions. Start-Sessions message and Start-Ack message are used for starting test sessions. Stop-Sessions message is used to stop the session. 
The negotiation of test Modes is carried out during the connection setup process, if the selected Mode is not the authenticated mode and/or encrypted mode, the encryption and authentication are also implemented. That's why TWAMP has so many types of control packets.
There are six types of packets in FPM, which include four types of control packets (ACT, ACT-ACK, DEA, DEA-ACK) and two types of measurement packets (FM and BR).
For each measurement, the TWAMP-control messages are usually only transmitted once. So packets size of these messages has little impact on the network. But TWAMP-test packets are constantly transmitted during a test session. So the high transmitted frequency and/or large packet size may influence the real traffic.
As mentioned in RFC5357, the padding size of the TWAMP-test packets is at least 27 octets in unauthenticated mode, and at least 56 octets in authenticated and encrypted modes. Then the size of the TWAMP-test packets is at least 56 octets in unauthenticated mode, and at least 156 octets in authenticated and encrypted modes. However, the size of the FPM FM packets is only 20 octets, and size of BR packets is 36 octets. Note that here the packet size doesn't include the size of the IP header.
In addition, the transmission of FM packets are periodical with a specific time interval, or a certain number of traffic packets should be sent between two contiguous FM packets. 
In theory, the transmission frequency of TWAMP-test packets should be similar as the normal service packets in order to reflect the network performance more accurate. So the transmission frequency of TWAMP-test packets is certainly larger than that of the FM packets.
As both the measurement packets size and the packets transmission frequency of TWAMP are larger than those of FPM, TWAMP may have more negative impact on real traffic than FPM.

  3) The granularity of test flow
In TWAMP, all packets of a specific control connection have the same DSCP field in the IP header. Then the only granularity of the test is (SIP, DIP, sPort, dPort, DSCP).
In FMP, flow can be defined by different combinations SIP, DIP, PT, DSCP, sPort and dPort. Three types of combinations are suggested: (SIP, DIP, PT), or (SIP, DIP, PT, DSCP) , or (SIP, DIP, PT, sPort, dPort). Network operators can fetch measurement with different granularities according to their specific requirements.

  4) Security mechanism  
As in TWAMP, the measurement packets are independent of the real traffic, so when authenticated mode and encrypted mode are selected, an independent security mechanism needs to be established for the test.
But in FPM, the measurement packets are tightly coupled with the real service packets, so the security mechanism can be shared. Then there is no necessary to establish a specific security mechanism for the measurement packets.


2. Statistics
The metrics in the RFCs are sampled based on a Poisson process. The Poisson process is used to schedule the measurements.
In FPM, the schedule of the FM packets is the equivalent of the schedule of the measurements. In current version, FM packets are scheduled periodically. For each pair of FM/BR packets, a statistical sample can be achieved. 
In fact, we have just give the measurement methodology here, and the description of statistics in FPM need to be further considered and to be explained in detail.  

(1) packet reordering
In RFC 4737, it listed several reasons for packet reordering and has defined a reordering metric. It also proposed the methods how to determining whether or not packet reordering has occurred and how to quantify the degree of reordering. 
The discussion of packet reordering in FPM (section 6.2) is not for reordering metric. The reordering discussed here is between the real service packets and the OAM measurement packets. If reordering occurs between the FM packet and the specific service packet (the FM packet may arrive earlier than the last service packet sent before it, or later than the first service packet sent after it.), it will result in statistical error of packet loss. In TWAMP, it doesn't have such a problem.
However, FPM can use the method proposed in RFC 4737 without adding any message to determine whether service packet reordering has occurred and to quantify the degree of reordering.

(2) packet delay
RFC2679 defines a metric for one-way delay of packets across Internet paths. The methodology proposed in this RFC is that: the Src forms a test packet, places a timestamp in the test packet, and sends it to the Dst. The Dst takes a timestamp as soon as possible upon the receipt of the packet. By subtracting the two timestamps, an estimate of one-way delay can be computed.  
It is assumed that the Src and Dst are synchronized, otherwise the error analysis of a given implementation of the method must take into account the closeness of synchronization
between Src and Dst. 
This methodology can also be applied to FPM. In FPM, it also achieves the two timestamps, one-way delay can be calculated by this two timestamps.
But as you mentioned the assumption that the forward delay is always equal to the reverse delay is not really proper.  

(3) PDV/IPDV/jitter
Both RFC3393 and RFC5481 suggest to avoid using the term "jitter" and stick to delay variation, for "jitter" has a much broader than packet transfer performance. 
I think we also need to revise the related parts of our draft.
For the methodology proposed in RFC3393, the IP packet delay variation (ipdv) is the difference between the one-way-delay of the selected packets. By subtracting the first value of One-Way-Delay from the second value the ipdv value of the pair of packets is obtained.
RFC5481 presents the formulations of IPDV and PDV:
IPDV(2) = D(2) - D(1) = (R2-T2) - (R1-T1) = (R2-R1) - (T2-T1),  PDV(i) = D(i)-D(min)
The single instance of an ipdv and pdv measurement is the same in FPM. So FPM is easy to follow the methodology defined in RFC3393 and RFC5481.
    
Above all, the definition of the statistics and calculation methods proposed in FPM need to be further considered and the description should also follow the regulations of the IPPM RFCs. However, the needed parameters have been already obtained by the FPM, so it can fulfill the requirements without adding any message or field. 


Thanks again for your comments. And I'm looking forward to more comments. We may give careful modification for next version.

Best Regards,
Fang Yu





> -----Original Message-----
> From: Al Morton [mailto:[e-mail]]    
> Sent: 2 Aug 2012 2:27
> To: [e-mail]
> Cc: [e-mail]
> Subject: [tsvwg] Fwd: Re: [PMOL] Fwd: I-D Action: draft-sun-tsvwg-flowbased-pm-00.txt


tsvwg,

When I saw this draft

>Zhu Lei - Flow-based Performance Measurement                      (10 min)
>draft-sun-tsvwg-flowbased-pm

on the I-D announcement list, I asked for and received a
review from the Performance Metrics Directorate. I've
inserted a few of my own comments below.  We try to do
early review when requested by WG chairs, and since this
draft is now on the tsvwg agenda we are sharing our comments
with the tsvwg list.

Overall, I would like to suggest that the authors examine
the IPPM measurement protocols more fully and implement them
to see if their need for flow-based measurement can be met
without developing new protocols.

Al
PMDir admin


>From: Andreas Johnsson A <[e-mail]>
>To: "[e-mail]" <[e-mail]>
>Date: Mon, 2 Jul 2012 11:31:22 +0200
>
>
>Hi,
>I have tried to put together a short review. Please see my comments 
>below (and please provide further comments).

[ACM] = Al's additional comments.


>Best regards,
>Andreas Johnsson
>
>-----
>
>Comments on draft-sun-tsvwg-flowbased-pm-00.txt.
>
>1) The draft describes a new active measurement protocol with the 
>aim of measuring delay, jitter and loss over an IP path. The 
>protocol is similar to TWAMP (RFC5357), however the measurement 
>setup is performed in a TCP-like manner instead of using a specific 
>control plane. Despite its similarities to TWAMP there are no 
>references nor does the draft contain a comparison.
>
>2) The draft includes a description on how to calculate statistics 
>based on time stamps obtained by the protocol. The metrics are 
>delay, jitter and loss. An advice is to use RFC 6390 section 5.4.5 
>guidelines for describing the metrics.
>
>3) The draft does not <make use of> prior art in IPPM (or elsewhere) 
>related to delay, jitter or loss. Suggested documents are
>
>IPPM Delay (http://tools.ietf.org/html/rfc2679)
>IPPM Loss (http://tools.ietf.org/html/rfc2680)
>IPPM Delay variation (http://www.ietf.org/rfc/rfc3393.txt)

[ACM] IPPM Packet Reordering RFC 4737


>ITU-T Y.1540 (http://www.itu.int/rec/T-REC-Y.1540-200711-S)
>
>4) I would like the authors to elaborate on their delay calculation 
>method. The draft assumes that the forward delay is always equal to 
>the reverse delay. How often is this the case?
>
>5) The jitter calculations are based upon the delay calculations. 
>Since the delay is calculated as the RTT (minus the time a packet 
>spends at the PM responder) divided by 2, the jitter will be the 
>same in both directions. Does this really reflect what is happening 
>on the path? Please elaborate on this.

[ACM] In general, the statistics suggested here are not widely used,
especially the jitter as delay variance. Suggest that the authors
look at http://tools.ietf.org/html/rfc5481 and decide what measurement
task and PDV method suits their needs.





>-----Original Message-----
>From: [e-mail] [mailto:[e-mail]] On Behalf 
>Of Al Morton
>Sent: den 29 juni 2012 13:58
>To: [e-mail]
>Subject: [PMOL] Fwd: I-D Action: draft-sun-tsvwg-flowbased-pm-00.txt
>
>After reading the abstract, ...
>
>Would anyone volunteer to review this in a timely way?
>
>Al
>PMDir admin
>
>
> >From: [e-mail]
> >To: [e-mail]
> >Subject: I-D Action: draft-sun-tsvwg-flowbased-pm-00.txt
> >X-Test-IDTracker: no
> >X-IETF-IDTracker: 4.21p1
> >Date: Fri, 29 Jun 2012 01:40:50 -0700
> >X-BeenThere: [e-mail]
> >X-Mailman-Version: 2.1.12
> >Reply-To: [e-mail]
> >
> >
> >A New Internet-Draft is available from the on-line Internet-Drafts
> >directories.
> >
> >
> >         Title           : Flow-based Performance Measurement
> >         Author(s)       : Lishun Sun
> >                           Wendong Wang
> >                           Fang Yu
> >         Filename        : draft-sun-tsvwg-flowbased-pm-00.txt
> >         Pages           : 15
> >         Date            : 2012-06-29
> >
> >Abstract:
> >    The performance measurements of service flow are becoming significant
> >    important for administrators monitoring the fitness of the network.
> >    This memo defines an end-to-end application-based performance
> >    measurement method, which is achieved by generating synthetic
> >    measurement packets, injecting them to the network and analyzing the
> >    statistics carried in the measurement packets.
> >
> >
> >The IETF datatracker status page for this draft is:
> >https://datatracker.ietf.org/doc/draft-sun-tsvwg-flowbased-pm
> >
> >There's also a htmlized version available at:
> >http://tools.ietf.org/html/draft-sun-tsvwg-flowbased-pm-00
> >
> >
> >Internet-Drafts are also available by anonymous FTP at:
> >ftp://ftp.ietf.org/internet-drafts/
> >
> >_______________________________________________
> >I-D-Announce mailing list
> >[e-mail]
> >https://www.ietf.org/mailman/listinfo/i-d-announce
> >Internet-Draft directories: http://www.ietf.org/shadow.html or
> >ftp://ftp.ietf.org/ietf/1shadow-sites.txt
>
>_______________________________________________
>PMOL mailing list
>[e-mail]
>https://www.ietf.org/mailman/listinfo/pmol
>_______________________________________________
>PMOL mailing list
>[e-mail]
>https://www.ietf.org/mailman/listinfo/pmol