Re: [AVTCORE] Comments on draft-ietf-avtcore-monarch-09

[Qin Wu <[e-mail]>]

Hi, Magnus:
Sorry for late.
Thank for your valuable review. please see my comments inline below.
Also the new version is now available at :
http://tools.ietf.org/html/draft-ietf-avtcore-monarch-10
The Diff is:
http://tools.ietf.org/rfcdiff?url2=draft-ietf-avtcore-monarch-10.txt

Regards!
-Qin
----- Original Message ----- 
From: "Magnus Westerlund" <[e-mail]>
To: "IETF AVTCore WG" <[e-mail]>
Sent: Wednesday, February 15, 2012 9:57 PM
Subject: [AVTCORE] Comments on draft-ietf-avtcore-monarch-09


Hi,

As a WG chair I got the request to start a WG last call on this
document. I have been a bit slow to get to this, but when I looked at
the document before starting the WG last call I think the document isn't
really ready for a WG last call. Thus instead I request that the WG
helps develop the document to be ready for WG last call.

The main issue I see with this document is focused on section 3 which is
intended to provide an overview of the monitoring architecture for RTP.
I see a number of issues with this section and especially figure 1.

1. Figure one fails to place third party monitor in relation to any of
the packet flows. I think they belong on the RTP/RTCP paths between the
sender, intermediate and the receiver.

[Qin]: Okay, I remember you comment this before, 

I will put some text to make this clear and also 

get the "1" arrow through the third party monitor

 to indicate that it is placed on the RTP/RTCP 

paths between the sender, intermediate and the receiver.


-----------------------------
2. I think the document needs to discuss a bit about how the information
flows from the different nodes to the management systems. There is an
arrow 5 from the RTP sender to the management system. First of all I
think this arrow should be separated from the other 5. In addition I
think there should exist arrows from the Third party monitor, the
monitor in the intermediate and receiver directly to the management
systems. And these arrows could be RTCP information being forward in
real-time. However, they could also be other solutions.

[Qin]: Good suggestion and will redraw figure to reflect this.

--------------------------------------------
3. In general I think 3.1 should discuss what exist already for
monitoring which there is no discussion. Like the RTP MIB. I can also
see other solutions like using IPfix to capture a packet flow and then
process using an RTP and RTCP receiver.

[Qin] I agree. In order to fix this, We propose 

the following text at the beginning of section 

3.1 to say:

"

There are many ways in which the performance 

of an RTP session can be monitored. These 

include RTP-based mechanisms such as the RTP 

SNMP MIB [RFC2959], or the SIP event package 

for RTCP summary reports [RFC6035], or non-RTP 

mechanisms such as generic MIBs, NetFlow, IPFix, 

and so on. Together, these provide useful 

mechanisms for exporting data on the performance 

of an RTP session to non-RTP network management 

systems. It is desirable to also perform in-session 

monitoring of RTP performance. RTCP provides the 

means to do this. In the following, we specify an 

architecture for using and extending RTCP for 

monitoring RTP sessions. One major benefit of 

such architecture is with ease of integration 

with other RTP/RTCP mechanism.

"
--------------------------------------
4. The document argues for RTCP XR based monitoring, but can you please
provide a clear indication of what the benefits are.

[Qin]: One major benefit I think is Ease of 

integration with other RTP/RTCP mechanisms. 

Another benefit I can see is RTCP-based monitoring

Takes both transport quality and application quality 

into account and can provide more accurate measurement 

for the Real time application quality.



Below is a number of issues I found
----------------------------------------------------------
5. Title: "Monitoring Architectures for RTP" is the plural in
architectures intentional? If, you better have the document live up to
that. The abstract doesn't appear to match that either.

[Qin]:To be consistent, I think we should use singular 
rather than plural. Thank for catching this.
-----------------------------------------------
6. Section 1.
  "As more users and subscribers rely on real time application services,
   uncertainties in the performance and availability of these services
   are driving the need to support new standard methods for gathering
   performance metrics from RTP applications."

What users and subscribers are you discussing above? The ones using RTP
to build services or the actual user of the service using RTP?

[Qin]: I think it is the latter. How about change it as:

"

As the delivery of multimedia services using the Real-Time

 Transport Protocol (RTP) over IP network is gaining an 

increasing popularity, uncertainties in the performance and

 availability of these services are driving the need to 

support new standard methods for gathering performance 

metrics from RTP applications.


----------------------------------------
7. Section 1, These rapidly emerging
   standards, such as RTCP XR [RFC3611]

Please expand abrivetations on first usage. And in this case I think the
correct title should be used prior to the reference ... such as "RTP
Control Protocol Extended Reports (RTCP XR)" [RFC3611] ...

[Qin]: Okay. I will also update other places in the 

document with the similar issues.


---------------------------------------
8. Section 1. " Quality of Experience (QoE)." Can you please include
some reference that explains the concept of QoE?

[Qin]: Good suggestion, I think we should reference 

RFC6390 or P.10/G.100 Amendment 2. I prefer to use

RFC6390.


-------------------------------------
9. Section 1: "Since different applications layered on RTP may have
   some monitoring requirements in common, therefore these metrics
   should be satisfied by a common design."

The above sentence needs to be reformulated.

[Qin]: How about rephrase like:

"Given that different applications layered on RTP may have

   some monitoring requirements in common, these metrics

   should be satisfied by a common design.



-----------------------------
10. Section 2: Section title doesn't match content.

[Qin]: Agree and will change title into "Terminology".
--------------------------------------------------
11. Section 2: One example of such metrics is the QoE Metric
      specified in QoE metric reporting Block [MQ].

I don't think that reference is a good example. The reference defines
how to transport and identify a large number of application level
metrics in RTCP XR, rather than being an example of a metric.

[Qin]: Do we have better reference than this one. 

I am aware that some ITU-T documents (e.g., ITU-T P.NAMS, 

P.NBAMS) define measurement method and model for 

QoE evaluation while other documents like RFC6390,

E.800, P.10/G.100 Amendment 2 only give definition 

of QoE. Also I think QoE metric reporting Block [MQ] 

not only define data format for QoE metric and but 

also provide Metric name and description for QoE metric.

If you really concerns this, I can remove this sentence.

---------------------------------
12. Section 2:
Composed metrics

      Metrics that are calculated based on direct metric or combination
      of direct metric and derived metrics.

What is an derived metric? Appears to be an reference without an
explanation.

[Qin]: Derived metric is actually one kind of composed metric. 

How about fix this like:

"

Composed metrics

     Metrics that are calculated based on Direct metric that have been 
     measured or combination of Direct metrics that are identical to the 
     metric being composed.
"
----------------------------------------------
13. Section 2:
   Interval metrics

      It is referred to as the metrics of which the reported values
      apply to the most recent measurement interval duration between
      successive metrics reports

Is "interval duration" actually good English? To me the duration of an
interval is the actual interval.

[Qin]:My understanding is Yes, Interval metrics is defined
 in relation to "cumulative metric" as a metric that apply
 to the duration of an interval. The interval should be the 
most recent interval to be measured rather than accumulative
 period within the whole session.
-------------------------------------------------
14. Section 2:
   Sampled metrics

      It is referred to as the metrics of which the reported values only
      apply to the value of a continuously measured or calculated that
      has been sampled at end of the interval.

Doesn't the above exclude metrics measured at give instance. Or is that
what "sampled at end of the interval" means? But then it isn't actually
an interval, is it?

[Qin]: Ideally, I think sample metric can be applied to any 
instance of one measurement interval. But I assume the more 
sampled data you gather during one measurement interval, the 
more accurate the sampled metric you can calculate. Does this 
make sense?
------------------------------------------------
15. Section 3.1:
According to the definition of
   monitor in the RTP Protocol [RFC3550], the end system that source RTP
   streams, an intermediate-system that forwards RTP packets to End-
   devices or a third party that does not participate in the RTP session
   (i.e., the third party monitor depicted in figure 1) can be
   envisioned to act as the Monitor within the RTP monitoring
   architecture.

Shouldn't the RTP receiver be included in the above?

[Qin]: You are right, I like to fix this by the following 

proposed change.

OLD Text:

"

According to the definition of
monitor in the RTP Protocol [RFC3550], the end system that source RTP
streams, an intermediate-system that forwards RTP packets to End-
devices or a third party that does not participate in the RTP session
(i.e., the third party monitor depicted in figure 1) can be
envisioned to act as the Monitor within the RTP monitoring
architecture.


"

New Text:

"

According to the definition of
monitor in the RTP Protocol [RFC3550], the end system that 

runs an application program that sends or receives RTP data 

packets, an intermediate-system that forwards RTP packets to 

End-devices or a third party that does not participate in the 

RTP session(i.e., the third party monitor depicted in figure 1) 

can be envisioned to act as the Monitor within the RTP monitoring
architecture.

"
--------------------------------------------
16. Section 3.1: When considering the monitoring architecture, I think
it is important to remember that we have topologies like the SSM, where
few sources are potentially distributed to many receivers and where
feedback is aggregated in feedback targets for potential processing and
filtering back upstream.

[Qin]: Okay, how about add one new paragraph at the 

end of Section 3.1 as follows:

"

RTP can be used to multicast groups, both ASM and SSM. These groups 

can be monitored using RTCP. In the ASM case, the monitor is a member 

of the multicast group and listens to RTCP XR reports from all members 

of the ASM group. In the SSM case, there is a unicast feedback target 

that receives RTCP feedback from receivers and distributes it to other

 members of the SSM group (see figure 1 of RFC5760). The monitor will 

need to be co-located with the feedback target to receive all feedback 

from the receivers (this may also be an intermediate system). In both 

ASM and SSM scenarios, receivers can send RTCP XR reports to enhance 

the reception quality reporting.

"
----------------------------------------------------
17. Section 3.2:
   o  RTCP NACK is used to provide feedback on the RTP sequence number
      of the lost packets [RFC4585].

In this case I would be a bit careful. Because the sequence numbers
reported on in NACK may in fact be a subset of all lost packets. Already
the media stream receiver creating NACK requests may in fact filter on
which sequence number is reported. There was a proposal for a
informative block (draft-ietf-avt-rtp-selret-05.txt) that allowed for
this type of information. There is other cases where a receiver will be
able to determine that it doesn't need to ask for some lost packets.

[Qin]: Okay, How about rephrase this sentence as:

"

   o  RTCP NACK is used to provide feedback on the RTP sequence number

      on a subset of the lost packets or the total lost packets [RFC4585].

"
---------------------------------------
18. Section 3.3:
For instance, application level metrics for QoE
   related performance parameters are usually measured at the user
   device in the home network.

I think you need to be careful with "usually" here. There is a lot of
hidden assumptions going into this sentence. I would recommend that one
attempts to be more objective and less assuming in the description. Also
there is a question what definition you have of a given location. Like
the home network which might look significant different depending on
where and whos network you look at.

[Qin]: Agree, how about change like:

"

The location of the RTP Sender/Receiver entities may impact a set of

 meaningful metrics.  For instance, application level metrics for QoE

 related performance parameters are under most conditions measured at 

the user device that receives RTP data packets.

"

-------------------------------------------------

19. Section 4:
o  Using large block.

I think "large" is a bad term, when you in reality are talking about
compound or aggregated metrics blocks. I would suggest picking some
other term and ensure that "large" isn't used in this meaning.

[Qin]:Agree, I think compound metric block is a better term that replaces 
large block in relation to "single metric block".
-----------------------------------------------------
20. Section 4:
"It may be also fixed across multiple RTP sessions from
      the same source. "

I think the above is missing "at the same time".

[Qin]: Agree and will add it into the update.
------------------------------------------------
21. Section 5.1:
To avoid this pitfall, this memo proposes the use of small RTCP XR
   metrics blocks each containing a very small number of individual
   metrics characterizing only one parameter of interest to an
   application running over RTP.

This one is similar with 19. Instead of small, maybe single metric or
something like this.



[Qin]:Agree, will use "single metrics blocks" to be consistent with the title of section 5.1.
------------------------------------------------
22. Section 5.2:
non- RTP-based / non-RTP-based



[Qin]: Fixed.
-----------------------------------------
23. Section 5.3:
This memo proposes to
   define a new XR Block that will be used to convey the common time
   period and the number of packets sent during this period [MI].

"this memo" is reference the monarch document no the MI, which I think
is the intention.



[Qin]: Agree, will remove [MI] from this sentence.
-------------------------------------------
24. Section 5.3:
 If
   the measurement interval for a metric is different from the RTCP
   reporting interval, then this measurement duration in the [MI] SHOULD
   be used to specify the interval.

New RTCP XR metric blocks that rely
   on the Measurement information block [MI] MUST specify the response
   in case the new RTCP XR metric block is received without an
   associated measurement information block;

Normative statements? in a informative draft. Is the intention to quote
the drafts text, then please make it a quote.



[Qin]: The above text is not quoted from other draft like [MI], 

but intend to provide guideline for [MI]. I like to fix this 

by changing the language into lowercase, since this memo is 

informative.
--------------------------------------------
25. Section 6.
This implies the creation of a subsidiary namespace to
   enumerate the PDV metrics which may be transported by this block, as
   discussed further in [PDV] .

I would like to note that this assume a certain simularity in metric
structure that allows the different metrics values to be carried in the
same basic structure.


[Qin]: You are right. I will put some text in the section 6 to highlight this.

------------------------------------------------------------------
26. I have also noted a number of cases where the "." is not in the
appropriate place, often additional space before the ".".



[Qin]: Fixed.

--------------------------------------------
Cheers

Magnus Westerlund

----------------------------------------------------------------------
Multimedia Technologies, Ericsson Research EAB/TVM
----------------------------------------------------------------------
Ericsson AB                | Phone  [phone]
Färögatan 6                | Mobile [phone]
SE-164 80 Stockholm, Sweden| mailto: [e-mail]
----------------------------------------------------------------------

_______________________________________________
Audio/Video Transport Core Maintenance
[e-mail]
https://www.ietf.org/mailman/listinfo/avt