Re: [lmap] more comments on draft-akhter-lmap-framework

[<[e-mail]>]

thanks, a few more follow-ups!

In the terminology draft we tried to distinguish the "Measurement Agent" and the "Measurement Peer". The MA interacts with the Controller and Collector, whilst the MP only interacts with the MA.
below you talk about MA to MA comms and control channel. i'm not sure if this is simply a terminology difference (ie what I'd call MA-MP comms - OWAMP, IPPM stuff etc) or whether you also want comms between (what I term) two MAs?

you mention a test that 'saturates the path'. how to achieve this depends where the bottleneck is expected to be? if the access link, then the MA has got to be involved?
thanks for the picture in your sldes for the meeting - that will be very helpful

i think your Measurement Task ID is similar to the Cycle ID in draft-eardley-terminology?
Cycle-ID: A tag that is sent by the Controller in an Instruction and
   echoed by the MA in its Report; Measurement Results with the same
   Cycle-ID are expected to be comparable.

   The purpose of the Cycle-ID is to allow the data analysis tools to
   identify easily Measurement Results that are expected to be
   comparable, typically because the associated Measurement Tasks all
   operate the same Measurement Method with the same values for its
   parameters.  This set of Measurement Tasks could be termed the
   Measurement Cycle.



[I'd been thinking that a specific Cycle-ID would apply only to a particular MA, but I guess there's no reason why a Controller shouldn't send the same Cycle-ID to more than one MA, if it knows that the Measurement Result will be comparable]



thanks

phil

________________________________
From: Aamer Akhter (aakhter) [[e-mail]]
Sent: 26 July 2013 23:58
To: Eardley,PL,Philip,TUB8 R; [e-mail]
Cc: Paul Aitken (paitken)
Subject: RE: more comments on draft-akhter-lmap-framework

Hi Phil,

Thanks for the comments.

Inline with AA>

From: [e-mail] [mailto:[e-mail]] On Behalf Of [e-mail]
Sent: Friday, July 26, 2013 10:46 AM
To: [e-mail]
Subject: [lmap] more comments on draft-akhter-lmap-framework

My final batch of comments

First, I think some of the material in Section 3 could be helpful expansion (on what was rather briefly mentioned in draft-eardley-lmap-framework). Sections 4 & 5 have lots of interesting stuff – question is whether this is a general framework for large-scale measurements (where we need to talk in detail about the tests) or whether it’s a framework for LMAP WG (which only needs a brief mention of areas outside the scope of lmap wg, and so wouldn’t have much about the tests, as this is ippm).

AA> Thanks and do not disagree completely. However, when it comes to the methods used for controlling MA to MA we do need to be mindful of what the options are as well as the tradeoffs. I do like the earlier suggestion (I think from Dan) that suggests separating out the properties of the active test control protocol that we (in the LMAP case) would need to worry about. This might even suggest if there is even a direct MA to MA test control channel or if it all needs to go via the controller.


Secondly, there were several comments about multiple measurement agents (& peers?) acting together. (Section 3.2 & 6.2.1). you mention several different cases, I’m not sure I understood properly or got them all.

AA> The multiple measurement agent case is when there is no anchor measurement agent (or the anchor measurement agent does not have enough capacity to properly saturate the path—in the case of a path capacity measurement). The lack of an anchor measurement agent may come about when the regulator is trying to make an access link test (within the ISP) but as it’s a non-ISP entity it would not have test resources that are well bandwidth connected within the ISP. Also, the regulator does not want to stress the ISP upstream interface. If we were to pool the resources of multiple ISP connected MAs (sitting in other broadband sites) we would be able create the saturation w/o stressing the other broadband sites links. I put a diagram in the presentation that Paul will be presenting that might be more helpful.

(1) I’m sure there’s a case where one test triggers another (say, the dns result followed by request to website) – this is ok.
(2) I think you have a ‘latency under load’ test (S3.2), where the latency is measured for a pkt/file being sent between MA & Measurement Peer, but another Peer is generating other traffic so that a bottleneck queue is loaded. I haven’t thought about this much – do you think there are other tests involving a second Peer? or even 3rd, 4th…? Or is it just ‘latency under load’?  I’m wondering whether it would be best to think about the specifics of how to do the ‘latency under load’ test, rather than a general capability for coordinating multiple Peers in one test.

AA> it was meant to just be an aggregate path capacity measurement. Sorry it was not clear—I think this section needs more verbiage etc.

(3) you mention (S6.2.1) a case where multiple Measurement Agents submit results all with the same Measurement Task ID. I guess this is a test where lots of coordinated measurements are made. You also mention a shared key between multiple MAs, which I think must be related. I don’t understand this case, please explain more.
Am interested to understand this whole area better, regardless of where the WG decides to draw the line between in scope & out.

AA> The measurement task ID is simply the index that is used to collate the individual results submitted by the MAs post-collection. This would be when the MAs are all given the same instruction etc.. I did a search in the document for ‘shared key’ and this is only in S4.2.2. This shared key is completely different and is a security device used to secure test control protocol (eg. OWAMP) communications between the MAs.


Third, you have some stuff about task scheduling where it seems important to have highly synchronised time between the controller, MAs and collector (and measurement peer?). why would this be needed? You mention about NTP and the issue that NTP doesn’t give very accurate time sync to a device on the end of an access line (because you can’t triangulate to multiple time servers). I suppose the error might be a few millisecs - the sorts of cases I had in mind it would easily be accurate enough. Incidentally I don’t see why your solution (NTP clock sent from controller) helps, since you still have the lack of triangulation on the end of an access line.

AA> The context of the time synchronization was when (for example) a one-way-latency test is run between two MAs. In the case of a round-trip-time test such time synchronization would not be needed, and of course we simply can’t do RTT/2. The time concerns are really just bubbling up from the OWAMP spec, but I believe them to be valid.

AA> regarding the NTP source as the controller – this was simply an attempt to get the MAs on to the same time hierarchy rather than differing unknown ones. It doesn’t really have to be the controller itself, just known time source that everybody agrees on. Will clarify.

AA> that said ( I don’t believe this is in the draft-aakhter-lmap-framework) I do see some value in having the collector being loosely (vs tightly) synchronised with the MAs. This could be used as a way to detect a MA that is completely out of time sync when it submits its reports. Perhaps useful….

Thanks again,
aa


Thanks
Best wishes
phil