Re: [lmap] Support for multiple report table formats from one task

[Ron Stana <[e-mail]>]

Juergen,

The changes to version 4 of the LMAP YANG model (
https://tools.ietf.org/html/draft-ietf-lmap-yang-04) allow for multiple
tables to be sent for one task, so thank you.  However the client
application will need to be able to determine what data is in each of the
different tables.  There were two suggestions made in this thread:

1) use different registry references for each table via the metric object
2) use different values in the 'tag' object for each table

The problem I see is neither the 'metric' or 'tag' objects are within the
new 'table' object so each table cannot define its own value for these.

Regards,
Ron


On Mon, Feb 22, 2016 at 3:56 AM, Juergen Schoenwaelder <
[e-mail]> wrote:

> Ron,
>
> the YANG data model already has a tag to distinguish multiple
> 'tables'. But even with that tag in place, I think the reporting model
> is not working well yet. I agree that there is room for improvement.
>
> One of the odd things is also that results are produced by actions,
> not by claim to report by tasks. This alone causes trouble because
> actions may use options that impact the result format. Originally, we
> did not have the notion of actions and somehow the result model simply
> lags behind.
>
> I need to think more about how to do this properly. I do not have a
> solution proposal yet but I agree that the reporting model needs some
> more work (once we have the configuration/instruction model stable).
>
> /js
>
> On Tue, Feb 16, 2016 at 09:46:23AM -0500, Ron Stana wrote:
> > Tim,
> >
> > I am trying to determine how one task can produce results that require
> > multiple table formats (what I called report types) and send them to the
> > collector at the same time.  Most tasks we have will require this type of
> > functionality.
> >
> > This is my understanding of what we have discussed so far:
> > 1) The registry would be used to define the different table formats...for
> > the purpose of this discussion let's go with your suggestion that there
> > would be one entry for each table format.
> > 2) The multiple registry entries are then assigned to a single task using
> > multiple instances of the the lmap/tasks/task/metric object.
> > 3) When the task goes to send these different tables of results to the
> > Collector, it identifies the table that is being sent using a single
> > instance of the the report/task/metric object.
> >
> > If we agree thus far then the part that remains is how to send multiple
> > tables to the collector at one time.  The current report model only
> allows
> > each task to have one entry so the MA would have to POST a separate
> report
> > for each table.  I am asking if the tables could all be sent together.
> >
> > Ron
> >
> > On Tue, Feb 16, 2016 at 1:14 AM, Carey, Timothy (Nokia - US) <
> > [e-mail]> wrote:
> >
> > > Ron,
> > >
> > >
> > >
> > > I was thinking about this some more last night.
> > >
> > >
> > >
> > > Are you asking to categorize the reports by registry entry? - which I
> > > think you call report type?  If so I don’t see an issue with that – we
> can
> > > use the registry entry as part of the report results…
> > >
> > >
> > >
> > > BR,
> > >
> > > Tim
> > >
> > >
> > >
> > > *From:* Carey, Timothy (Nokia - US)
> > > *Sent:* Monday, February 15, 2016 10:31 PM
> > > *To:* 'Ron Stana'
> > > *Cc:* [e-mail]; [e-mail]
> > > *Subject:* RE: [lmap] Support for multiple report table formats from
> one
> > > task
> > >
> > >
> > >
> > > Ron,
> > >
> > >
> > >
> > > I would talk to the IPPM team then – its really how they define the
> > > metrics that make up the registry.
> > >
> > >
> > >
> > > As for a report per task constraint – I don’t know if saving inputs and
> > > outputs as you suggest warrant breaking that constraint – why not just
> have
> > > 2 tasks and duplicate the registry entry.
> > >
> > >
> > >
> > > I like the when the task is complete – generate the results for
> reporting
> > > and then report results based on the schedule to the collector. I know
> in
> > > our BBF implementation this works quite well.
> > >
> > >
> > >
> > > Otherwise you get into the nasty problem of a state machine that
> > > correlates running the tests with reporting results in the context of
> the
> > > overarching scheduled task (You call it running and final but there is
> a
> > > context to those states for that particular task).
> > >
> > >
> > >
> > > BR,
> > >
> > > Tim
> > >
> > >
> > >
> > >
> > >
> > > *From:* Ron Stana [mailto:[e-mail]
> > > <[e-mail]>]
> > > *Sent:* Monday, February 15, 2016 5:40 PM
> > > *To:* Carey, Timothy (Nokia - US)
> > > *Cc:* [e-mail]; [e-mail]
> > > *Subject:* Re: [lmap] Support for multiple report table formats from
> one
> > > task
> > >
> > >
> > >
> > > Tim,
> > >
> > >
> > >
> > > While multiple registry entries could be used to define each of the
> report
> > > types (table formats), this would require those registry entries to
> > > replicate the inputs ('fixed parameters' in registry terms) and any
> outputs
> > > that are common across the different types.  It would be better to
> have one
> > > registry entry that defined the inputs to the task once, all possible
> > > outputs once and then allowed the outputs to be grouped into different
> > > report types. The report type is just some string value. When the
> tables
> > > were sent from the MA to the Collector they would then just need to
> include
> > > the report type.
> > >
> > >
> > >
> > > Example:
> > >
> > > 1) Task measures bandwidth for one or more streams
> > >
> > > 2) One registry entry is defined for the task. It includes:
> > >
> > >     a) the numerous inputs that define the stream's packet format
> > >
> > >     b) an output definition for the 'last one second bandwidth'
> > >
> > >     c) an output definition for the 'average bandwidth since task
> start'
> > >
> > > 3) The 'last one second bandwidth' is only useful while the task is
> > > running so it is assigned to the report type 'running'.  The
> description of
> > > the output
> > >
> > > 4) The 'average bandwidth since task start' is useful while the task is
> > > running and as a final value when the task completes so it is assigned
> to
> > > the report types 'running' and 'final'.
> > >
> > > 5) When the reports are sent from the MA, they indicate the report
> type.
> > > This is where I suggested the 'tag' object in the report could be used.
> > >
> > >
> > >
> > > In the end I am asking about these three issues:
> > >
> > > 1) How does a task properly define multiple report types (tables)?
> > >
> > > 2) How does the MA identify the report type being sent?
> > >
> > > Even if #1 is addressed using multiple registry entries, and #2 is
> > > addressed by including a reference to the registry entry in the
> > > 'metric/uri' object of the task within the report, there is still this
> > > issue:
> > >
> > > 3) How does the MA send multiple report types at the same time? My
> > > understanding of the current report definition is that it limits each
> task
> > > to having one table definition per POST.  The MA could send each table
> > > definition to the collector separately but allowing each task to send
> an
> > > array of table definitions would be more efficient.
> > >
> > >
> > >
> > > You brought up a fourth issue which is how the data in each table is
> > > mapped to the registry.  I have been making the assumption this was
> going
> > > to be accomplished by setting the column header value in the report to
> one
> > > of the summary/name values defined in the registry for the metric.
> > >
> > >
> > >
> > > Ron
> > >
> > >
> > >
> > > On Sun, Feb 14, 2016 at 11:11 AM, Carey, Timothy (Nokia - US) <
> > > [e-mail]> wrote:
> > >
> > > Ron,
> > >
> > >
> > >
> > > The task has multiple registry entries – I would assume your example is
> > > that this task
> > >
> > > would have 3 registry entries.
> > >
> > >
> > >
> > > The columns are for the task and would have to have a union of all the
> > > metrics for registry entries. – Is this what you were asking?
> > >
> > >
> > >
> > > It seems that we lose the context of the registry entry to column
> though…
> > >
> > >
> > >
> > > BR,
> > >
> > > Tim
> > >
> > >
> > >
> > > *From:* Ron Stana [mailto:[e-mail]]
> > > *Sent:* Friday, February 12, 2016 4:31 PM
> > > *To:* [e-mail]
> > > *Cc:* [e-mail]
> > > *Subject:* [lmap] Support for multiple report table formats from one
> task
> > >
> > >
> > >
> > > LMAP Team,
> > >
> > >
> > >
> > > My understanding of the LMAP proposal is
> ietf-lmap-report:report/task/name
> > > is supposed to contain one of the values defined in
> > > ietf-lmap-control:lmap/tasks/task/name.
> > >
> > >
> > >
> > > If this is correct, and since each task instance within a report can
> only
> > > contain one table definition (one header + its row data) then how does
> a
> > > task send multiple tables of results that contain different table
> formats?
> > >
> > >
> > >
> > > Use Cases:
> > >
> > > 1) Y.1564 task has results for the Service Configuration and the
> Service
> > > Performance data, and each table has a different set of columns.
> > >
> > > 2) An RFC-2544 task would need a different table for the Throughput,
> > > Latency, Frame Loss and Back to Back result sets.
> > >
> > > 3) Any task may want to send a minimal result set while it is executing
> > > that is very different from the final result set when it has completed.
> > >
> > >
> > >
> > > The ietf-lmap-report:report/task/tag could be used to differentiate the
> > > result sets sent by the same task. However this is not very efficient
> since
> > > this approach requires the MA to send each report to the collector
> > > separately because each report can only contain one set of results per
> task
> > > name.
> > >
> > >
> > >
> > > Any guidance or clarification on the LMAP proposal would be
> appreciated,
> > >
> > >
> > >
> > > Ron
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
> > >
>
> > _______________________________________________
> > lmap mailing list
> > [e-mail]
> > https://www.ietf.org/mailman/listinfo/lmap
>
>
> --
> Juergen Schoenwaelder           Jacobs University Bremen gGmbH
> Phone: [phone]         Campus Ring 1 | 28759 Bremen | Germany
> Fax:   [phone]         <http://www.jacobs-university.de/>
>