Re: [ietf-sow] Fwd: Agenda Development RFP SOW for Community Input

[Russ Housley <[e-mail]>]

Robert:

> I'm glad the IOAC is moving work on building a better scheduling system
> forward. This is an improvement the community has been asking after for
> a long time, and I believe it will help us create more effective meetings
> with less effort. I have a long list of comments on the current sow document,
> and believe the document needs significant revision before being issued as 
> an RFP. Please accept these as suggestions for improvement, not as resistance
> to the work.
> 
> High level points:
> 
> .) There are imbedded questions that aren't something the responder should
> provide an answer to. (I'll comment on what I think the answers should be
> below).

We asked if the community would like to suggest answers.  Thanks for doing so.

> .) The document proposes changes to existing policy and practice.
> (Such as not allowing a group to request a 3rd session until _after_ the
> initial schedule has been prepared, and limiting 1st tier conflicts to 10 
> groups, and requiring justificaiton for a second slot request.). Why? 

This is actually current policy, I believe, even if some ADs are approving a third slot for very busy WGs.

> .) The document frequently makes UI decisions that it really should leave 
> to the proposer. The use of "dropdown" to mean the user should select from
> options is a particularly bad offender - the worst being when talking about
> selecting groups for the conflict lists.

I agree that we should allow some flexibility, but these came from the people that will be using the tool.

> .) The document should make it clear early that the tool is expected to
> automatically generate a proposed schedule for the meeting. Right now,
> that doesn't really appear until section 5 (page 17). though it leaks 
> forward a little into section 4.3.

Good suggestion.

> .) The document is inconsistent on whether the automatically generated
> schedule is created exactly once, or if it should be easy to say "no,
> we don't like that at all", go back and tweak the conflict lists or apply
> the "ignore" policies the document discusses and cause it to regenerate
> a new proposal from scratch.

This depends on the level of integration with the database, right?  However, I agree that it should be possible to rerun the schedule generation and overwrite a previous run.

> .) There is one major usecase for the tool that the document doesn't
> really seem to address. Currently there is a major multiparty session
> for working through conflicts, and we struggle to see the effects of
> the changes being made in realtime. There should be some requirements about
> allowing multiple remote users to review proposed changes in realtime.

Yes, it should be added.

> .) There is insufficient detail around authorization for the different
> access levels. The document needs to be much more specific about what
> "own session" or "own session request" means. The document also assumes
> a type of user that has no account - why? Is the tool going to reuse
> datatracker accounts? (It should). Those are not hard to obtain, and
> asking users to use them will improve transparency.

When I reviewed it, I assumed that to be the case.  It should be explicit.

> .) The document should call out early that the secretariat can take
> any action on behalf of an AD or WG chair, and an AD can take any
> action on behalf of a WG chair (this part would be new, but the tool
> should allow the policy).

Agree.

> .) It should be clear whether this tool or some other tool will be 
> used to edit and adjust the schedule as the meeting progresses. If that's
> part of what the RFP is asking the proposer to suggest, the question should
> be explicit.

Agree.

> .) There are other tools (such as the agenda views in the tracker) that
> depend on the schedule being populated in the database. The appendix
> suggests that an existing application might extract information into the 
> formats it needs. It should aso state that such an application needs to
> put data back _into_ the database once it's done.

Correct.  I wrote that part, and I missed that step.

> Details (in document order):
> 
> The note at the end of section 1.2 should be restated as the actual
> requirement earlier in the section. What you're really requiring is
> that the tool allow a list of room types (also called space) to be 
> managed, and a separate list of room configurations to be managed. 
> You have initial values for those lists that the tool should be configured
> with. It should also be clear whether new names you add to either list
> are only for the current meeting (that is the lists are meeting specific),
> or persist between meetings.

Agree.

> Why do you need both the ability to specify available time ranges in 1.2
> and blackout times in 1.3? If these are both being populate only with data
> from the contract (as the sections indicate), this should be collapsed into
> 1.2 and the focus should be on capturing the available times. If the intent
> of 1.3 is to mark space unavailable for some reason other than a meeting (and
> if that can happen after the meeting template is created), the requirement
> needs to be clarified.

I think either way works.  This seems to be the way the users think about it.

> Section 1.4 speaks of room and session names. It would help to clarify
> the terms (consistent use and definition).  Does session name mean
> something like "softwire" or something like "Afternoon Session I"?  
> Similarly, does room name mean something like "3F Banquet" or 
> something like "Breakout"?

Yes, this does need clarification.  I think it means "Afternoon Session I".

> There are multiple "types of space" lists that have different values
> in them (one in 1.2 and one in 1.4). There should be one list.

Agree.

> Section 2 appears to be defining access levels for editing
> session _requests_, but later parts of the document seem to be
> reusing the levels to describe authorization for affecting the actual
> schedule.

We should add one section on login and authorization.

> Are there any requirements for leaving a record of editing or deleting
> a session request? Will there be a history log, and will the tool
> expose it?

I do not know if this is needed.  Would the Secretariat find this useful?

> It might help to position section 2 after section 3 (so that "session
> request" is defined).

Good idea.

> The document asks {{{ Do sessions have different categories....
> The answer to that affects the content of a session request, and depends
> on whether we want to use session requests for leadership meetings or always
> have the secretariat schedule those directly. Other parts of the document
> assume the later, so I think the answer to the question (at least for this
> iteration) is no, but that might change if we want to use the tool to schedule
> the IAB/IESG breakout rooms or allow the nomcom to use it to schedule their
> room.

We do not want this tool to schedule time in breakout rooms.  Maybe in the future, but I do not want to bite off that much right now.

> Section 3.1 states that a Level 3 user can make an area meeting request.
> I don't think that's what's intended.

ADs or Secretariat should be able to do this action.

> As noted in the major observations above, the notion of the tool refusing
> to allow the request of a 3rd session until after the first draft of the
> agenda has been created seems detrimental. If such a session is known
> to be needed (approved by an AD), not accounting for that in the initial
> draft just adds pain when we have to throw large parts of the inital
> draft away. The requirement instead should be to make it possible for ADs
> to approve or reject exceptional requests between when they are submitted
> and when the initial draft is created.

I addressed this above.

> Again as noted in the major observations section, the tool should allow
> the secretariat to take any action on behalf of an AD. Given that, the
> requirement currently stated as "The Secretariat can also marked the
> session request as approved in the tool, since on occasion approval is
> given verbally or via email" is not needed, and could be dangerous. It 
> could result in some special knob the secretariat could turn. Instead, 
> the secretariat should just do whatever the AD would have done.

Agree.

> The section that talks about specifiying conflict lists requires
> dropdown UI motifs, but shows examples of lists of checkboxes. All of these
> should be carefully stated as either examples or suggestions. There are 
> many other possible UI designs, and the RFP should allow the proposer to
> propose one. It would be good to be explicit on whether the tool will allow
> a requester to put a given group in both the 1st and 2nd tier conflict lists.

I addressed this above.

> It seems odd that the document requires a requester to justify a 2nd slot
> request (it shouldn't) but not justify a "Please do not schedule on" request
> (which it should).

I do not understand this comment.  Robert, please try again.

> Under special requests, the document notes "There will be a statement
> here noting that cost may apply for some items." Please provide additional
> detail. Do you want this to appear when working group chairs are making
> a normal working group session request, or was the intent for this to only
> apply to breakout or side-meetings (such as corporate or affiliate meetings)?

I think this is about side-meetings, not breakout rooms.

> There is design in the document around allowing requests to be made without
> an account. As noted in the major comments section, this doesn't seem 
> necessary - datatracker accounts are not hard to create, using them would
> increase transparency and would avoid some of the edge conditions the document
> current concerns itself with. The notion of including links that would return
> to a request should be retained though - that will be useful even if you have
> logged in.

Agree.  Especially with the community enhancements that are underway.  Most active IETF participants will have datatracker accounts.

> The document asks {{ Is there any reason to involve this tool in side meeting
> room reservations for the IESG, IAB, or IAOC }}. The answer depends on whether
> this tool directly provides view of the agenda (that is, it maintains live data
> during the meetings), or whether it is put aside once the printed agenda is
> created. I think it would be more valuable to create a tool that would allow
> managing the meeting in real time (dealing with room substitutions on the fly).

I think the Secretariat should set the requirements, and then the IAOC should decide whether it can implement this aspect with the current budget.  It might have to wait if it is too expensive.

> Section 3.5 says the tool will act automatically. It should explain what
> automatic actions the tool will take.

Yes, they need to be listed.

> In section 5, why should the tool wait until the secretariat "clicks a button"
> to generate a proposed schedule. Why not have it build proposals on the fly as
> each new request is received?

This depends on the approach taken -- two extremes are in the Annex.

> Under the agenda manipulation sections - why don't we allow users (of whatever
> level) to save proposed agendas and refer to them by name (preferably with a URL 
> that goes back to them). This would allow several people to propose conflict
> resolutions. It would be even better if the tool provided a way to easily 
> visualize the difference between any two saved agendas. This saving motif
> could be leveraged to simplify some of the requirements around identifying
> what the current draft or "final" agenda are.

I'm not sure.  This could lead to a very significant amount of storage.  Also, it will not be useful after a few days.

> The document, in section 5.2, asks whether manipulation should be limited
> to ADs. I think the answer should be no - let anyone logged in play with the 
> agenda to their hearts content and save it. Just don't make it where the ADs or
> secretariat have to look at each saved thing (or fight through lists of saved
> things to see what's officially on the table). 

See above.

> There is detail missing from section 5.3 - are we requiring the tool to
> know when the draft schedule has been published? Will it start behaving
> differently? Will it behave differently after the "final" schedule has
> been published?

I think so.  I think changes will be batched for posting to the community.

> When discussing creating ics files, the document calls out local timezones 
> and conversions. Please point to the "floating" ical capability.

Good suggestion.

> The document currently mixes discussing reporting on conflicts and
> performing conflict resolution. These should be carefully kept separate.
> The requirement to explore "if X moves to Y, show conflicts" should be moved
> into the section discussing manipulating the schedule, and the requirement
> should be to _always_ show conflicts on any given proposed schedule.

I like this idea.

> The document asks {{{ What should the conflict report look like...}}}
> Adam Roach has a prototype that shows a graphical conflict report. Please
> consider including a screenshot from it as an example.

I like this idea too, but we need to allow flexibility as with other aspects of the UI.

> Section 7.9 is too vague to implement. Since this is an RFP, could this
> be cast as a "what other custom reporting capabilities can you support?"

I think we can add a list of examples, but I would like this to be as flexible as possible within reasonable cost.

> Section 8.2 calls out [WG Acronym] but not all session requests are for
> working group meetings.

Right.  This needs to be fixed.

> Section 8.3 mentions the "draft agenda phase". Where are the phases
> described?

Yes, the timeline should be added to the introduction.

> Seciton 9 concerns itself with colored markup of the final agenda in word
> and excel. It should instead focus on clear markup of the actual final agenda
> on the website.

We need to make sure the needs of the Secretariat are met.  Are these delivered to the venue?

> Please consider moving definitions early in the document.

Agree.

Russ