[ietf-sow] Fwd: Agenda Development RFP SOW for Community Input

Robert Sparks <rjsparks@nostrum.com> Tue, 08 November 2011 21:09 UTC

Return-Path: <rjsparks@nostrum.com>
X-Original-To: ietf-sow@ietfa.amsl.com
Delivered-To: ietf-sow@ietfa.amsl.com
Received: from localhost (localhost []) by ietfa.amsl.com (Postfix) with ESMTP id 5818521F85A1 for <ietf-sow@ietfa.amsl.com>; Tue, 8 Nov 2011 13:09:19 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -102.599
X-Spam-Status: No, score=-102.599 tagged_above=-999 required=5 tests=[BAYES_00=-2.599, HTML_MESSAGE=0.001, SPF_PASS=-0.001, USER_IN_WHITELIST=-100]
Received: from mail.ietf.org ([]) by localhost (ietfa.amsl.com []) (amavisd-new, port 10024) with ESMTP id sfcpds27Z-D8 for <ietf-sow@ietfa.amsl.com>; Tue, 8 Nov 2011 13:09:17 -0800 (PST)
Received: from nostrum.com (nostrum-pt.tunnel.tserv2.fmt.ipv6.he.net [IPv6:2001:470:1f03:267::2]) by ietfa.amsl.com (Postfix) with ESMTP id F09BF21F8591 for <ietf-sow@ietf.org>; Tue, 8 Nov 2011 13:09:13 -0800 (PST)
Received: from [] (pool-173-71-45-197.dllstx.fios.verizon.net []) (authenticated bits=0) by nostrum.com (8.14.3/8.14.3) with ESMTP id pA8L9BcF032293 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=NO) for <ietf-sow@ietf.org>; Tue, 8 Nov 2011 15:09:11 -0600 (CST) (envelope-from rjsparks@nostrum.com)
From: Robert Sparks <rjsparks@nostrum.com>
Content-Type: multipart/alternative; boundary=Apple-Mail-11-1029451472
Date: Tue, 8 Nov 2011 15:09:11 -0600
References: <20111107194844.32BA821F8B08@ietfa.amsl.com>
To: ietf-sow@ietf.org
Message-Id: <D8BAEA17-F755-417F-906A-F1BE47F0C70A@nostrum.com>
Mime-Version: 1.0 (Apple Message framework v1084)
X-Mailer: Apple Mail (2.1084)
Received-SPF: pass (nostrum.com: is authenticated by a trusted mechanism)
Subject: [ietf-sow] Fwd: Agenda Development RFP SOW for Community Input
X-BeenThere: ietf-sow@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: SOW Feedback List <ietf-sow.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/ietf-sow>, <mailto:ietf-sow-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/ietf-sow>
List-Post: <mailto:ietf-sow@ietf.org>
List-Help: <mailto:ietf-sow-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/ietf-sow>, <mailto:ietf-sow-request@ietf.org?subject=subscribe>
X-List-Received-Date: Tue, 08 Nov 2011 21:09:19 -0000

I'm glad the IOAC is moving work on building a better scheduling system
forward. This is an improvement the community has been asking after for
a long time, and I believe it will help us create more effective meetings
with less effort. I have a long list of comments on the current sow document,
and believe the document needs significant revision before being issued as 
an RFP. Please accept these as suggestions for improvement, not as resistance
to the work.

High level points:

.) There are imbedded questions that aren't something the responder should
provide an answer to. (I'll comment on what I think the answers should be

.) The document proposes changes to existing policy and practice.
(Such as not allowing a group to request a 3rd session until _after_ the
initial schedule has been prepared, and limiting 1st tier conflicts to 10 
groups, and requiring justificaiton for a second slot request.). Why? 

.) The document frequently makes UI decisions that it really should leave 
to the proposer. The use of "dropdown" to mean the user should select from
options is a particularly bad offender - the worst being when talking about
selecting groups for the conflict lists.

.) The document should make it clear early that the tool is expected to
automatically generate a proposed schedule for the meeting. Right now,
that doesn't really appear until section 5 (page 17). though it leaks 
forward a little into section 4.3.

.) The document is inconsistent on whether the automatically generated
schedule is created exactly once, or if it should be easy to say "no,
we don't like that at all", go back and tweak the conflict lists or apply
the "ignore" policies the document discusses and cause it to regenerate
a new proposal from scratch.

.) There is one major usecase for the tool that the document doesn't
really seem to address. Currently there is a major multiparty session
for working through conflicts, and we struggle to see the effects of
the changes being made in realtime. There should be some requirements about
allowing multiple remote users to review proposed changes in realtime.

.) There is insufficient detail around authorization for the different
access levels. The document needs to be much more specific about what
"own session" or "own session request" means. The document also assumes
a type of user that has no account - why? Is the tool going to reuse
datatracker accounts? (It should). Those are not hard to obtain, and
asking users to use them will improve transparency.

.) The document should call out early that the secretariat can take
any action on behalf of an AD or WG chair, and an AD can take any
action on behalf of a WG chair (this part would be new, but the tool
should allow the policy). 

.) It should be clear whether this tool or some other tool will be 
used to edit and adjust the schedule as the meeting progresses. If that's
part of what the RFP is asking the proposer to suggest, the question should
be explicit.

.) There are other tools (such as the agenda views in the tracker) that
depend on the schedule being populated in the database. The appendix
suggests that an existing application might extract information into the 
formats it needs. It should aso state that such an application needs to
put data back _into_ the database once it's done.

Details (in document order):

The note at the end of section 1.2 should be restated as the actual
requirement earlier in the section. What you're really requiring is
that the tool allow a list of room types (also called space) to be 
managed, and a separate list of room configurations to be managed. 
You have initial values for those lists that the tool should be configured
with. It should also be clear whether new names you add to either list
are only for the current meeting (that is the lists are meeting specific),
or persist between meetings.

Why do you need both the ability to specify available time ranges in 1.2
and blackout times in 1.3? If these are both being populate only with data
from the contract (as the sections indicate), this should be collapsed into
1.2 and the focus should be on capturing the available times. If the intent
of 1.3 is to mark space unavailable for some reason other than a meeting (and
if that can happen after the meeting template is created), the requirement
needs to be clarified.

Section 1.4 speaks of room and session names. It would help to clarify
the terms (consistent use and definition).  Does session name mean
something like "softwire" or something like "Afternoon Session I"?  
Similarly, does room name mean something like "3F Banquet" or 
something like "Breakout"?

There are multiple "types of space" lists that have different values
in them (one in 1.2 and one in 1.4). There should be one list.

Section 2 appears to be defining access levels for editing
session _requests_, but later parts of the document seem to be
reusing the levels to describe authorization for affecting the actual

Are there any requirements for leaving a record of editing or deleting
a session request? Will there be a history log, and will the tool
expose it?

It might help to position section 2 after section 3 (so that "session
request" is defined).

The document asks {{{ Do sessions have different categories....
The answer to that affects the content of a session request, and depends
on whether we want to use session requests for leadership meetings or always
have the secretariat schedule those directly. Other parts of the document
assume the later, so I think the answer to the question (at least for this
iteration) is no, but that might change if we want to use the tool to schedule
the IAB/IESG breakout rooms or allow the nomcom to use it to schedule their

Section 3.1 states that a Level 3 user can make an area meeting request.
I don't think that's what's intended.

As noted in the major observations above, the notion of the tool refusing
to allow the request of a 3rd session until after the first draft of the
agenda has been created seems detrimental. If such a session is known
to be needed (approved by an AD), not accounting for that in the initial
draft just adds pain when we have to throw large parts of the inital
draft away. The requirement instead should be to make it possible for ADs
to approve or reject exceptional requests between when they are submitted
and when the initial draft is created.

Again as noted in the major observations section, the tool should allow
the secretariat to take any action on behalf of an AD. Given that, the
requirement currently stated as "The Secretariat can also marked the
session request as approved in the tool, since on occasion approval is
given verbally or via email" is not needed, and could be dangerous. It 
could result in some special knob the secretariat could turn. Instead, 
the secretariat should just do whatever the AD would have done.

The section that talks about specifiying conflict lists requires
dropdown UI motifs, but shows examples of lists of checkboxes. All of these
should be carefully stated as either examples or suggestions. There are 
many other possible UI designs, and the RFP should allow the proposer to
propose one. It would be good to be explicit on whether the tool will allow
a requester to put a given group in both the 1st and 2nd tier conflict lists.

It seems odd that the document requires a requester to justify a 2nd slot
request (it shouldn't) but not justify a "Please do not schedule on" request
(which it should).

Under special requests, the document notes "There will be a statement
here noting that cost may apply for some items." Please provide additional
detail. Do you want this to appear when working group chairs are making
a normal working group session request, or was the intent for this to only
apply to breakout or side-meetings (such as corporate or affiliate meetings)?

There is design in the document around allowing requests to be made without
an account. As noted in the major comments section, this doesn't seem 
necessary - datatracker accounts are not hard to create, using them would
increase transparency and would avoid some of the edge conditions the document
current concerns itself with. The notion of including links that would return
to a request should be retained though - that will be useful even if you have
logged in.

The document asks {{ Is there any reason to involve this tool in side meeting
room reservations for the IESG, IAB, or IAOC }}. The answer depends on whether
this tool directly provides view of the agenda (that is, it maintains live data
during the meetings), or whether it is put aside once the printed agenda is
created. I think it would be more valuable to create a tool that would allow
managing the meeting in real time (dealing with room substitutions on the fly).

Section 3.5 says the tool will act automatically. It should explain what
automatic actions the tool will take.

In section 5, why should the tool wait until the secretariat "clicks a button"
to generate a proposed schedule. Why not have it build proposals on the fly as
each new request is received?

Under the agenda manipulation sections - why don't we allow users (of whatever
level) to save proposed agendas and refer to them by name (preferably with a URL 
that goes back to them). This would allow several people to propose conflict
resolutions. It would be even better if the tool provided a way to easily 
visualize the difference between any two saved agendas. This saving motif
could be leveraged to simplify some of the requirements around identifying
what the current draft or "final" agenda are.

The document, in section 5.2, asks whether manipulation should be limited
to ADs. I think the answer should be no - let anyone logged in play with the 
agenda to their hearts content and save it. Just don't make it where the ADs or
secretariat have to look at each saved thing (or fight through lists of saved
things to see what's officially on the table). 

There is detail missing from section 5.3 - are we requiring the tool to
know when the draft schedule has been published? Will it start behaving
differently? Will it behave differently after the "final" schedule has
been published?

When discussing creating ics files, the document calls out local timezones 
and conversions. Please point to the "floating" ical capability.

The document currently mixes discussing reporting on conflicts and
performing conflict resolution. These should be carefully kept separate.
The requirement to explore "if X moves to Y, show conflicts" should be moved
into the section discussing manipulating the schedule, and the requirement
should be to _always_ show conflicts on any given proposed schedule.

The document asks {{{ What should the conflict report look like...}}}
Adam Roach has a prototype that shows a graphical conflict report. Please
consider including a screenshot from it as an example.

Section 7.9 is too vague to implement. Since this is an RFP, could this
be cast as a "what other custom reporting capabilities can you support?"

Section 8.2 calls out [WG Acronym] but not all session requets are for
working group meetings.

Section 8.3 mentions the "draft agenda phase". Where are the phases

Seciton 9 concerns itself with colored markup of the final agenda in word
and excel. It should instead focus on clear markup of the actual final agenda
on the website.

Please consider moving definitions early in the document.

Begin forwarded message:

> From: IETF Administrative Director <iad@ietf.org>
> Date: November 7, 2011 1:48:44 PM CST
> To: IETF Announcement list <ietf-announce@ietf.org>
> Cc: wgchairs@ietf.org, ietf@ietf.org, iaoc@ietf.org, rfc-interest@rfc-editor.org, iab@iab.org, iesg@ietf.org, tools-development@ietf.org
> Subject: Agenda Development RFP SOW for Community Input 
> The IETF Administrative Oversight Committee (IAOC) plans to issue a Request for Proposal (RFP) in November for a
> vendor to develop the IETF Meeting Agenda Scheduling Tool.  To that end the IAOC invites community comments on the 
> draft Statement of Work (SOW) that is located at <http://iaoc.ietf.org/rfpsrfis.html>.
> The community comment period ends 14 November 2011. The IAOC will consider all comments it receives during that period.
> The SOW describes the requirements for a Secretariat Meeting Agenda creation tool. The agenda tool will handle all meeting 
> scheduling, as well as management of any/all space in use during an IETF meetings. This includes working sessions, leadership 
> meetings, tutorials, BoFs, office hours, registration, breaks, and more.  Agenda scheduling is currently a manual process.  
> The Draft SOW includes some questions for which feedback and suggestions are desired.
> Community comment is due no later than 14 November to ietf-sow@ietf.org. One may subscribe to that list here: 
> https://www.ietf.org/mailman/listinfo/ietf-sow. 
> All information submitted in response to this announcement is voluntary and may be used in the development of the SOW 
> and a subsequent RFP.
> Thanks for your continuing advice and support.
> Ray Pelletier 
> IETF Administrative Director 
> Internet Society
> _______________________________________________
> IETF-Announce mailing list
> IETF-Announce@ietf.org
> https://www.ietf.org/mailman/listinfo/ietf-announce