Re: REVISED Last Call: <draft-ietf-pwe3-vccv-impl-survey-results-02.txt> (The Pseudowire (PW) & Virtual Circuit Connectivity Verification (VCCV) Implementation Survey Results) to Informational RFC

["Andrew G. Malis" <[e-mail]>]

Abdussalam,

Thanks again, following IETF last call I'll discuss actions to take on the
draft with the IESG.

Cheers,
Andy


On Thu, Sep 5, 2013 at 6:00 PM, Abdussalam Baryun <
[e-mail]> wrote:

> Thanks Andrew, I am happy to see a survey draft, I never seen one
> before in IETF, however, if there was a survey done before in IETF, it
> will be interesting to mention that if you think necessary related.
>
> On 9/5/13, Andrew G. Malis <[e-mail]> wrote:
> > Abdussalam,
> >
> > Many thanks for your review and comments on the draft. I have some
> answers
> > inline.
> >
> > On Wed, Sep 4, 2013 at 10:24 PM, Abdussalam Baryun <
> > [e-mail]> wrote:
> >
> >> The Reviewer: Abdussalam Baryun
> >> Date: 05.09.2013
> >> I-D name: draft-ietf-pwe3-vccv-impl-survey-results
> >> Received your Request dated 04.09.2013
> >> ++++++++++++++++++++++++++
> >>
> >> The reviewer supports the draft subject to amendments. Overall the
> >> survey is not easy to be used as source of information related to such
> >> technology users, but easier as source of information related to
> >> respondings of companies.
> >>
> >> AB> I prefer the title to start as: A Survey of ..........
> >>
> >
> > Andy> The draft is reporting the results of the survey, rather than being
> > the survey, so the title couldn't start as you suggested. A possibility
> > could be "The Results of a Survey on Pseudowire (PW) & Virtual Circuit
> > Connectivity Verification (VCCV) Implementations", but I think the
> existing
> > title is more concise.
>
> Yes that was my aim, thanks,
> >
> > Abstract> This survey of the PW/VCCV user community was conducted to
> >> determine implementation trends. The survey and results is presented
> >> herein.
> >>
> >> AB> How did the survey determine implementations related to users (are
> >> they general known or uknown or chosen by authors...etc). What kind of
> >> results?
> >>
> >
> > Andy> The survey was of service providers deploying pseudowires and VCCV.
> > The "users", in this case, are service providers.
>
> ok, if described in the document, and how were they selected, is it on
> there work volume basis, or etc.
> >
> >
> >> AB> the abstract starts interesting but ends making the results not
> >> clear what it was (good, reasonable, expected, positive, had
> >> conclusions..etc)?
> >> AB> The draft states that it has no conclusion, because it is not
> >> intended for that but to help in knowing results to help in other
> >> future drafts. However, the abstract mentions that the survey
> >> conducted to determine (not understood how to determine without
> >> conclusions or analysis).
> >>
> >
> > Andy> It wasn't the job of the people conducting the survey to draw
> > conclusions from the results, it was for them to report the results so
> that
> > the working group could collectively draw conclusions in their ongoing
> > work. At the time, the WG needed information on which combinations of PW
> > and VCCV options were actually in use, and the survey was used to collect
> > that information.
>
> Ok, the WG needs information, but if I still remember, the document
> does not state/define such need to match the survey.
>
> >
> >
> >> Introduction>
> >> In order to assess the best approach to address the observed
> >> interoperability issues, the PWE3 working group decided to solicit
> >> feedback from the PW and VCCV user community regarding
> >> implementation.  This document presents the survey and the
> >> information returned by the user community who participated.
> >>
> >> AB> the introduction needs to show the importance of the survey, or
> >> what makes such decision from the WG (i.e. seems like the WG has not
> >> cover all types of community, not sure)?
> >> AB> Why did the WG decide the survey by using questionnair?
> >>
> >
> > Andy> The part of the Introduction on page 3 provides the background,
> > rationale, and importance of the survey. We used a questionnaire as that
> > form of survey is easiest for the respondents and allowed us to use
> > SurveyMonkey to conduct the survey.
>
> The questionnaire method has advantages and disadvantages, so if on
> section mentions the result validity in linked to method, I think the
> reader will know how much he can depend on such results.
> >
> >
> >> AB> suggest amending> the document presents the questionnair form
> >> questions and information returned ......
> >>
> >
> > Andy> We could change the sentence to say "This document presents the
> > survey questionnaire and the information returned by the user community
> who
> > participated."
> >
>
> my language may not be perfect, but I agree that amending it to show
> survey method and method of result collection.
> >
> >> Sections 1.1 1.2 and 1.3>
> >> ......questions based on direction of the WG chairs......
> >> There were seventeen responses to the survey that met the validity
> >> requirements in
> >> Section 3.  The responding companies are listed below in Section 2.1.
> >>
> >> AB> Why were thoes methodologies and why that way of quetions chosen
> >> for this survey? The answer to this is important for the document
> >> (informational) and future drafts.
> >>
> >
> > Andy> While the survey questions were originally suggested by the WG
> > chairs, they were written by the survey authors and reviewed by the WG
> > prior to the collection of results. We could add that if you like.
>
> I think that is an important information, because the WG is part of
> the community, not sure if you have service providers respondent which
> are joined in the WG, if so then that information is important also,
> even if there is no respondant in the WG participating mentioning that
> is important also.
> >
> >
> >> AB> The reason of the survey's methodology should be mentioned in
> >> clear section,  as the athors' opinion.
> >>
> >> Section 1.2> Form>
> >> Why the form did not make security consideration related to
> >> implementations in the form questions? which then may be used in
> >> security section.
> >>
> >
> > Andy> Because security information wasn't the subject of the survey.
>
> I thought the subject is implementation and interoperability of such
> technology, therefore, security is always important subject for users.
> However, if the survey does not want the security, mentioning that in
> one section will show that it was out of scope (if I remember the
> security related to implementations of users was not mentioned as out
> of scope).
> >
> >
> >> Results section 2>
> >> AB> are difficult to read or find related to section 1.2.
> >> AB> Usually the section mixes between what was returned and what was
> >> given. It is prefered to have two separate sections as 1 (what was
> >> given including the form), and what was returned as results.
> >>
> >
> > The questions are in section 1.2, and the results are in section 2.
>
> In section 2 you mention many things that you have given to
> respondants but that is not a result, what you give is a survey
> method, but the respondants actions is the result. So I suggest to
> take any information of your survey from the result section 2.
>
> Thanking you,
>
> AB
>