Re: REVISED Last Call: <draft-ietf-pwe3-vccv-impl-survey-results-02.txt> (The Pseudowire (PW) & Virtual Circuit Connectivity Verification (VCCV) Implementation Survey Results) to Informational RFC

[Abdussalam Baryun <[e-mail]>]

Thanks Andrew, I am happy to see a survey draft, I never seen one
before in IETF, however, if there was a survey done before in IETF, it
will be interesting to mention that if you think necessary related.

On 9/5/13, Andrew G. Malis <[e-mail]> wrote:
> Abdussalam,
>
> Many thanks for your review and comments on the draft. I have some answers
> inline.
>
> On Wed, Sep 4, 2013 at 10:24 PM, Abdussalam Baryun <
> [e-mail]> wrote:
>
>> The Reviewer: Abdussalam Baryun
>> Date: 05.09.2013
>> I-D name: draft-ietf-pwe3-vccv-impl-survey-results
>> Received your Request dated 04.09.2013
>> ++++++++++++++++++++++++++
>>
>> The reviewer supports the draft subject to amendments. Overall the
>> survey is not easy to be used as source of information related to such
>> technology users, but easier as source of information related to
>> respondings of companies.
>>
>> AB> I prefer the title to start as: A Survey of ..........
>>
>
> Andy> The draft is reporting the results of the survey, rather than being
> the survey, so the title couldn't start as you suggested. A possibility
> could be "The Results of a Survey on Pseudowire (PW) & Virtual Circuit
> Connectivity Verification (VCCV) Implementations", but I think the existing
> title is more concise.

Yes that was my aim, thanks,
>
> Abstract> This survey of the PW/VCCV user community was conducted to
>> determine implementation trends. The survey and results is presented
>> herein.
>>
>> AB> How did the survey determine implementations related to users (are
>> they general known or uknown or chosen by authors...etc). What kind of
>> results?
>>
>
> Andy> The survey was of service providers deploying pseudowires and VCCV.
> The "users", in this case, are service providers.

ok, if described in the document, and how were they selected, is it on
there work volume basis, or etc.
>
>
>> AB> the abstract starts interesting but ends making the results not
>> clear what it was (good, reasonable, expected, positive, had
>> conclusions..etc)?
>> AB> The draft states that it has no conclusion, because it is not
>> intended for that but to help in knowing results to help in other
>> future drafts. However, the abstract mentions that the survey
>> conducted to determine (not understood how to determine without
>> conclusions or analysis).
>>
>
> Andy> It wasn't the job of the people conducting the survey to draw
> conclusions from the results, it was for them to report the results so that
> the working group could collectively draw conclusions in their ongoing
> work. At the time, the WG needed information on which combinations of PW
> and VCCV options were actually in use, and the survey was used to collect
> that information.

Ok, the WG needs information, but if I still remember, the document
does not state/define such need to match the survey.

>
>
>> Introduction>
>> In order to assess the best approach to address the observed
>> interoperability issues, the PWE3 working group decided to solicit
>> feedback from the PW and VCCV user community regarding
>> implementation.  This document presents the survey and the
>> information returned by the user community who participated.
>>
>> AB> the introduction needs to show the importance of the survey, or
>> what makes such decision from the WG (i.e. seems like the WG has not
>> cover all types of community, not sure)?
>> AB> Why did the WG decide the survey by using questionnair?
>>
>
> Andy> The part of the Introduction on page 3 provides the background,
> rationale, and importance of the survey. We used a questionnaire as that
> form of survey is easiest for the respondents and allowed us to use
> SurveyMonkey to conduct the survey.

The questionnaire method has advantages and disadvantages, so if on
section mentions the result validity in linked to method, I think the
reader will know how much he can depend on such results.
>
>
>> AB> suggest amending> the document presents the questionnair form
>> questions and information returned ......
>>
>
> Andy> We could change the sentence to say "This document presents the
> survey questionnaire and the information returned by the user community who
> participated."
>

my language may not be perfect, but I agree that amending it to show
survey method and method of result collection.
>
>> Sections 1.1 1.2 and 1.3>
>> ......questions based on direction of the WG chairs......
>> There were seventeen responses to the survey that met the validity
>> requirements in
>> Section 3.  The responding companies are listed below in Section 2.1.
>>
>> AB> Why were thoes methodologies and why that way of quetions chosen
>> for this survey? The answer to this is important for the document
>> (informational) and future drafts.
>>
>
> Andy> While the survey questions were originally suggested by the WG
> chairs, they were written by the survey authors and reviewed by the WG
> prior to the collection of results. We could add that if you like.

I think that is an important information, because the WG is part of
the community, not sure if you have service providers respondent which
are joined in the WG, if so then that information is important also,
even if there is no respondant in the WG participating mentioning that
is important also.
>
>
>> AB> The reason of the survey's methodology should be mentioned in
>> clear section,  as the athors' opinion.
>>
>> Section 1.2> Form>
>> Why the form did not make security consideration related to
>> implementations in the form questions? which then may be used in
>> security section.
>>
>
> Andy> Because security information wasn't the subject of the survey.

I thought the subject is implementation and interoperability of such
technology, therefore, security is always important subject for users.
However, if the survey does not want the security, mentioning that in
one section will show that it was out of scope (if I remember the
security related to implementations of users was not mentioned as out
of scope).
>
>
>> Results section 2>
>> AB> are difficult to read or find related to section 1.2.
>> AB> Usually the section mixes between what was returned and what was
>> given. It is prefered to have two separate sections as 1 (what was
>> given including the form), and what was returned as results.
>>
>
> The questions are in section 1.2, and the results are in section 2.

In section 2 you mention many things that you have given to
respondants but that is not a result, what you give is a survey
method, but the respondants actions is the result. So I suggest to
take any information of your survey from the result section 2.

Thanking you,

AB