Re: [PANRG] [irsg] IRSG review request draft-irtf-panrg-what-not-to-do-13

[Spencer Dawkins at IETF <[e-mail]>]

Mallory,

My apologies for taking a bit of time to respond to your thorough review.
Please see below, but the high order bit is that I'm suggesting that we
adopt most of your suggestions, except for one I think is too big to be
tractable.

I created a PR for this at
https://github.com/panrg/draft-dawkins-panrg-what-not-to-do/pull/4, and
I'll submit the result as -14, when the submission queue is reopened next
week. .

Please note that most of the section numbers changed as I implemented some
of the changes you suggested in presentation order.  The diff isn't THAT
bad, because I'm mostly moving text around.

I attached a diff, so that people can see the changes more easily.

I'll let the PANRG participants express opinions as we chat, and I'll do my
best to track additional suggested changes.

Best,

Spencer

On Tue, Oct 20, 2020 at 2:47 PM Mallory Knodel <[e-mail]> wrote:

> Dear Spencer, panrg and irsg:
>
> It was my pleasure to read this document as it was a thorough compendium
> of Path Aware Networking in the IETF and serves as a great primer!
>

Thank you, on behalf of PANRG. A lot of people worked on it.


> My main comments on the draft are really to increase accessibility and
> readability of what surely is thorough, complete and clear documentation
> of a high quality appropriate for IRTF documents. I also realise that
> because my suggested edits are quite structural, that it's very likely
> you've had these conversations in the RG already and so if this walks
> back a previous agreement, feel free to disregard.
>
> While I read, I did a lot of flipping back and forth, particularly
> between Sections 2 and 5. I usually do a lot of flipping anyway; this
> review was uniquely flip-heavy. You also have a lot of internal
> references. I'd suggest a restructure thusly:
>
>   * Section 1: Pairing down mostly meta information about the RG
> (documented elsewhere), doc (not all the background is pertienent
> context), etc so that the reader gets to the lessons quicker.
>

I've at least moved this out of the Introduction section!


>   * Section 4: Make this a methodology section right after the
> introduction, also to potentially guide future contributions. You can
> then explain how you first collected contributions and then extracted
> the important details, drew lessons learned and conclusions. You can
> explain here categories for Table 1.
>

I moved this section significantly forward in the document. It's now
called  Methodology for Contributions, and I added some words about how the
contributions were collected and analyzed.

I think Section 2 is explaining those categories - did you mean something
else?

>
>   * Compress Sections 2, 3 and 5. I can see value in just getting the
> lessons upfront, but if you perhaps nest the various contributions
> wisely you can create sub-sections and even sub-sub-sections here,
> allowing you to give the reader the highest points (Section 3), followed
> by generalisations (Section 2), followed by Sub-sections (Section 5).
> Your headings are well phrased, such that if one doesn't want to read
> the details of Section 5, they can stop reading and/or skip. This also
> allows your Section 5 to be sparse-- perhaps there's sufficient
> documentation elsewhere on the internet that you could simply reference
> and leave in just a summary/abstract of lessons learned without the
> burden of having to include the data of a full contribution.
>

I think your suggestion to move section 3 in front of section 2 makes
sense.

I also added a sentence to section 5, pointing out that "Our expectation is
that most readers will not need to read through this section carefully, but
we wanted to record these hard-fought lessons as a service to others who
may revisit this document, so they'll have the details close at hand."


> If you take my last suggestion, you might want to add to your template
> for contributions that you'd like a summary/abstract (a la Section 2)
> and a suggested category (a la Table 1).
>

I'd defer to the research group here, because I'm not sure how "living"
this document should be. The template as described is what we used, to get
to the Lessons Learned in the document, and the contributions are
universally understood to be either undeployed or only minimally deployed.
There are other path-aware networking techniques I'd LIKE to include, like
any of the multipath QUIC variants or Application Aware Networking (which
is actually a BOF at IETF 109, and I'm seeing references to the PANRG
document in e-mail on their mailing list), but I'm not sure we can make any
fact-based statements about their level of deployment yet.

But that is an interesting question for the research group ...

Lastly I'll point out that "What Not To Do" is a terrific framing, but
> then you need to ensure that your Section 2 headings/section titles have
> the same value frame (positive vs negative). Eg "DO Justify Deployment"
> or "DON'T React to Distant Signals". Whatever it is, just pick one and
> then adjust the rest. Or spell it out in each sub-section title.
>
> Nits:
>
>   * Someone like me needs just one sentence in the introduction about
> the problem that PAN solves. It's probably just so obvious to you all!
>

I moved the text from I-D.irtf-panrg-questions into a subsection that
follows the first paragraph of the Introduction.


>   * The first paragraph of 1.1. seems to be missing a specific reference
> to "path aware networking" otherwise it's quite generic a statement.
>

Nice catch :-)

Added this.


>   * RFC 5218 seems like an appropriate reference for 2.1, 2.2 and 2.3?
> Again, pretty generic and not PAN specific, seemingly.
>

Yeah, I wasn't using the background RFCs from the IAB as references in this
section, but that makes sense.


>   * 2.1 would suggest "_U.S._ American English" since most of America
> (the hemisphere) doesn't even speak English, and we also appreciate
> Canadian Anglophones who may or may not use this phrase.
>

That's fair :-)

Added this.


>   * 2.6 last sentence: You might have meant "These _changes_ may make
> deployment..."
>

I might have meant that, indeed.

Added this.


>   * 2.7 Needs perhaps another sentence that answers "Why?".
>

I think you're right. I added "Often, similar benefits can be achieved with
much less finely-grained state."


>   * 1.4, 2.12 there are sometimes notes about what might be done to
> mitigate the problem faced. These might be out of scope? But probably
> they are extra valuable and should be emphasized in a separate
> sub-section or summary, somewhere.
>

I'm thinking that 1.4, which is

1.4.  A Note About Path-Aware Techniques Included In This Document

   This document does not catalog every proposed path aware networking
   technique that was not adopted and deployed.  Instead, we limited our
   focus to technologies that passed through the IETF community, and
   still identified enough techniques to provide background for the
   lessons included in Section 2 to inform researchers and protocol
   engineers in their work.

   No shame is intended for the techniques included in this document.
   As shown in Section 2, the quality of specific techniques had little
   to do with whether they were deployed or not.  Based on the
   techniques cataloged in this document, it is likely that when these
   techniques were put forward, the proponents were trying to engineer
   something that could not be engineered without first carrying out
   research.  Actual shame would be failing to learn from experience,
   and failing to share that experience with other networking
   researchers and engineers.

is mostly saying "if it hurts when you do that, don't do that". What am I
missing?

But I agree with you about 2.12. We've been talking about doing a document
that's "if we're not gonna do THAT, what should we do?", and the comment
about TAPS should go in that follow-on document, so I removed it.

Thanks again for your help! You're now acked ...

Best,

Spencer


> -Mallory
>