Re: [Roll] Way forward for draft-clausen-lln-rpl-experiences

Ulrich Herberg <ulrich@herberg.name> Wed, 16 May 2012 19:34 UTC

Return-Path: <ulrich@herberg.name>
X-Original-To: roll@ietfa.amsl.com
Delivered-To: roll@ietfa.amsl.com
Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id F276921F86CF for <roll@ietfa.amsl.com>; Wed, 16 May 2012 12:34:40 -0700 (PDT)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -3.965
X-Spam-Level:
X-Spam-Status: No, score=-3.965 tagged_above=-999 required=5 tests=[AWL=1.012, BAYES_00=-2.599, FM_FORGED_GMAIL=0.622, GB_I_INVITATION=-2, RCVD_IN_DNSWL_LOW=-1]
Received: from mail.ietf.org ([12.22.58.30]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1Baxh57J6L3p for <roll@ietfa.amsl.com>; Wed, 16 May 2012 12:34:39 -0700 (PDT)
Received: from mail-qc0-f172.google.com (mail-qc0-f172.google.com [209.85.216.172]) by ietfa.amsl.com (Postfix) with ESMTP id 6F8FD21F85ED for <roll@ietf.org>; Wed, 16 May 2012 12:34:39 -0700 (PDT)
Received: by qcsq13 with SMTP id q13so923476qcs.31 for <roll@ietf.org>; Wed, 16 May 2012 12:34:39 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=herberg.name; s=dkim; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding; bh=qGJbcwTK2mAZ1cQjOBC0OsQD0tE77hrwFIH2BpRllI4=; b=lyJDJzbXtvies2laAAGnUNbq/A4AJcdGmz6v3d+bIzQps3BZQfjQvnw4WadHrEZ4yG tL8ChxjxS2skgimmDD1nKSh04LJBN6DPAt++xzLn28tPAyIVOljnXmIX3AAs6CUqmpAT yCgZr0MKjwY0r5tKwdlXIqR0kHtpEOFhT3090=
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type:content-transfer-encoding:x-gm-message-state; bh=qGJbcwTK2mAZ1cQjOBC0OsQD0tE77hrwFIH2BpRllI4=; b=G4kTWuxIL1ht6IBTt2IW2Hm3gVZ9yztndp8I+3vaCUsNuhBDnRT6W4so9z+YzcE1OM VXpNyewpKzKHwZgMO5HgOtBjPGpGdnNPAm8mS7ByOw7NKokhzxyCYoUqFFk/sH/+DJZ6 oYsr+cToQk8FY2oiCJq0dxRsXQStSaB4GV0zOMamF+SzaW7A19FRAAS0dyvNPishrQ3v XZqZnT+ppBR0KtRgaQ8hglIz0dn8d/nzx91b0rNZ4ajZL3SlgxFyn401G2vc16ggifkL hLgQj9qntSEyoB7TX5MDjs3SulkhVR5R0MFuSrF66jc5xX2XQ11bwYwXHQZgWTuT93hq nKIw==
MIME-Version: 1.0
Received: by 10.224.189.19 with SMTP id dc19mr11923597qab.76.1337196878884; Wed, 16 May 2012 12:34:38 -0700 (PDT)
Received: by 10.229.229.75 with HTTP; Wed, 16 May 2012 12:34:37 -0700 (PDT)
In-Reply-To: <1011855243.422503.1337192744447.JavaMail.root@mail17.pantherlink.uwm.edu>
References: <CAK=bVC-BaPiUFa1H+8ROmP+6e+d-kkOLwafhKvQ3GmDKQOvxBg@mail.gmail.com> <1011855243.422503.1337192744447.JavaMail.root@mail17.pantherlink.uwm.edu>
Date: Wed, 16 May 2012 12:34:37 -0700
Message-ID: <CAK=bVC-P7+1SXESqMCTRe+CkKPCA9B_0+UcNZqZ=9gjOW1QvGQ@mail.gmail.com>
From: Ulrich Herberg <ulrich@herberg.name>
To: Mukul Goyal <mukul@uwm.edu>
Content-Type: text/plain; charset="windows-1252"
Content-Transfer-Encoding: quoted-printable
X-Gm-Message-State: ALoCoQmLCDe9s4qqAkNKJ3T4oRXiwCaYBk23cHun8Pe2yCL8JZihUg5YvNgbr8nzritDFpjzxRzr
Cc: roll WG <roll@ietf.org>, Michael Richardson <mcr@sandelman.ca>
Subject: Re: [Roll] Way forward for draft-clausen-lln-rpl-experiences
X-BeenThere: roll@ietf.org
X-Mailman-Version: 2.1.12
Precedence: list
List-Id: Routing Over Low power and Lossy networks <roll.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/roll>, <mailto:roll-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/roll>
List-Post: <mailto:roll@ietf.org>
List-Help: <mailto:roll-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/roll>, <mailto:roll-request@ietf.org?subject=subscribe>
X-List-Received-Date: Wed, 16 May 2012 19:34:41 -0000

Mukul,

On Wed, May 16, 2012 at 11:25 AM, Mukul Goyal <mukul@uwm.edu> wrote:
>
> >>Seriously, do the authors think that this document would pass the muster for publication in any decent academic journal?
>
> >I don't know since when that is any criteria for IETF work. Was it a criteria for the standardization of RPL itself?
>
> RPL is a protocol specification whereas your document is an analysis of its performance.


No, our draft is not a performance analysis, but outlines observations
of the RPL RFC.


> The performance analysis papers fall in academic domain


No. The IETF is about engineering, not academic research. Our draft
was submitted as Internet Draft to the IETF, and not as academic
publication.


> and must follow a certain rigor and have a certain level of details when referring to experimental work.


As Thomas noted, all the observations were directly based on the RPL
RFC; the experiments only triggered the deeper investigation of the
RPL RFC, and the observations described in our draft do not depend on
a specific implementation.


> Your document fails to do so.

This is an assertion, not a constructive argument.


> >>Right now, the draft reads more like propaganda than information: written to bad mouth a protocol on the basis of biased/frivolous arguments.
>
> >I am sorry that you see the draft that way. We have tried to be as objective as possible. Can you point out any concrete paragraph or section that you see as "propaganda"? And honestly, I would have hoped that we can have an objective, argumentative discussion on the mailing list, and not a dismissal of our arguments without any constructive discussion.
>
> The problem is that the document is so biased that it is hard to see it any other way.


This is an assertion, not a constructive argument.



> A section-by-section review of the document coming in a couple of days.

Thank you, we appreciate that!




> >>Why would the authors completely ignore P2P-RPL even though it resolves many issues they have pointed out.
>
> >1) Because P2P-RPL is not even an RFC yet, whereas RPL is, and is already deployed.
>
> So, you basically throw to dustbin 2+ years of hard work people have done on P2P-RPL?

That is not what I said.



> Even though it is designed to solve precisely the problems you highlight in your document?

Does that mean you agree that the problems mentioned in our draft are
accurate for RPL? In that case, I wonder why we need a companion
document to solve these problems in the first place, and why they have
not been solved in RPL. As written further down in this email, I would
like to point out again that P2P-RPL is Experimental, and therefore
"solving all the problems [we] highlight" in the Standards Tracks RPL
document is only possible to a very limited degree. The P2P-RPL also
does not officially "update" (in IETF terminology) RPL.


> Same goes for the compression problem. The working group spent so much time deliberating on the problem and how best to solve it.

I would very much like to discuss how to solve it. Our observations
are based on the RPL RFC (and other standards that are related); any
previous discussions on mailing lists only is irrelevant for these
observations. It would be very helpful if you pointed out concrete
suggestions to the appropriate sections in our draft.


> But your document conveniently chose to ignore all that discussion. At the very least, your document could have been complete by listing all the issues people pointed out.


Pointed out where? Our draft is based on the RPL standards, not
previous discussions on mailing lists.


> >2) There is no dependency from RPL to P2P-RPPL, i.e., it should work stand-alone.
>
> Then, no protocol should ever be extended. Each protocol must work well in all possible scenarios.

RPL is not "extended" by P2P-RPL. I don't see that the latter
officially updates or obsoletes the RFC. That would be impossible
anyway, since P2P-RPL is Experimental.


> >3) P2P-RPL is experimental, RPL is std. track.
>
> So, P2P-RPL would never become standards track?

I understood that the current understanding of the ROLL WG is that
P2P-RPL is Experimental. It is, of course, possible to submit a new
draft in some years with intended Std. Track to update/obsolete the
Experimental version. But such a draft is not even planned, as far as
I know. Looking at other WGs, that process usually takes between 5-10
years, for gaining enough experience before publishing a new document
as std track.


> >4) I thought that ROLL is chartered to provide routing solutions for extremely constrained devices (which was, I assume, one of the reasons to not consider MANET work). One of our arguments in the draft is that RPL is very complex (the specification alone plus necessary companion documents has several hundred pages). Adding P2P-RPL as necessity to mitigate some of the issues would increase the code size even more.
>
> Two points:
>
> 1. A long RFC does not necessarily mean a complex protocol.

Well, not necessarily; but it may be an indication.


> Do you need to implement every thing in RPL RFC for a particular deployment? It would have been much more useful if your document had identified the minimal set of RPL features that MUST be present in each deployment.

I disagree. Such a recommendation should be made by the applicability
statements. Our draft does not give recommendations for implementers
in specific deployments, but describes general observations of RPL.


> 2. If you had added 34 pages of P2P-RPL to 157 pages of RPL RFC, it would not worsen the RPL's "complexity" too much. Would it? But then you would have much less to complain about RPL in your document.

First, see my argument above (P2P-RPL does not "update" the RPL
specification, as Experimental draft).
Then, there is not just the RPL RFC, but also other required (read:
"normative") documents, which add to the already long RFC. I really
thought that we are talking about extremely constrained devices, but
it seems to me from your argument, that the devices are actually not
that constrained, so that it does not really matter much whether to
add another protocol.


> >>There are numerous similar sins of omission spread throughout the document.
>
> >Where?
>
> Some additional sins I pointed out above. Please wait for my detailed review for a more complete list.

Okay.


> >>As a result, most conclusions the document reaches are open to doubt if not outright incorrect.
>
> >Is that proof by authority or a real argument? Please be more constructive.
>
> You reached the conclusions you reached based on incomplete arguments or by generalizing things that you saw in specific implementations/deployments. That is why these conclusions are doubtful.

I disagree. These observations are based on the RPL RFC, and not on
specific implementations or deployments. We have delivered concrete
observations in the draft; just doubting them without counter-proof is
not constructive.



> >>Sure, RPL is not a perfect protocol - no protocol is. But this document is not an unbiased scientific analysis of the protocol.
>
> >How do you come to that conclusion? We provide concrete, objective arguments. I do not see that in your email.
>
> Please see the previous comment.
>
>
> >>As Pascal said, this document could have served a valuable constructive purpose. But, perhaps this was not the intention of the authors. This document should be recognized for what it is: a political document written to further a particular destructive agenda.
>
>
> >I really don't know what to say to that non-constructive argument.
>
> Rather than guiding a reader how to deploy RPL and avoid the potential pitfalls, your message is that RPL is so fundamentally flawed that it should be dumped altogether. This is any thing but constructive.

I think we made it clear what our goals are (they are also described
in the draft).

Regards
Ulrich



>
>
> ----- Original Message -----
> From: "C Chauvenet" < c.chauvenet@watteco.com >
> To: "JP Vasseur" < jpv@cisco.com >
> Cc: "roll WG" < roll@ietf.org >, "Michael Richardson" < mcr@sandelman.ca >
>
> Sent: Wednesday, May 16, 2012 9:13:56 AM
> Subject: Re: [Roll] Way forward for draft-clausen-lln-rpl-experiences
>
>
>
> Hi,
>
> I definitely agree that implementation feedback is always good to know, so your experiences are welcomed.
>
> I also think that problems investigations need a complete and exact view, so I would encourage you to put much more details about the scenario and the environment where you experimentations took place.
> For instance, I would enjoy a "RPL Implementation Description" section in you draft listing the hardware your used, your RPL parameters, the RPL drafts and mechanisms implemented, your OS etc...
> If I read a paper with orthogonal observations with the same level of details as in your draft, then how could I forge my opinion ?
>
> Looking at this draft, it seems that it gathers lots of previous discussions that occurred during the past year on various mailing lists, and IETF meetings.
>
> Does your experimentations takes care about these recommendations ?
> If not, does your draft mention the propositions that have been made to address the problems you point out in your draft ?
> I think it could be worth to leverage on these previous discussions.
>
> Your draft is a list of Description and Observations.
> Maybe you could add a "Resolution Proposal" section for each problem, gathering previous discussion and your own proposals ?
> Identifying what is wrong in your implementation is a good first step, but the hardest part is to propose some corrections.
>
> Best Regards,
>
> Cédric Chauvenet.
>
> Le 16 mai 2012 à 15:04, JP Vasseur a écrit :
>
> > Dear Thomas,
> >
> > On May 16, 2012, at 2:08 PM, Thomas Heide Clausen wrote:
> >
> >> Dear JP and Michael,
> >>
> >> Thank you for your mail.
> >>
> >> On May 16, 2012, at 09:18 , JP Vasseur wrote:
> >>
> >>> Dear Thomas,
> >>>
> >>> On May 11, 2012, at 8:25 AM, Thomas Heide Clausen wrote:
> >>>
> >>>> Dear JP, Michael, all
> >>>>
> >>>> Upon JPs invitation, draft-clausen-lln-rpl-experiences was presented and discussed at the Paris meeting.
> >>>>
> >>>> The authors consider the document complete and "done", and are looking to take it forward in the IETF
> >>>> process for publication as "Informational RFC" in the very near future.
> >>>>
> >>>> We would therefore like to ask the WG chairs, if the ROLL WG is willing to accept and progress this
> >>>> document towards publication?
> >>>
> >>> Thanks for your suggestion. So far we haven't see a lot of discussion/interest from the WG but your request is
> >>> perfectly fair.
> >>
> >> Thank you - I aim to be fair.
> >>
> >>> So far there are no details on the scenarios and testing environments that led to the issues that
> >>> you reported, thus I would suggest you to first include them so that people interested could be able to reproduce
> >>> it. Once the drat is updated, we'll be happy to pool the WG.
> >>>
> >>> Does that make sense ?
> >>
> >> Not really. Let me explain my disagreement.
> >>
> >> We tried RPL (and, I note, several different independent implementations of RPL) in a number of different scenarios and deployments, and observed the behaviors exhibited - noting that what we observed across the different implementations, scenarios and deployments was fairly universal.
> >>
> >> We then went back to the specification, to understand these behaviors in detail, and understand their universality as independent from a specific scenario or deployment or implementation - but rather, as artifacts of the RPL protocol design.
> >>
> >> We therefore believe that _any_ deployment, scenario or testing environment of RPL needs to pay attention to the issues presented, and find a way of addressing them. The way of addressing these issues in a given deployment or scenario would be appropriate for an "applicability statement" for that deployment or scenario.
> >
> > JP> Thanks for the clarification; that being said, for the WG to make sure that nothing is "scenario" dependent and the outcome could indeed apply to all scenarios,
> > it might be worth being more explicit. For example, you pointed out to the MTU issue, to which I mentioned that 15.4g would bring a solution, so you may want to
> > explain that you did not use 15.4g and there are a number of such examples ….
> >
> >>
> >> (For example, a deployment using only L2s which provides guaranteed bi-directional links  for L3 would address this by in the applicability statement stating "As all L2-links are guaranteed bi-directional, this addresses the issues raised in section 9 in draft-clausen-lln-rpl-experiences".)
> >>
> >> Thus, we believe that it would actually be misleading (not to mention, unnecessarily verbose) to put the "details on the scenarios and testing environments" into this I-D.
> >>
> >> Doing so would mislead the reader to believe that the issues presented only manifest themselves in those precise scenarios - which definitely isn't the case.
> >
> > JP> see the previous comment and tell us what you think; we could provide other examples.
> >
> > Note that we do not oppose to asking to the WG; we just request you first to add additional information to proceed forward.
> >
> > thanks.
> >
> > JP and Michael.
> >
> > JP.
> >
> >>
> >> Best,
> >>
> >> Thomas
> >>
> >>> Thanks.
> >>>
> >>> JP and Michael.
> >>>
> >>>>
> >>>> Best,
> >>>>
> >>>> Thomas, Ulrich, Yuichi, Jiazi and Axel
> >>>
> >>
> >
> > _______________________________________________
> > Roll mailing list
> > Roll@ietf.org
> > https://www.ietf.org/mailman/listinfo/roll
> >
>
>
> _______________________________________________
> Roll mailing list
> Roll@ietf.org
> https://www.ietf.org/mailman/listinfo/roll
> _______________________________________________
> Roll mailing list
> Roll@ietf.org
> https://www.ietf.org/mailman/listinfo/roll
>