Re: [OSPF] IETF 67 OSPF WG Meeting minutes - Correct file appended

[Richard Ogier <[e-mail]>]

Acee Lindem wrote:

> Hi Richard,
> Richard Ogier wrote:
>
>> Acee,
>>
>> I am not sure I understand what you mean.  The MDR and OR/SP
>> drafts have already been evaluated exensively via GTNetS
>> simulations.  INRIA's solution has not yet participated
>> in any such evaluation. So if we require all the drafts
>> to participate in the GTNetS evaluation (which was the
>> original plan two years ago), then we *are* holding all
>> drafts to the same experimental publication criteria.
>
> GTNetS Simulation results were presented in San Diego so I believe
> MPRs have been implemented. The code should be made available for
> public inspection and comparison with the other drafts.


We seem to be in agreement.  The question is, when will INRIA make
their (up-to-date) code available to allow a fair comparison.
The reason I am concerned is because I don't want to see a repeat
of the MANET debate between OLSR and TBRPF, in which there
was no cooperation to allow a fair comparison.  (I will avoid giving
the details here, since it might result in more arguing.)

One of the reasons I decided to work on the MDR design is because
there was a good plan for comparing the performance of the
different proposals (using GTNetS).

Richard

>
>
>>
>> Or, are you saying that we should give INRIA a free pass
>> to avoid participating in the GTNetS evaluation?
>> I really don't think this would be fair, and therefore
>> seriously doubt that the consensus would agree with this.
>
> I agree.
>
>>
>> I don't think the voting at the meeting clearly distinguished
>> between the two options of accepting 2 versus 3 drafts.
>> This distinction was not made explicit at the meeting.
>
> You are right that the question of 2 or 3 wasn't the primary
> focus of the dialog. While we've agreed to allow for more
> than one experimental draft, I don't think we should lower
> our standards. I don't think anyone who was at the meeting
> would disagree.
> Thanks,
> Acee
>
>
>>
>> Richard
>>
>>
>> Acee Lindem wrote:
>>
>>> Hi Richard,
>>> I think we agreed upon a process to move along and we should
>>> continue to hold all the drafts to the same experimental publication
>>> criteria. I guess the point was that we should not limit the number to
>>> 2 if we're going to publish more than 1. Without injecting too much 
>>> judgment
>>> on the MPR draft's maturity, did everyone at the meeting hear the
>>> same message?
>>>
>>> Thanks,
>>> Acee
>>>
>>> Richard Ogier wrote:
>>>
>>>>>      Acee: Show hands on what should be done:
>>>>>            - Quit working on OSPF MANET: none
>>>>>            - Continue to drive to consensus: none
>>>>>            - Refine drafts and publish as experimental: 2/3's of 
>>>>> people
>>>>>              in room. To be validated on list.
>>>>
>>>>
>>>>
>>>>
>>>> Acee,
>>>>
>>>> Correct me if I am wrong, but since the latest version of INRIA's
>>>> draft was available only last week, and since previous versions did
>>>> not fully specify the protocol (as pointed out by Phil Spagnolo in
>>>> his 9/28/06 post to the ospf-manet list), it has not yet been decided
>>>> that INRIA's draft will be published as experimental.
>>>>
>>>> Moreover, since INRIA has not participated in the GTNetS simulation
>>>> comparison that Boeing has been conducting for the last two
>>>> years, in which the MDR draft has been compared to Cisco's
>>>> OR/SP drafts (results can be found at Boeing's OSPF-MANET website
>>>> http://hipserver.mct.phantomworks.org/ietf/ospf/ ),
>>>> it is only fair that we should do such a comparison with INRIA's draft
>>>> before deciding to publish it as experimental.
>>>>
>>>> In fact, that has been the plan since the Dallas IETF meeting in 
>>>> March,
>>>> and Philippe agreed to this in his message of 4/5/06:
>>>>
>>>> Philippe Jacquet wrote on 4/5/06:
>>>> > Yes it would be great to synchronize our efforts on GTNet.
>>>> > Let's see how to proceed.
>>>>
>>>> Now, 7 months later, INRIA has implemented their solution in GTNetS,
>>>> so the next step would be for Boeing to work with INRIA to make
>>>> sure the code is debugged and implemented in a manner that allows
>>>> a fair comparison, just as Boeing has done with the OR/SP and
>>>> MDR solutions over the last two years.  Hopefully, this work can
>>>> be completed by the next IETF meeting.
>>>>
>>>> I think it is reasonable and fair to require such a comparison
>>>> to be done before INRIA's draft is accepted, especially
>>>> since they promised to synchronize efforts 7 months ago.
>>>> Let me know if you agree or disagree.
>>>> IMO, to give INRIA a free pass and avoid such a comparison
>>>> would be unfair to those of us who worked hard for the last two
>>>> years on the GTNetS simulation effort.
>>>>
>>>> Richard
>>>>
>>>
>>>
>>
>
>



_______________________________________________
OSPF mailing list
[e-mail]
https://www1.ietf.org/mailman/listinfo/ospf