Re: Comments on draft-morin-l3vpn-mvpn-considerations-02

[Thomas Morin <[e-mail]>]

Hi working group, Eric,

This is a bit "off" in terms of timing, given that the debate on WG
direction has evolved in the last days.  However, I think that it can be
useful to address the numerous comments made by Eric which challenge our
draft. So, please look at this reply, not as a meant to revive a
point-by-point debate, but as a attempt at adjusting the balance after
Eric vocal opposition to this draft.

My answers are inlined below as reply to Eric's comments, but before
jumping in, I want to make the following comments:
- it seems that Eric first opposes the goal of this draft and would
advocate an approach based on profiles -- this looks to me as
fundamentally backwards to what the IETF tries to do, and if taken, we
would need to know *why* multiple profiles are needed
- among the recommendations that we make, Eric seem to oppose
essentially two of them : (a) making BGP, alongside with PIM, mandatory
for C-multicast routing ; and (b) making segmented inter-AS mandatory  ;
yet I didn't see any solid arguments explaining why adopting (a) and (b)
would be limiting for deployments (and possibly pushing for having
multiple profiles)

Though disagreeing with Eric on many points (see below), I still want to
acknowledge the fact that we found many of the comments useful to
improve the document in many places. Last revision of the draft (03) was
made based on many of such comments.

-Thomas




Eric Rosen : 
> I  have  thoroughly  read  draft-morin-l3vpn-mvpn-considerations-02,  and  I
> regret that I continue to feel that  it is nowhere near ready to be accepted
> as  a  WG document.   There  are  too many  places  where  the reasoning  is
> superficial  or incomplete,  or  where  the reasoning  given  just does  not
> support the putative conclusions.  I will go into this in some detail below.
> 
> If the WG  really wants to have a document which  provides a sound technical
> comparison  of various options,  I would  suggest that  a different  sort of
> "design team" is needed.
> 

( Wow, that was a tough introduction! I can only say that I hope people
had enough perspective to take this with a grain of salt; the answers
below can hopefully help people make their own idea about the relevance
of each of your comments. )  


> 1.  Introduction
> 
>    The current proposal for multicast in BGP/MPLS
>    [I-D.ietf-l3vpn-2547bis-mcast] includes multiple alternative
>    mechanisms for some of the required building blocks of the solution.
>    However, it does not identify the core set of mechanisms which must
>    be implemented in order to ensure interoperability.  This may lead to
>    a situation where implementations may support different subsets of
>    the available optional mechanisms leading to implementations that do
>    not interoperate.
> 
> **** Since one size does not fit all, not all service providers will want
> **** the same sets of options all the time. 

It seems to me that blindly applying this type of arguments would make
it hard to produce standards.  For what is related to mVPN, please be
more specific and indicate which of the recommendations that we make
would be limiting to fit the needs of an operator.

> **** To ensure interoperability, one
> **** needs to define a set of profiles, each of which contains a specific
> **** set of options that work with each other.

You are suggesting that what you describe is the only way, whereas it is
obvious that this is only one way among many others, and which happen to
be the opposite but of what is most often done in the IETF (producing
one standard with a set of mandatory procedures ensuring
interoperability).


> ****  Then the question of whether any profile
> **** should be mandatory can be addressed.  Just providing a single
> **** mandatory set of procedures does nothing to ensure interoperability
> **** when non-mandatory options are used.  

I agree that it would be useful to have a place where the compatibility
between different options is documented.  This is probably an area where
draft-ietf-22547bis-mcast could be improved.

Then, before jumping in on the "profile" approach, we would need to have
an idea of what the compatibility matrix look like : if the number of
"non-compatibility" points in the matrix is low, then it would probably
be more efficient to just document these than list all the combinatory
cases of the things that can work together.



> 3.  Examining alternatives mechanisms for MVPN functions
> 
> 3.1.  MVPN auto-discovery
> 
>    Section 5.2.10 of [RFC4834] states "The operation of a multicast VPN
>    solution SHALL be as light as possible and providing automatic
>    configuration and discovery SHOULD be a priority when designing a
>    multicast VPN solution.  Particularly the operational burden of
>    setting up multicast on a PE or for a VR/VRF SHOULD be as low as
>    possible".
> 
>    The current solution document [I-D.ietf-l3vpn-2547bis-mcast]
>    addresses this requirement by proposing two different mechanisms for
> 
>    MVPN auto-discovery:
> 
>    1.  BGP-based auto-discovery (described in section 4).
> 
>    2.  Discovery using PIM running on a MI-PMSI implemented with a
>        shared tree using multicast ASM, or MP2MP LDP with the same
>        common tree identifier configured in all VRFs of an MVPN.
> 
>    It is the recommendation of the authors that BGP-based auto-discovery
>    is the preferred solution for auto-discovery and should be supported
>    by all implementations while PIM/shared-tree based auto-discovery
>    should be optionally considered for migration purpose only.
> 
> **** I would agree that the spec should say that BGP-based discovery SHOULD
> **** be used.  However, the arguments below seem to be mostly specious to
> **** me.

You have actually a good point. Though I would disagree with calling
them "specious", not all of the arguments used are needed to came to the
conclusion on which everybody seems to agree : that the BGP-based
autodiscovery procedure is among the procedures that should be part of
the  set of mandatory procedures.  

(we have now reduced this section to the most relevant arguments)


> Furthermore, PIM based discovery is only applicable to deployments
>    using a shared tree on an MI-PMSI, whereas BGP-based auto-discovery
>    does not place any restrictions on the type of multicast trees that
>    can be used.     BGP-based auto-discovery is independent of the type of
>    P-multicast tree used thus satisfying the requirement in section
>    5.2.4.1 of [RFC4834] that "a multicast VPN solution SHOULD be
>    designed so that control and forwarding planes are not
>    interdependent". 
> 
> **** I think this paragraph is confused.  MI-PMSI is not a forwarding plane,
> **** it is a service.

( Sorry, but, where do you see that the above paragraph suggest that an
MI-PMSI "is a forwarding plane" ?  )

Let me try to guess what may have confused you : the second line of the
paragraph is not well worded, and should read "using a shared tree to
instantiate an MI-PMSI" (instead of "on an MI-PMSI").

We have improved the wording to clear up whatever confusion there may
have been... (so much for a phrase not very well worded ?)



> ( the authors note that, in order to support the coexistence of both
>    protocols (for example during migration scenarios), implementations
>    could support both alternatives by providing a per-VRF configuration
>    knob that would allow recognizing new PIM neighbors based on the
>    reception of PIM hellos on a shared P-multicast tree, even for
>    neighbors that did not advertise a BGP auto-discovery route )
> 
> **** Are you saying that PEs should refuse to accept PIM hellos in advance
> **** of receiving the BGP auto-discovery routes?  I don't think that
> **** behavior is currently required by any specification.  

Our text is here not specific enough : we think about the case
where BGP-based autodiscovery is used and BGP-based C-multicast routing
is used, and expect that in that case, there should not be any PIM
Hello, and thus no reason to accept them...
...but in a migration from PIM/MI-PMSI/shared-tree autodiscovery, there
is a reason to accept them and a tuning knob could be used in that case.

-> the idea was to highlight a migration path from draft-rosen to the
BGP-based autodiscovery + BGP-base C-multicast routing.  

This is related to autodiscovery, but would be better place in the
section related to migration. We have moved it and improved the wording.


> 
> **** What would be the advantage of imposing such a rule?
> 

Your question/suggestion is, if I understand, "When BGP-based
autodiscovery is used, what would be the advantages of refusing Hello
from neighbor not having advertised a BGP autodiscovery route"... ?

Maybe it could be helpful in avoiding some issues : in cases where
BGP-based autodiscovery is used by all PEs, such Hellos should not
exist, and seing any could be a sign of some issue, right ?

(in any case, that would relate more to draft-ietf-2547bis-mcast than to
our draft...)


> 3.2.  S-PMSI Signaling

> The current solution document [I-D.ietf-l3vpn-2547bis-mcast] proposes
>    two mechanisms for S-PMSI Signaling:
> 
>    1.  A new UDP-based TLV protocol specifically for S-PMSI signaling
>        (described in section 7.2.1).
> 
>    2.  A BGP-based mechanism for S-PMSI signaling (described in section
>        7.2.2).
> 
> 
> **** Please note that the adjective "new" has been applied to the wrong
> **** solution ;-) A typo, no doubt.

Wops, I now see the double meaning : we meant "new" as "additional", not
as "more recent".  In any case, why "additional" would be the proper
term is explained later in that section and has no real place in the
intro of the section.  We've fixed this.


> It is the recommendation of the authors that BGP is the preferred
>    solution for S-PMSI signaling and should be supported by all
>    implementations while the UDP-based S-PMSI signaling protocol should
>    be considered optional.
> 
>    Part of the rationale for this recommendation is similar to that for
>    BGP-based auto-discovery and is based on section 5.2.10 of [RFC4834]
>    and the desire to avoid introducing and deploying additional
>    protocols unless strictly necessary.
> 
> **** See my comment above about referring to the major additions to BGP as
> **** "no new protocol".

No, I disagree, given that the BGP-based is agreed upon as a required
mechanism (at least for autodiscovery), and that the S-PMSI signaling
part is only a variation of it, the BGP-based S-PMSI signaling certainly
doesn't qualify as a significant addition to the stack, which contrasts
with the UDP-based approach.


> Furthermore:
> 
>    o  The BGP-based S-PMSI signaling mechanism can be efficiently used
>       in an inter-AS option B deployment context while the use of the
>       UDP-based protocol does not preserve AS routing independence when
>       used in an inter-AS option B context (i.e. the decision by a PE in
>       an AS to use an S-PMSI for a given customer flow will impact
>       routing state in other ASes).  Co-existence with unicast inter-AS
>       VPN options is strongly encouraged by section 5.2.6 of [RFC4834].
> 
> **** I don't understand what is being said here.

When the UDP-based signaling is used in an inter-AS case, a UDP message
will signal the S-PMSI toward all the PEs in remote ASes, which then
have to join this S-PMSI if they have receivers for the corresponding
mVPN traffic, thus creating state in the P-routers in their AS.  In an
inter-AS option B context, this is new because what PE in a remote AS
decides isn't in unicast, supposed to impact the state in the P routers
of the remote AS.   The BGP-based method to signal S-PMSI, when
segmented-tree are used, doesn't have this drawback.

We have improved the wording.



> 
>    Therefore, it is the opinion of the authors that BGP is the preferred
>    solution for performing S-PMSI signaling.
> 
> **** Generally, if one lists the advantages of scheme A and the
> **** disadvantages of scheme B, one finds that A looks to be better than B.
> **** If one takes the trouble to also list some of the advantages of B and
> **** disadvantages of A, one may draw a different conclusion.

You are making here a claim that we are not impartial.
That claim could appear as legitimate and properly supported if the
example you took was an illustration of relevant "disadvantages of A"
that "we would have seen if we had been taking the trouble" : your
example is in fact nothing obvious, and totally debatable, both on its
veracity and on its relevance.

Let's me expose why...

Eric: 
> **** For instance, note that the BGP-based S-PMSI signaling requires all PEs
> **** of a particular MVPN to maintain state for each C-flow that has been
> **** assigned to an S-PMSI.  The PEs must maintain this state even if they
> **** in fact have no receivers for that C-flow.  If one uses a
> **** datagram-based mechanism, a PE does not have to retain any state for a
> **** flow unless it has receivers for that flow.
> 
> **** It is true, as the authors point out, that one can't use the
> **** datagram-based procedure unless one has set up a tunnel to carry the
> **** datagrams.  If one doesn't want to set up tunnels that carry only
> **** control messages, then an "out of band" signaling mechanism such as
> **** that offered by BGP (or, perhaps, unicast PIM) is needed.  On the other
> **** hand, if one does have the tunnel available to carry the datagrams, why
> **** should one be forced to maintain all the extra state imposed by the
> **** BGP-based procedure?

...
So, this was discussed on the list by Rahul and Yicun, but let me
summarize the outcome of their short discussion: if an S-PMSI is
signaled, a PE has to maintain S-PMSI-switching signaling related state
if we want it to be able to honor a join coming from a CE quickly. This
is quite obviously totally independent of the choice of the protocol
used to indicate the switching.

Now, a few questions: do the PEs strictly _have_ to maintain this extra
state ? and is this specific to the UDP or BGP protocol ?
  --> no, in with both protocol they can drop this state, and both can
retrieve it later :
    * with UDP, by waiting for a retransmission
    * with BGP, by asking the BGP peers to refresh their 
  --> the resulting effect is that they may not be able to timely honor
a join coming from a PE

Thus, your point is totally irrelevant: the cases where we would think
it wise for a PE to drop a state are very few (maybe extreme control
plane load??), and even if we are in a such case, the two approaches for
S-PMSI switching signaling could *both* drop the SPMSI-switching states.

So, we could certainly tackle this issue in the draft, but not before
you explain us how it can be helpful in distinguishing one approach as
better than another one...


> **** Further, in the absence of any well understood criteria for assigning
> **** and/or removing particular C-flows to/from particular P-tunnels, we
> **** don't know what the rate of change is likely to be or what impact this
> **** is likely to have on the BGP route reflectors.

If you were trying to help toward a balanced analysis, you would
probably not insist "on this impact of [BGP-based S-PMSI signaling] on
the BGP route reflectors", but you would look at the overall impact on
the *whole* provider backbone infrastructure (PE, P, RR).

Let me be more precise : under an hypothetical scenario where S-PMSI
would rapidly be signaled in the backbone, and supposing that this
signalling is honored by downstream PEs, that could impact all the
provider infrastructure:
* the PE routers attached to receivers would have to process the
signaling and trigger the related procedures
* the P routers would likely see fast paced changes in the P-tunnels
control plane (PIM-SM/SSM signaling, mLDP signaling, RSVP-TE
signalling...)
* the RR  (if the BGP-based signaling is used with RR) would have to
process the signaling

The reason why we didn't include the study of the effects of a high rate
of S-PMSI signaling on the PE/RR infrastructure, is that, given the
impact on the P routers of such a scenario we would not expect a vendor
or provider to setup S-PMSI trigger criteria that wouldn't protect all
the infrastructure from such quick-paced changes.

That may be an overlook from us (the current solutions draft don't
recommend such a damping mechanism, AFAIK ?), but you are wrong in
seeing unfairness in this.

We have improved the document to better cover this point.

[ Thinking about it, it occurs to me that rate-limiting the pace of
creation of S-PMSI is easy to do on a per-PE basis, but for protecting
all the infrastructure, it would be even more relevant to limit S-PMSI
tunnels on a global scope.  I know you won't like it, but isn't it true
that the BGP RR could be used to globally limit the rate of S-PMSI
signaling and impact on S-PSMI tunnels signaling ?  :D  ]


> 
> **** I do not believe that the authors have given full consideration to all
> **** the pros and cons of each feature, and I therefore believe that their
> **** conclusions have not been properly supported.

( Let's give other people the freedom of deciding if your arguments are
good enough for you to dismiss our conclusions in a such definitive
way...  )


> 3.3.  PE-PE Transmission of C-Multicast Routing
> 
>    The current solution document [I-D.ietf-l3vpn-2547bis-mcast] proposes
>    multiple mechanisms for PE-PE transmission of customer multicast
>    routing information:
> 
>    1.  Full per-MVPN PIM peering across an MI-PMSI (described in section
>        5.2.1).
> 
>    2.  Lightweight PIM peering across an MI-PMSI (described in section
>        5.2.2)
> 
>    3.  The unicasting of PIM C-Join/Prune messages (described in section
>        5.2.3)
> 
>    4.  The use of BGP for carrying C-Multicast routing (described in
>        section 5.3).
> 
> 3.3.1.  PE-PE signalling scalability
> 
>    Scalability being one of the core requirements for multicast VPN, it
>    is useful to compare the proposed C-multicast routing mechanisms from
>    this perspective : Section 4.2.4 of [RFC4834] recommends that "a
>    multicast VPN solution SHOULD support several hundreds of PEs per
>    multicast VPN, and MAY usefully scale up to thousands" and section
>    4.2.5 states that "a solution SHOULD scale up to thousands of PEs
>    having multicast service enabled".
> 
>    At such scales of multicast deployment, the first and third
>    mechanisms require the PEs to maintain a large number of PIM
>    adjacencies with other PEs of the same multicast VPN (which implies
>    the regular exchange PIM Hellos with each other) and to refresh
>    C-Join/Prune states, thus limiting the scalability of these
>    approaches.
> 
> **** This analysis is questionable, and appears to be based on unstated
> **** assumptions.
> 
> **** If one assumes that the Join/Prune states don't change much, then of
> **** course the refresh overhead is useless overhead.  On the other hand, if
> **** one assumes that the Join/Prune states change frequently, perhaps more
> **** frequently than than the refresh rate, the overhead due to the
> **** refreshes of things that don't change is just in the noise.
> **** Furthermore, BGP has never shown itself to be scalable in the face of
> **** such rapid changes.  Without an analysis of rate of change, the
> **** conclusions above are unsupported.

You will find in Appendix A.4 an analysis that covers such issues, and
supports the conclusions of this section. 


> **** There is also no analysis to show that PE-PE PIM overhead is going to
> **** be the bottleneck, rather than CE-PE PIM, which the authors appear
> **** willing to tolerate.  

Our conclusions are focused on the PE-PE signaling, and this is nowhere
near being a "unstated assumption": the title of this section is "PE-PE
signaling scalability" and the introduction clearly reminds this by
mentioning PE-related orders of magnitude. 

Scalability issues at the PE-CE interface are not among the things on
which draft-ietf-2547bis-mcast proposes multiple alternative solutions.

We don't make any assumption on which between CE-PE and PE-PE, would be
the "bottleneck" side.  There certainly *are* realistic cases (if we
look at the orders of magnitude coming from the survey in RFC4834) where
the PE-PE side has scalability challenges, and we try to look how these
can influence the choice of PE-PE signaling technique.

Trying to get back to your point... Given the historical context in mVPN
and past discussion, I'm wondering if your underlying suggestion would
be to say : we should consider the CE-CE scalability case, because in
that case, evolution of PIM may be required (like PIM PORT), and then we
might considering using such evolutions to solve the PE-PE scalability
case.

If this is what we have in mind, then please be explicit about it, and 
please acknowledge that, given that how to use such an approach is not
described in draft-ietf-2547bis-mcast, it would be out of our scope to
introduce it in our draft.

(If this is not what you have in mind, please be explicit in how you
think the PE-CE signaling case would be helpful in discerning the
different approaches for PE-PE signaling).


> **** Further, the assumption that we can eliminate much of the state that
> **** constitutes the "PIM adjacencies" is questionable.  The fact is that
> **** whether one is doing PE-PE PIM or not, there is considerable state that
> **** must be devoted to keeping track of who the upstream PE is for each
> **** C-multicast flow.  One has to keep track of unicast routing changes
> **** that impact this, one has to make sure that packets one receives are
> **** from the right upstream PE, etc.  This is a good piece of what is
> **** involved in maintaining the PIM adjacencies.

What you mention above is a mix of control and dataplane work that (a)
has to happen whatever the approach is chosen for PE-PE signaling and
(b) is typically roughly a linear scalability issue with the number of
unicast/multicast states (or probably more likely linear with the number
of changes of these states).   So, that lead me to think that this is
not the major part of processing when the number of PE increases to high
numbers, nor something in which different approaches would perform
differently.  If you have data or examples that show this reasoning
wrong, then please provide them.



>    The third mechanism would reduce the amount of C-Join/Prune
>    processing for a given multicast flow for PEs that are not the
>    upstream neighbor for this flow, but would require "explicit
>    tracking" state to be maintained by the upstream PE, and would
>    require refresh-reduction mechanisms to be used to mitigate the fact
>    that PIM "Join suppression" cannot be used (what such a refresh-
>    reduction mechanism would be has not been described yet).  For these
>    reasons, it seems that this approach is not suitable for higher scale
>    scenarios.
> 
> **** I'm not sure I understand the relationship between PIM Join Suppression
> **** and refresh reduction.

( Well, both "Join Suppression" and "refresh reduction" are ways to
reduce the amount of messages exchanged between PEs.  The reasoning is
that if you don't have one then you likely need the other. ) 


> **** By the way, the BGP-based signaling has an entire set of messages and
> **** procedures whose purpose is to support explicit tracking: Leaf A-D
> **** routes.  These messages and procedures exist for two reasons: (a) to
> **** support aggregation schemes that try to aggregate trees that are more
> **** or less congruent, and (b) to support the use of RSVP-TE P2MP LSPs to
> **** instantiate S-PMSIs.
>
> ****  If the authors believe that explicit tracking is
> **** inherently unscalable, they should go on to point out that neither the
> **** use of RSVP-TE P2MP LSPs nor a strategy of "aggregation by congruence"
> **** are scalable schemes.

The explicit tracking you mention above is not specific to the use of
BGP for PE-PE signaling, but to aggregation and S-PMSI.

Our draft highlights a drawback of the "unicast PIM" approach that
*requires* explicit tracking in all cases , but we would have no
legitimacy to highlight a drawback of an *optional* deployment choice
that would require explicit tracking for *whatever* approach is chosen
for C-multicast routing; that would be totally out of our scope, because
not helpful in comparing the different options. 


(as a side note, I somehow disagree with your statement above : doing
explicit tracking *for all multicast states* or *for only multicast
states that are bound to S-PMSI or possibly subject to refined
aggregation* is different from a scalability point of view)



> The first and second mechanisms can leverage the "Join suppression"
>    behavior and thus improve the processing burden of an upstream PE,
>    sparing the processing of one Join message for each remote PE joined
>    to a multicast stream, but this improvement comes at the price of
>    requiring all PEs of a multicast VPN to process all PIM Joins sent by
>    any PE participating in the same multicast VPN whether they are the
>    upstream PE or not.
> 
> **** What one gains for this "price" though is that a PE won't send a Join
> **** at all if it sees that someone else has already sent it.  The price is
> **** thus well worth it if there are lots of nodes with receivers for the
> **** given group.  The price is only high when there aren't lots of nodes
> **** with receivers.  So the analysis seems to assume that the receivers for
> **** each group are sparsely distributed among the PEs of a particular VPN.
> **** The authors have not provided any grounds to support that assumption.

We don't say anything about whether or not the price outweights the
benefits. Therefore I don't see why you state that we make an
"unsupported assumption".  I'm not sure what was your point, except
challenging the use of the "comes at at price" phrase. We have changed
the wording but, and Appendix A has more data on quantification to
compare the benefit/price of this approach ; as far as I can't see it
doesn't change the conclusion that we can draw. 


> The fourth mechanism (the use of BGP for carrying C-Multicast
>    routing) would have a comparable drawback of requiring all PEs to
>    process a BGP C-multicast route only interesting a specific upstream
>    PE.  For this reason the C-multicast routing approach leverages the
>    Route-Target constraint mechanisms, which specifically allows only
>    the interested upstream PE to receive a BGP C-multicast route.  When
>    RT constraints are used the fourth mechanism reduces the processing
>    load put on the provider infrastructure for customer multicast
>    routing to the minimum 
> 
> **** I think this is a really nifty feature of the BGP method, but let's not
> **** get carried away.  One cannot say that this method keeps the processing
> **** load at a "minimum", because this method causes all nodes with
> **** receivers to send Joins, whereas a true Join suppression scheme would
> **** allow those joins to be, err, suppressed.

(this is not exact: a PE always sends its first Join as per PIM specs,
and with the BGP-based approach only the first Join is sent since there
aren't refreshes...)


> **** Further, the route 
> **** reflectors are processing all these joins, even if all the PEs are not.
> **** The authors just have not provided any grounds for calling this
> **** "reducing the processing load on the infrastructure to the minimum".

The whole paragraph that you quote was finishing by:

  "(Please refer to Handling the PIM routing processing load load, for a
detailed explanation of the differences in ways of handling the
C-multicast routing load, between the PIM-based approaches 
and the BGP-based approach)".

The misformating was due to XML postprocessing. Maybe you missed
"Appendix A", and maybe this is the reason.  That appendix tries to sort
out the differences between the "join suppression/prune override", the
"explicit tracking" and the "BGP C-multicast routing" approaches, and
was the place intended to provide grounds behind stating that the BGP
approach reduces the processing. 

This appendix was updated in the last revision to go further in
supporting the conclusions with quantified comparisons.

Now, to discuss your technical comments above...

> One cannot say that this method keeps the processing load at a
> "minimum", because this method causes all nodes with receivers to send
> Joins


See improved Appendix A in last revision for rationale backing the
"minimal processing" argument.


> Further, the route reflectors are processing all these joins, even
> if all the PEs are not


No, if a hierarchy of RR is used, then each RR essentially only
processes the C-multicast routes of the PEs/RRs directly peering with
him (ie. not all the joins of all PEs).

To address your concerns, we have improved appendix A to better quantify
the amount of processing done in each case by each network element, and
total.  The text in the "PE-PE signaling" had already be written with
such considerations as background, and was updated to give some
additional nuances, but as you will see, conclusions are not really
changed.



> **** One also needs to look at related processing loads which are not
> **** mentioned here.  If one needs to do explicit tracking, say, because one
> **** is using RSVP-TE P2MP LSPs as S-PMSIs, then not only does each PE with
> **** receivers have to send a Join, each one has to send a Leaf A-D route as
> **** well.

(As mentioned above : this out of scope of our document, what you say is
true whatever the approach is chosen for C-multicast routing...)


> (by avoiding any processing by "unrelated"
>    PEs, that are nor the joining PE nor the upstream PE), and inherits
>    BGP features that are expected to improve scalability (through, for
>    instance, providing a means to offload some of the processing burden
>    associated with client multicast routing onto BGP route-reflectors),
>    and being based on TCP has no refresh-related scalability limit.
>    (Please refer to Handling the PIM routing processing load load, for a
>    detailed explanation of the differences in ways of handling the
>    C-multicast routing load, between the PIM-based approaches and the
>    BGP-based approach)
> 
> **** The savings gained by using route reflectors is not nearly as large in
> **** multicast as in unicast.
> 
> **** In unicast, if there is no RR, then for a given VPN, each PE would need
> **** to unicast a copy of the VPN routing info to every other PE that is
> **** attached to the VPN.  So by using an RR, you end up transmitting much
> **** less info.  But in multicast, you don't unicast the same info to each
> **** PE, you multicast it, so you're only sending it once.  Using the RR
> **** provides no off-loading at all of the processing load to do the
> **** transmissions.

Sorry Eric, but here you are making a large confusion between the
multicast dataplane and the multicast control plane. 

You say, "in multicast, you don't unicast the same info to each PE, you
multicast it, so you're only sending it once". This is (obviously) true
for the *dataplane*, and mvpn uses provider trees (whatever flavor) to
do this efficiently. But this is wrong for the C-multicast routing
control plane : multicasting this info to all PEs is *one* way of doing
that (the one PIM uses on a LAN), but not necessarily the only one or
the most efficient one.  The function played by multicast routing across
an mvpn provider core is to let *one* PE --the upstream PE for a
multicast stream-- know when it has to start or stop sending traffic
across the core ; this routing function is in no way related to
"multicast", as the information sent is to be acted upon by *one* PE.

(Furthermore you also say that for unicast VPN routing, not having
multicast is ok, but this is in fact for this type of routing that you
could see an interest in using multicast, because unicast VPN routing
information interests multiple PEs... but well, today's subject is not
to discuss the use of multicast *by* BGP...)

So, well, it seems to me that your reasoning is wrong.

What I would say is that, for mVPN C-multicast routing, RR have the
advantage of :
a) not having to setup a full mesh of TCP sessions (more or less
relevant depending on platform limitation)  -- this is common with
unicast routing 
b) providing a way to spread the C-multicast routing state maintenance
-- this is specific to mVPN C-multicast routing


> **** In the BGP-based C-multicast scheme, every PE has to choose its
> **** own upstream PE for each C-multicast flow, track changes in the
> **** unicast routing that might affect its choice, participate in the
> **** signaling to construct the multicast trees, etc. The only
> **** processing load that the RR really saves is the  processing
> **** needed to do Join Suppression.  

Yes, the RR saves the processing needed to do Join suppression, and
doesn't suppress the rest.  
But:
- the rest, is *common* with all other approaches (and partly with
unicast routing too)
- the "only" in your last sentence is a quantitative evaluation that you
make, and which I don't see supported by any argument


> **** And if you have a need to do explicit tracking, it doesn't even
> **** do that.

It is the use of RSVP-TE based P2MP for S-PMSI or fine grained
aggregation, not BGP-based C-multicast routing, that makes  explicit
tracking needed (though it doesn't have to be done for all mcast
states, but only the one concerned by the feature ). 


>    However, it is to be noted that offloading customer multicast routing
>    processing onto BGP route-reflectors will increase the processing
>    load placed on the route-reflector infrastructure, which, in the
>    higher scale scenarios, is expected to call for adaptations such as:
> 
>    o  a separation of resources for unicast and multicast VPN routing :
>       using mvpn-dedicated BGP sessions and/or mvpn-dedicated BGP
>       instances on route-reflectors, and/or mvpn-dedicated route-
>       reflectors ;
> 
>    o  the deployment of additional route-reflectors resources :
>       increased processing resources on existing route reflectors or
>       additional route-reflectors.
> 
> **** So at the end of the section, we see a few understated sentences
> **** saying, "by the way, the BGP scheme may not work at scale with the
> **** existing route reflector infrastructure or even with existing route
> **** reflector implementations".  

The text you mention was essentially added to address issues you
mentioned, and doesn't look to me as "a few understated sentences" at
all. If it was, you would likely not use it as you do below.


> **** To me, that seems like a rather serious
> **** disadvantage, which should not be dismissed or minimized with wishful
> **** thinking about future "adaptations".

Do you think that "deploying mvpn-dedicated route reflectors" are
wishful thinking ?

Do you think that "increasing RR resource" to cope with an increasing
routing scale is wishful thinking, or not reasonable ?  It seems to me
as a normal approach to scaling (scalability is not unlimited growth
with a said equipement, but reasonable growth of equipement when
deployment and services increases in scale).

( With the default PIM LAN procedures, adding one PE or one multicast
stream creates additional load on *all* PEs of an mVPN. In context where
these number go high, it might require upgrading all these PEs. For
sure, suggesting *that* would be wishful thinking... )


> **** I do agree that C-multicast routing might call for major changes in the
> **** RR implementations. 

(by the way, I don't known with whom you "agree" -- with yourself maybe?
we didn't state anything like this...)


> **** The fact that the authors make a statement about the need to change all
> **** the route reflectors 

That is a misinterpretation (or at least a very rough shortcut of what
we say). What we say is that increased in mvpn scale may require an
evolution of the RR infra. In case of such a need, the most likely is
seeing dedicated mVPN RR be used.

> **** makes me worry that the BGP solution may not be
> **** anywhere near ready for PS status.  Perhaps it should be advanced as
> **** experimental until we have a better understanding of its properties.

The processing done on RR in the context of mVPN routing is pretty much
an intrinsic/design aspect of the BGP-based proposal (well, that is, if
the provider uses RR for mVPN...).  You are co-authoring the BGP
proposal, so I'm surprised to see you (fake?) discover that...

You are overinterpreting our text, which might draw us into
caricatural discussions.  We will do like if this overinterpretation is
just our fault and have tried to refine the text in question in the last
revision.




> 3.3.2.  P-routers scalability
> 
>    Mechanisms (1) and (2) are restricted to use within multicast VPNs
>    that use an MI-PMSI, thereby necessitating:
> 
>       the use of a P-multicast tree technique that allows shared trees
>       (for example PIM-SM in ASM mode or MP2MP LDP)
> 
>    or   the use of one P-multicast tree per PE per VPN, even for PEs
>       that do not have sources in their directly attached sites for that
>       VPN.
> 
>    By comparison, the fourth mechanism doesn't impose either of these
>    restrictions, and when P2MP trees are used only necessitates the use
>    of one tree per VPN per PE attached to a site with a multicast source
>    or RP (or with a candidate BSR, if BSR is used), thereby improving
>    the amount of state maintained by P-routers compared to the amount
>    required to build an MI-PMSI with P2MP trees.
> 
> **** I have to say I cannot follow the reasoning here. 

(you reformulate quite well the reasoning we made, in fact !)

> **** Perhaps the point the authors are trying to make is the following.  If
> **** one uses multicast distribution trees to transmit control packets,
> **** there is a chance that one will need to set up multicast distribution
> **** trees that only carry control packets (and that are not needed for
> **** carrying data) .  Each such tree requires state in the P routers.  One
> **** uses less state in the P routers if one only sets up the multicast
> **** trees that are needed for carrying data packets, and doesn't set up any
> **** just for carrying control packets.
>
> **** This reasoning depends heavily upon unstated assumptions about how the
> **** multicast sources are distributed around the customer sites.  


We have updated the text to make the assumption explicit and give a
reference that this is a relevant assumption for many use cases.

Please see the following extract of section 4.2.4.1 of RF4834 :

        4.2.4. PEs per Multicast VPN
        
        [...]
        
        4.2.4.1. ... with Sources

           The number of PEs (per VPN) that would be connected to
        sources seems to be significantly lower than the number of PEs
        per VPN.  This is obviously related to the fact that many
        respondents mentioned deployments related to content broadcast
        applications (one to many).
        
           Typical numbers are tens (6 responses) or hundreds (4
        responses) of source-connected PEs.  One respondent expected a
        higher number of several thousands.


> **** One might also ask whether this is anywhere close to being a bottleneck.

For a majority of cases, it would be a difference of an order of
magnitude, so it looks like it is relevant from a scalability point of
view. (See improved Appendix A)


> **** Nevertheless, it is shown in draft-rosen-mvpn-profiles how one can use
> **** MP2MP LSPs to avoid setting up tunnels that are not needed for data
> **** packets, while still using PIM.

a) this draft proposal is out of our scope
b) this proposal doesn't make our statement false : you need MP2MP
connectivity for this proposal, which may not be available for all tree
signaling protocols


> **** If MP2MP LSPs are made a mandatory feature, then the issue goes away
> **** entirely.

Making MP2MP LSPs mandatory looks like your preferred approach.

Please note that this is totally opposite of what RFC4834 concluded :
"the control plane SHALL NOT depend on which forwarding plane is used
(and vice versa), and the choice of forwarding plane SHOULD NOT be
limited by the design of the solution." .


> **** Since MP2MP LSPs are the only reasonable way to support
> **** BIDIR-PIM C-flows, they are needed anyway.
> **** (Note that the other way of supporting BIDIR-PIM flows require either
> **** upstream-assigned MPLS labels or the "PE as RP" scheme, neither of
> **** which is regarded as a mandatory feature.)

Please explain/detail what you mean by "the only reasonable way".

The "Partitioned PE" approach (which you introduced), does actually
require "upstream assigned labels" -- is "upstream assigned labels",
which you co-authored largely, not reasonable ?

Also, you implicitly put the "Full PIM peering" over a MI-PMSI
instantiated with P2MP trees among the "not reasonable" approaches for
C-bidir support (since it does not require upstream-assigned labels or
"PE as RP" scheme).  I'm not sure why you think it is "not
reasonable".  

Could you expand your statement so that we can properly cover all
pro/cons of all the approaches wrt. C-bidir support ?

If we have relevant conclusions that would result from such  a
comparison, we will include this in the draft.

 
> **** (Note that the other way of supporting BIDIR-PIM flows require either
> **** upstream-assigned MPLS labels or the "PE as RP" scheme, neither of
> **** which is regarded as a mandatory feature.)
> 

This specific point is indeed a weakness of draft-ietf-2547bis-mcast.
Explaining clearly which feature/sub features are required/not-required,
and compatible/not-compatible, is really missing from this doc.


> 3.3.3.  Impact of C-multicast routing on Inter-AS deployments
> 
>    Furthermore, co-existence with unicast inter-AS VPN options, and an
>    equal level of security for multicast and unicast including in an
>    inter-AS context, are specifically mentioned in sections 5.2.6, 5.2.8
>    and 5.2.12 of [RFC4834].
> 
>    The first three mechanisms impose direct PE to PE communications :
>    this does not apply well to an inter-AS option B context, because of
>    security and robustness issues that are involved by such a level of
>    reachability and interaction between PEs in different ASes.
> 
> **** This is not a technical argument, just an unsupported claim.  It would
> **** be interesting to see this point developed in proper detail though.

We have improved the text in last revision, and we hope it is clearer. 



> Their use in an inter-AS context is possible, but not without
>    limitations or additional engineering design trade-offs depending
>    upon the interconnect types.
> 
> **** See previous comment.
> 
>    By comparison, the fourth option (the use of BGP for carrying
>    C-Multicast routing) does not have any of the above limitations
>    related to inter-AS deployments, and also provides an additional
>    alternative to facilitate such deployments through the possibility of
>    using segmented inter-AS trees.
> 
> **** The claim here is just wrong.  The use of segmented inter-AS trees does
> **** not presuppose the use of BGP C-multicast routing.

You are right, the current specifications do not exclude it from being
done. We have corrected the text. 

However, some of the advantage of the models do not hold anymore if PIM
is used in the segmented model approach. We have updated the text to
cover that.



> 3.3.4.  Security and robustness
> 
>    BGP supports MD5 authentication of its peers for additional security,
>    thereby possibly benefit directly to multicast VPN customer multicast
>    routing, whether for intra-AS or inter-AS communications.  By
>    contrast, with a PIM-based approach, no mechanism providing a
>    comparable level of security to authenticate communications between
>    remote PEs has been yet fully described yet
>    [I-D.ietf-pim-sm-linklocal], and in any case would require
>    significant additional operations for the provider to be usable in a
>    multicast VPN context.
> 
> **** See prior comment on security.  Also, how common is it for SPs to use
> **** MD5 on the BGP connections between PEs and RRs?



> The robustness of the infrastructure, especially the existing
>    infrastructure providing unicast VPN connectivity, is key.  The
>    C-multicast routing function, especially under load, will compete
>    with the unicast routing infrastructure.  With the PIM-based
>    approaches, unicast and multicast VPN routing are expected to only
>    compete in the PE, for routing plane processing resources.  In the
>    case of the BGP-based approach, they will compete on the PE for
>    processing resources, and in the route-reflector if they are used.
>    It is identified that in both cases, mechanisms will be required to
>    arbitrate resources (e.g. processing priorities).  In the case of
>    PIM-based procedures, between the different control plane routing
>    instances in the PE.  And in the case of the BGP-based approach, this
>    is likely to require using distinct BGP sessions for multicast and
>    unicast, possibly toward distinct route-reflectors.
> 
>    Multicast routing is dynamic by nature, and multicast VPN routing has
>    to follow the VPN customers multicast routing events.  The different
>    approaches can be compared on how they are expected to behave in
>    scenarios where multicast routing in the VPNs is subject to an
>    intense activity.  Such a load would be comparable to the higher
>    scale scenarios described in xx (Section 3.3.1) and the fourth (BGP-
>    based) approach - when deployed to handle a significant multicast VPN
>    routing load - is expected to be the most efficient approach in a
>    such case.  
> 
> **** I missed the argument which leads to the conclusion that "the fourth
> **** approach ... is expected to be the most efficient approach"
> 
> **** Of course, if "expected" really just means "hoped", no additional
> **** reasoning is needed ;-)

Hey what a funny joke! ;D

More seriously, Appendix A.4 has now content that support the fact that
under dynamic load, the default PIM LAN Procedures have a cost that
differ from an order of magnitude (O(#mVPN_PEs) vs. O(1)) compared to
other approaches, for total message processing when a PE joins or
leaves. 

We have updated the text in question to be more precise.


> On the other hand, while the BGP-based approach is likely
>    to suffer a slowdown under a load raising beyond processing resources
>    (because of possibly congested TCP sockets), the PIM-based approaches
>    would react to such a load by dropping messages, with later failure
>    recovery through message refreshes, this being at the expense of some
>    predictability.
> 
>    In fact both situations are problematic, and what seems important is
>    the ability for the VPN backbone operator to (a) limit the amount of
>    multicast routing activity that can be triggered by a multicast VPN
>    customer, and to (b) provide the best possible independence between
>    distinct VPNs.  It seems that both of these can be addressed through
>    local implementation improvements, and that both the BGP-based and
>    PIM-based approach could be engineered to provide (a) and (b).  It
>    can be noted though that the BGP approach proposes ways to dampen
>    C-multicast route withdrawals and/or advertisements, and thus already
>    describes a way to provide (a), while nothing hasn't been yet
>    described for the PIM-based approach, though these type of approaches
>    rely on a per VPN dataplane to carry the mvpn control plane, and thus
>    might naturally benefit from this first level of separation to solve
>    (b).
> 
> **** It seems to me that the reasoning here just doesn't provide a basis for
> **** the conclusions.


This text that was added to address comments you made in December. I
agree that it doesn't help us much in finding a difference in the
approaches.  Do you think we shall remove it ?



> 3.3.5.  C-multicast VPN join latency
> 
>    Section 5.1.3 of [RFC4834] states that "the group join delay [...] is
>    also considered one important QoS parameter.  It is thus RECOMMENDED
>    that a multicast VPN solution be designed appropriately in this
>    regard.".  In a multicast VPN context, the "group join delay"of
>    interest is the time between a CE sending a PIM Join to its PE and
>    the first packet of the corresponding multicast stream being received
>    by the CE.
> 
>    The different approaches proposed seem to have different
>    characteristics in how they are expected to impact join latency:
> 
>    o  the PIM-based approaches minimize the number of control plane
>       processing hops between the PE of a new receiver and the PE of the
>       multicast source, and being datagram based introduce minimal
>       delay, thereby possibly having a best-case join latency as good as
>       possible depending on implementation efficiency
> 
>    o  the BGP-based approach uses TCP exchanges, that may introduce an
>       additional delay depending on BGP implementation performances, but
>       are expected to control the worst-case join latency under load
> 
>    o  the BGP-based approaches is designed to allow the introduction of
>       route-reflectors which will introduce an additional processing
>       delay between the receiver-PE and the source-PE
> 
>    o  in higher scale scenarios, the BGP-based approach is expected to
>       provide some control of the worst-case join latency whereas the
>       PIM-based approaches may behave less efficiently if PIM messages
>       are lost
> 
> **** There's that "is expected to" again.  What is the reasoning that would
> **** support this conclusion?  As far as I can tell, the BGP-based approach
> **** provides no more control over the worst case than any other possible
> **** approach.



Past discussions to which you participated would let me synthesize by
saying that : BGP uses TCP which may be slowed down by a congestion (if
load creates a congestion), while in the case of a congestion PIM would
drop packets and would require waiting for the refresh for the stream to
be sent.  We used "more control" to say with few words that :
(a) the BGP-based approach would spread the degradation of performance
evenly on streams being joined to (slowdown), while the PIM-based
approach would randomly concentrate the degredation on some streams for
which PIM messages are lost
(b) the "Join suppression" mechanism would itself be less efficient if
PIM Joins are lost ; that this important mechanism for PIM LAN
procedures to scale is itself less efficient in the problematic high
scale scenarios certainly makes PIM behavior less predictable in such
scenarios, thus "less control"

We improved the text to make these reasons clearer. 



> o  in higher scale scenarios, the introduction of route-reflectors in
>       the BGP architecture are expected to provide processing efficiency
>       which is expected to improve latency compared to the PIM-based
>       approaches
> 
> **** I don't see why.  Please provide the reasoning that supports this
> **** conclusion. 

We have improved the text to explain the reasons.


> This qualitative comparison of approaches tend to highlight that the
>    BGP based approach is designed for controlling the "worst-case" join
>    latency 
> 
> **** This claim has not been supported by any reasoning.  

"controlling the 'worst-case'" is maybe not the right formulation.  We
needed to formulate the fact that in case of packet loss between PEs,
PIM-based signaling changes at once, for some random streams, from
"good" to "very bad" (10s of seconds), while the BGP/TCP-based approach
will recover the failure and will result in a lower impact spread evenly
on the considered streams.  Text was I think improved to cover this.


> whereas for the PIM-based approaches seem to structurally be
>    able to reach the shorter "best-case" group join latency (especially
>    compared to deployment of the BGP-based approach where route-
>    reflectors are used). 
> 
> **** In other words, PIM provides lower latency in the typical case, where
> **** the environment is neither lossy nor congested.

(*and* where the control plane is would be overloaded.) 


> Doing a quantitative comparison is not
>    possible without referring to specific implementations and
>    benchmarking procedures, and would possibly expose different
>    conclusions, especially for best-case group join latency for which
>    performance is expected vary with implementations.  We can also note
>    that improving a BGP implementation for reduced latency of route
>    processing would not only benefit multicast VPN group join latency,
>    but the whole BGP-based routing.
> 
> **** Improving BGP is always useful, but it's hard to see what that has to
> **** do with any of the issues here.

This is simple: the additional delay due to passing through the RR could
be anything between nearly-zero, significant, and large... We only want
to highlight the fact that there are non-mVPN drivers for improving
RR/BGP performance.


> Last, it is to be noted that the C-multicast routing procedures will
>    only impact the group join latency of a said multicast stream for the
>    first receiver that is located across the provider backbone from the
>    multicast source.
> 
> **** This is not the case.  For one thing, different receiving PEs may
> **** select different upstream PEs for the same C-source.  PE1 may have
> **** joined C-(S,G) via PE2, but when PE3 tries to join via PE4, join
> **** latency is still an issue.

(this is not really typical, and having 1 receiver or having 2 receivers
seeing the mvpn join latency among <n> doesn't really change much...)


> **** We might also take a closer look at the case where PE1 and PE2 have
> **** both joined C-(S,G) via PE3.  If an I-PMSI is being used, then
> **** presumably PE1 and PE2 are already joined to it, and can immediately
> **** begin receiving the traffic.  But suppose an S-PMSI is being used, PE1
> **** is already joined to it, but PE2 has not joined it yet.  Since the
> **** BGP-based S-PMSI procedures require PE2 to know already that C-(S,G) is
> **** bound to a particular S-PMSI, then PE2 just has to join that S-PMSI.
> **** So that does reduce latency, but at the expense of state; each PE has
> **** to know, for ALL flows in the MVPN, which flows have been bound to
> **** which S-PMSIs.

(note that this is true whatever mechanism is used for C-multicast
routing, and whatever mechanism is used for S-PMSI signaling.)

> ****  Now suppose that the S-PMSI is instantiated by an
> **** RSVP-TE P2MP LSP.  In this case, PE2 cannot just join the S-PMSI.
> **** First it must issue a BGP Leaf A-D route specifying that it is
> **** interested in C-(S-G).  When PE3 sees this, it initiates the signaling
> **** to add PE2 to the LSP.  So although PE3 doesn't have to get a join from
> **** PE2, it does have to get a BGP message from PE2.

I don't see how it is relevant to compare approaches for C-multicast
routing.  What you are telling is that latency will be higher when
S-PMSI are used with RSVP-TE.  Well, yes, this is true, but this is here
irrelevant in the comparison of the different concurrent approaches
proposed.  If the provider chooses to use P2MP RSVP-TE for an S-PMSI,
then it just means that he is ready to pay the latency price (which will
be low or high maybe depending on the implementations), and this will be
true whatever mechanisms would be used...



> **** Increased prune latency of course, will keep unwanted traffic coming
> **** for a longer time.

And... ?

Since you mention prune latency : keep in mind that the "Full PIM
Peering" approach, by default (if explicit tracking isn't done), relies
on the "Prune override" mechanism that *will* induce an increased prune
latency (by default by 3s).

My point of view would be that an increased prune latency has only one
impact, which is a slight impact on average dimensioning to be done by
the operator, and thus not key to distinguish between approaches. But if
there are arguments against that, we can consider it in next revision.


> 3.3.6.  Architectural considerations
> 
>    The fourth mechanism (the use of BGP for carrying C-Multicast
>    routing) would appear to fit well with the current unicast
>    architecture as BGP is the customer routing distribution protocol
>    used in unicast VPNs and therefore using BGP for customer routing
>    distribution within multicast VPNs avoids the introduction of an
>    additional protocol that would require additional OAM processes and
>    tools.  
> 
> 
> **** As long as PIM is still used for CE-PE multicast routing, it is false
> **** to say that using PIM PE-PE requires an additional protocol or
> **** additional "OAM processes and tools".  I also do not understand why it
> **** is simply presupposed that the new BGP address families (and the 50
> **** pages of new specification added to BGP) do not impose requirements for
> **** any new OAM procedures.

Understanding the details of "PIM on a LAN" is different from
understanding how PIM behaves on a P2P link. So while some PIM OAM and
troubleshooting has to be done in all cases, doing it in the context of
PIM LAN procedures is certainly more complex.

But we have move away most contents of this section, this this is true
that PIM OAM will still necessarily be very present.


> An illustrative example of the benefit brought by consistency with
>    unicast design is how the "extranet" feature can be implemented :
>    when BGP-based mechanisms are used, the already defined and well
>    understood BGP route target import/export semantics are just reused
>    and applied to BGP mVPN routes.  By contrast, it is not specified how
>    implementing the same feature would be done in the context of other
>    alternative mechanisms, and unclear if this is possible without
>    significant engineering trade-offs given that their control plane is
>    tied to a specific MI-PMSI tunnel.  Note that the support for the
>    Extranet feature is stated as a MUST in sections 5.1.6 of [RFC4834].
> 
> **** Even the existing MVPN deployments, available for years, provide
> **** extranet support.  So there's a proof in practice that the BGP-based
> **** mechanisms are not needed for this purpose, notwithstanding the
> **** authors' theoretical arguments about abstract qualities of
> **** "consistency".

As far as I know:
- Extranet has *not* any IETF description in the context of draft-rosen
or for any PIM-based approach in the current mvpn solution draft
- the only thing that is remotely similar is Cisco's feature description
for Extranet with draft-rosen (easily found on the web), but :
  a) it applies to draft-rosen only (not to draft-ietf-2547bis-mcast)
  b) it lacks key details for being an implementable spec 
  c) in any case, it doesn't highlight any consistency-related
advantage!

Please point us to an IETF spec where Extranet is described in the
context of PIM-base C-multicast routing.  At that point we can see if
the same consistency advantage exists or not, than in the BGP case, and
see if the consistency argument has to be even more specifically
illustrated (or possibly removed if both approach expose the same
consistency advantage).


> Section 5.2.10 of [RFC4834] states that "as far as possible, the
>    design of a solution SHOULD carefully consider the number of
>    protocols within the core network: if any additional protocols are
>    introduced compared with the unicast VPN service, the balance between
>    their advantage and operational burden SHOULD be examined
>    thoroughly".  Considering that the recommendation of the authors
>    would be BGP for auto-discovery and S-PMSI signaling, the choice of
>    BGP for customer multicast routing would be consistent with the
>    protocol choice for unicast VPNs and would adequately address this
>    requirement.
> 
> **** Since PIM is needed in the PEs anyway, the choice of PIM addresses this
> **** requirement equally well.

True, strictly speaking, "number of protocols" would be equal...
We have removed this weak argument.


> 3.3.7.  Conclusion on C-multicast routing
> 
>    The fourth approach (BGP-based) for customer multicast routing
>    clearly presents some advantages over the PIM-based alternatives.
>    However it has yet to be deployed within an operational MVPN, and
>    only limited experience exists with its implementations.  By
>    contrast, PIM-based mechanisms lack many of these benefits and have
>    identified limitations in how they can handle customer multicast
>    routing load in higher-scale scenarios.  Despite these, experience
>    showed that the "Full PIM peering" approach is operationally viable.
> 
>    Consequently, at the present time and until there is experience with
>    all of the proposed mechanisms it is not clear which of the above
>    mechanisms should be recommended as the preferred solution to
>    implementers.  However, it would appear prudent for implementations
>    to consider supporting both the fourth (BGP-based) and first (full
>    per-MPVN PIM peering) mechanisms.  Further experience on both
>    implementations is likely to be required before some best practice
>    can be defined.
> 
> **** Let's see, we have one mechanism which is known to be operationally
> **** viable, and another that has only recently been invented, with which
> **** there is zero operational experience, and which has only one partial
> **** implementation.  Which one is eligible to be called a "best practice"?
> **** Which one should be called an "experiment"?  

Please develop how your comment above relates to the text in our draft ?
We do not suggest to make any particular approach "best practice".


> Moreover to improve the clarity of the proposed specifications,
>    considering that neither hello suppression nor refresh-reduction
>    procedures are currently specified or documented and that it is not
>    clear what the impact to the PIM state machine of these additional
>    procedures may be, the authors recommend that the proposals for
>    lightweight PIM peering across an MI-PMSI (the second mechanism) and
>    for the unicasting of PIM C-Join/Prune messages (the third mechanism)
>    be removed from the current solution document
>    [I-D.ietf-l3vpn-2547bis-mcast] (at least until they have been further
>    specified and both their impact and benefit on a multicast VPN
>    deployment is spelled out).
> 
> **** Those proposals appear only in the "framework" section, which seems
> **** perfectly appropriate.  If they do get developed at some later time, we
> **** wouldn't want them ruled out without consideration as being
> **** inconsistent with the framework, would we?  (BTW, there is a draft for
> **** TCP-based support of PIM now, being presented to the PIM WG.)  
> 
> **** Serious comments about how to improve the "clarity of the proposed
> **** specifications" would of course be welcome. 

draft-ietf-2547bis-mcast is already huge and complex, I think that the
final revision of this document should whether fully document things or
just not talk about them.  If needed an appendix briefly documenting
approaches that haven't been retained can be added for future reference.



> 3.4.  Encapsulation techniques for P-multicast trees
> 
>  ...
> 
>    Current unicast VPN deployments use a variety of LDP, RSVP-TE and
>    GRE/IP-Multicast for encapsulating customer packets for transport
>    across the provider core of VPN services.  
> 
> **** There are current unicast VPN deployments that also use MPLS-in-L2TPv3.
> **** Why is this not in the list of things that have to be supported for
> **** MVPN then?  

We don't pretend to give a comprehensive list, and didn't had any
contributor suggesting MPLS/L2TPv3.


> It is recommended that
>    implementations support the three corresponding multicast tree
>    encapsulations techniques, namely: mLDP, P2MP RSVP-TE and GRE/
>    IP-multicast in order to allow the same encapsulations to be used for
>    unicast and multicast traffic as well as facilitating migration from
>    [I-D.rosen-vpn-mcast] to an MPLS label based encapsulation.
> 
> **** Given that there is no specification requiring support for all these
> **** encapsulations in unicast, it hardly makes sense to require support for
> **** all of them in multicast.  Anyway, I'd say it is unreasonable to
> **** REQUIRE support for every possible tunnel encapsulation (except L2TPv3,
> **** of course ;-)) just because for each encaps there is someone who wants
> **** it.

Yes, this is really an aspect of things where IETF specs are often
silent (e.g. for unicast 2547).  

We changed the wording to only say that "mVPN specs should recommend
implementations to support for multicast the P2MP variants of the
encapsulations and signaling protocols they support for unicast" and
then merely mention the most canonical examples.



> All three of the above encapsulation techniques support the building
>    of P2MP multicast trees.  In addition mLDP and GRE/IP-ASM-Multicast
>    implementations may also support the building of MP2MP multicast
>    trees.  The use of MP2MP trees may provide some scaling benefits to
>    the service provider as only a single MP2MP tree need be deployed per
>    VPN, thus reducing the amount of multicast state that needs to be
>    maintained by P routers.  This gain in state is at the expect of
>    bandwidth optimization, since sites that do not have multicast
>    receivers for multicast streams sourced behind a said PE group will
>    still receive packets of such streams, leading to non-optimal
>    bandwidth utilization across the VPN core. 
> 
> **** The use of MP2MP LSPs is perfectly compatible with the use of S-PMSIs
> **** for optimizing the routing of individual C-flows.  The claim that they
> **** lead to non-optimal bandwidth utilization thus seems to be completely
> **** unsupported.

I think you didn't read our text as closely enough.
What we do say is that MP2MP offers a way to have one tree *per VPN*
(not *per VPN per PE*), and that this does have a scaling advantage, but
will in the general case use more bandwidth.

You maybe had your -profile proposal in mind using one MP2MP tree per
VPN per upstream PE.  This proposal doesn't provide the benefit of "only
one tunnel per VPN". (Moreover as said before, that proposal is out of
the scope of our document.)


> One thing to consider is
>    that the use of MP2MP multicast tree will require configuring the
>    same tree identifier or multicast ASM group address in all PEs, and
>    will not provide the kind of autoconfiguration possible with P2MP
>    trees.
> 
> **** If you think that it would be better not to have to configure the ASM
> **** group identifier at all the PEs, this can easily be avoided.  You
> **** really only have to configure it at one PE, and all the other PEs can
> **** auto-discover it.  This requires only a minor change to the mvpn-bgp
> **** spec.

Good point.
But it would be nice to have this fully described so that we can
properly point to some reference, and understand the exact details and
implications (which PE decides, what happens when/if it goes down, etc.)

In the meantime, we have added a sentence suggesting that this could be
avoided.

> **** I'm not sure how important that is though, you're configuring all the
> **** PEs with the RTs for the MVPN, what's the marginal difficulty in
> **** configuring them with a group identifier as well?
> 
> **** The text above seems like an example of stretching to try to find
> **** something bad to say [...]

No, we don't imply to put that much weight on that argument. We will
rephrase to make this clear.  This is only *some* additional work...
Operators co-authoring the draft have insisted that this can still be a
significant effort though.


> **** [...] about a scheme which is really the technically
> **** superior scheme.

I think that, to say the least, not everyone is convinced that MP2MP or
ASM is "the technically superior scheme".  If it was, maybe operators
writing this draft (and others) would express such a view !



> MVPN services can also be supported over a unicast VPN core through
>    the use of ingress PE replication whereby the ingress PE replicates
>    any multicast traffic over the P2P tunnels used to support unicast
>    traffic.  While this option does not require the service provider to
>    modify their existing P routers (in terms of protocol support) and
>    does not require maintaining multicast-specific state on the P
>    routers in order for the service provider to be able deploy a
>    multicast VPN service, the use of ingress PE replication obviously
>    leads to non-optimal bandwidth utilization and it is therefore
>    unlikely to be the long term solution chosen by service providers.
>    However ingress PE replication may be useful during some migration
>    scenarios or where a service provider considers the level of
>    multicast traffic on their network to be too low to justify deploying
>    multicast specific support within their VPN core.
> 
> **** The spec also has a strange combination of ingress replication with
> **** multicast trees, whereby one unicast tunnels the packets to the root of
> **** a shared tree, and then decapsulates them and distributes them down the
> **** tree.  Senders then get their own transmissions back!  I wonder why the
> **** authors have not asked for this partially specified and not very useful
> **** technique to be removed from the spec.

Well, remember that our point is not to *remove* anything from the
current architecture draft...

In some cases we suggest that the readability of the draft could be
improved by setting aside some options. As you suggest, that is a
possibility for the option that you mention, but we just happen not to
focus on it.



> 3.5.  Inter-AS deployments options
> 
> ...
> 
>    The segmented inter-AS solution would appear to offer the largest
>    degree of deployment flexibility to operators, however the non-
>    segmented inter-AS solution can simplify deployment in a restricted
>    number of scenarios and [I-D.rosen-vpn-mcast] only supports the non-
>    segmented inter-AS solution and therefore the non-segmented inter-AS
>    solution is likely to be required by some operators for backward
>    compatibility and during migration from [I-D.rosen-vpn-mcast] to
>    [I-D.ietf-l3vpn-2547bis-mcast].
> 
>    The applicability of segmented or non-segmented inter-AS tunnels to a
>    given deployment or inter-provider interconnect will depend on a
>    number of factors specific to each service provider.  However, due to
>    the additional deployment flexibility offered by segmented inter-AS
>    tunnels, it is the recommendation of the authors that all
>    implementations should support the segmented inter-AS model.
>    Additionally, the authors recommend that implementations should
>    consider supporting the non-segmented inter-AS model in order to
>    facilitate co-existence with existing deployments, and as a feature
>    to provide a lighter engineering in a restricted set of scenarios,
>    although it is recognized that initial implementations may only
>    support one or the other.
> 
>    Additionally, the authors note that the proposed BGP-based approaches
>    for S-PMSI signaling and C-multicast routing information distribution
>    provide a good fit with both segmented and non-segmented inter-AS
>    tunnels.  In contrast the UDP-TLV based approach for S-PMSI signaling
>    appears to be incompatible with segmented inter-AS tunnels, and it is
>    unclear if the proposed PIM-based approaches for C-multicast routing
>    information distribution would be fully applicable to segmented
>    inter-AS tunnels.
> 
> **** I don't see any reason why one couldn't instantiate an MI-PMSI service
> **** with segmented inter-AS trees, so I don't see why there is any
> **** incompatibility between the segmented inter-AS trees and the various
> **** non-BGP approaches to signaling.


( Segmented inter-AS S-PMSI signaled with UDP is not described
anywhere ; which is the reason for our error.  )

But, while UDP-sig/segmented-tunnels is possible, it doesn't have one of
the advantage of the segmented-tunnels which allows different ASes to
use distinct P-tunnels flavors.  

Same for PIM : it may be compatible, but the scalability advantage is
lost (PEs in an AS see the C-multicast routing activity of all other PEs
of all other ASes).

We have included these ideas in the document, in the sections about
C-multicast routing and S-PMSI signaling.


> **** While segmented inter-AS trees do have the advantages cited, there are
> **** some scalability concerns.  The S-PMSIs are individuated on a
> **** per-root-PE basis, not on a per-root-AS basis.  To make efficient use
> **** of these you have to have aggregation via upstream-assigned labels at
> **** the border routers.  Of course, upstream-assigned labels are not a
> **** required feature.  The border routers need to be aware of each MVPN
> **** that passes traffic through them.  While unsegmented trees are not a
> **** particularly good solution in multi-provider scenarios, it is far from
> **** clear that the segmented inter-AS trees provide a good solution either.
> **** This is an area where I think more study and more review would be
> **** appropriate.

Saying that "there are scalability concerns with S-PMSIs with segmented
inter-AS trees" is totally misleading : you may require some features
for the scalability benefits of inter-AS trees to fully apply to
S-PMSIs, *but* segmented inter-AS will always scale *at least as well*
as non-segmented, and in most cases better.

Given that you know very well about that, I find your choice of wording
very suspect ("scalability concerns", "far from clear", etc.). 

Note that I didn't use the "FUD" acronym. ;)


> 6.  Summary of recommendations
> 
>    The following list summarizes the authors' recommendations.  These
>    recommendations are not intended to prevent the implementation of
>    alternative solutions, rather they are the authors' recommendations
>    for the mechanisms that should be made mandatory in
>    [I-D.ietf-l3vpn-2547bis-mcast] and therefore be supported by all
>    implementations.
> 
>    It is the authors' recommendation:
> 
>    o  that BGP-based auto-discovery be the mandated solution for auto-
>       discovery ;
> 
>    o  that BGP be the mandated solution for S-PMSI signaling ;
> 
>    o  that the mandated solution for S-PMSI switch-over be the mechanism
>       based on the source-connected PE switching traffic from the I-PMSI
>       tunnel to the S-PMSI tunnel, without transmitting traffic on both
>       at the time ;
> 
>    o  that implementations support both the BGP-based and the full per-
>       MPVN PIM peering solutions for PE-PE transmission of customer
>       multicast routing until further operational experience is gained
>       with both solutions ;
> 
> **** I really don't think this provides a reasonable basis for requiring
> **** support for BGP-based C-multicast routing.  If one wants to require
> **** support for something, the alternative that is already deployed
> **** and known to work is the one that should be mandated, and the
> **** alternative that is still experimental should be made optional.


You now seem to challenge object to something you explicitly agreed to
in December.

Quoted below, Eric Rosen, December 12th, 2007 (l3vpn WG mailing list):
> 
> "Consequently, at the present time and until there is experience with
> all of the proposed mechanisms it is not clear which of the above
> mechanisms should be recommended as the preferred solution to
> implementers.  However, it would appear prudent for implementations to
> consider supporting both the fourth (BGP-based) and first (full
> per-MPVN PIM peering) mechanisms.  Further experience on both
> implementation is likely to be required before some best practice can
> be defined."
>         
>         I certainly agree with this  conclusion.  Perhaps it should be
>         given more prominence. 




> **** As a wise man once said, "the proof of the pudding is in the eating,
> **** not in the debate about the pudding." ;-)

This is very true. Some providers and vendors are actually cooking and
eating that pudding...  Do you prefer eating your pudding cold ?
(or serving cold pudding maybe ?)
(or, if I may say, do you focus on some MP2MP-only pudding recipe ?)
;-)


> o  that implementations support the following multicast tree
>       encapsulations: mLDP, P2MP RSVP-TE and GRE/IP-Multicast ;
> 
>    o  that implementations support segmented inter-AS tunnels and
>       consider supporting non-segmented inter-AS tunnels (in order to
>       maintain backwards compatibility and for migration) ;
> 
> **** I really do like the segmented inter-AS tunnel idea, and in theory it
> **** is a very good thing, but of course we don't know how it will pan out
> **** in practice, and there are scalability concerns.

(see explanation above, talking about "scalability concerns for the
segmented inter-AS model" just doesn't make sense)

> **** I don't think this is ready to be a "must implement" yet.


If you think so, please state where and how, and let's see how to
progress that in the document where these are specified (which you
co-author!). It seems clear to me that this approach is key to inter-AS
mVPN scalability, and that "SHOULD" would be appropriate.