Re: [6tsch] architecture with remote BBR

["Pascal Thubert (pthubert)" <[e-mail]>]

Hello Thomas:

When the broadcast domain is split inside an IP subnet, there must be a fashion to transport ND/ARP, and at the same time, we need to avoid the risk of propagating a broadcast storm across the overlay.

So the solutions require some learning of what is on each side and some protocol to share that over the pseudowire. I see a lot of work in that area, but certainly the registration model could help in that the registrar could talk to one another over the tunnel and then setup forwarding states in a rapid and deterministic fashion (I mean here that the registrar state is a solid state from an active registration as opposed to a snooped information).

All this has yet to settle, and we need to maintain a prudent observation. In particular we can observe how TRILL, LISP  and NVO3 http://datatracker.ietf.org/wg/nvo3/charter/.

Cheers,

Pascal

From: [e-mail] [mailto:[e-mail]] On Behalf Of Thomas Watteyne
Sent: vendredi 21 juin 2013 02:02
To: Pascal Thubert (pthubert)
Cc: 6TSCH
Subject: Re: [6tsch] architecture with remote BBR

Pascal,

You need the BB to be a MAC-level broadcast domain to be able to do the proxy ND functionality, right? (see min 17:02 of https://cisco.webex.com/ciscosales/lsr.php?AT=pb&SP=MC&rID=66409322&rKey=254105c68fefa880). But you're absolutely right, I can see how this property affects the PCE discovery functionality.

The case where there are many little PCEs covering different segment makes perfect sense. In other cases (mobility?), we might imagine an architecture where a single PCE is managing multiple LLNs in parallel.

Thomas



On Thu, Jun 20, 2013 at 7:00 AM, Pascal Thubert (pthubert) <[e-mail]<mailto:[e-mail]>> wrote:
Hello Thomas:

If I understand you well, your reference to the multicast domain has to do specifically with the PCE discovery, is that correct?
Even so, the product of the discussion is probably useful in the architecture document.

My initial mind was to emulate the model of DHCP, though we probably need to extend it quite a bit to scale and distribute.
For all I know - but I'm not JP -, there is art for distributed PCE, in particular between ISPs so that each PCE can compute a local segment of a global route.
In your case 1, it could make sense that there is at least one PCE on each subdomain so as to avoid crossing boundaries for local route computation.

In the future, we can probably expect to find many virtualized PCE available wherever needed via virtual service engines anyway.

What do you think?

Pascal

From: [e-mail]<mailto:[e-mail]> [mailto:[e-mail]<mailto:[e-mail]>] On Behalf Of Thomas Watteyne
Sent: mercredi 19 juin 2013 06:13
To: 6TSCH

Subject: Re: [6tsch] architecture with remote BBR

Pascal, Raghuram,

I'm fully aware that VPN/tunnels are a proven technique. Obviously, this discussion only covers the case where a PCE is used.

I want to make sure that we don't add unnecessary complexity. The easiest thing is for 6TSCH not to define how the PCE and the BBRs discover each other, neither how they communicate, and just give "examples" in the architecture draft. This might be the right thing to do to start with, but it would be awfully nice if I could connect my BBR to a third-party PCE. I'm happy to talk about this on the phone on Friday.

Thomas

On Tue, Jun 18, 2013 at 8:53 PM, Raghuram Sudhaakar (rsudhaak) <[e-mail]<mailto:[e-mail]>> wrote:
Thomas,
I agree conceptually with the cons that you mention for option 1. However, tunneling and VLANs are a well understood concept in the network setup/administration and used widely by IT teams. Pascal has pointed out the specific Cisco technologies too. So, in the practical world tunneling is the best/proven solution.

In Option 2, the reliance of the BBR on the PCE to identify its peer may be a cause for concern. It means that we implicitly mandate a PCE. This may not be applicable to certain deployments that may want to use 6TSCH without a PCE. Or a different routing computation technique/protocol/standard.

IMO, the PCE, ND must be maintained as separate elements for the above reasons as well as applicability to wider range of scenarios.

I lean toward the idea that 6TSCH does not need to define anything to create the connectivity between the BBRs. Instead the requirement can be detailed along with possible solutions leaving the decision open. This will hopefully help  in wider applicability and interoperability.

-raghuram


From: Pascal Thubert <[e-mail]<mailto:[e-mail]>>
Date: Tuesday, June 18, 2013 10:08 AM
To: Thomas Watteyne <[e-mail]<mailto:[e-mail]>>
Cc: 6TSCH <[e-mail]<mailto:[e-mail]>>
Subject: Re: [6tsch] architecture with remote BBR

Hello Thomas:
And then we need to add the case of the backhaul that looks like your case 1 but has applications on the other side of the VPN as opposed to another wlan.
This is actually being studied at ISA100.15 ...
For your option 1, is it often (/ sometimes?) mandatory that the 2 LLNs share a same L2 domain ( / IPv6 subnet) ? When they do not, we are back in classical routing, with VPN iff crossing an untrusted area ( ; eg the IT network from an OT perspective ; )
The case of a single subnet crossing layer 3 boundaries is very classical in datacenters. We use overlays to solve the issue; e.g. cisco OTV, but also LISP, L2TP, and pseudowires in general.
We should probably describe the case in the architecture and explain how this can be achieved with the above technologies; and that probably 6TSCH does not need to add anything new. Or does it?
About option 2, I see the links to the PCE as either a single vlan or a mix of vlan and vpn, depending on which domain must be crossed. The structure has its benefits, but we probably need to come up with the same model and multiple disjoint paths via multiple PCEs for high availability and load balancing.
what do you think?
Pascal

2013/6/17 Thomas Watteyne <[e-mail]<mailto:[e-mail]>>
All,

There is a case I believe we are not covering explicitly in the architecture.

The architecture draft now considers the following topology:


               ---+------------------------

                  |      External Network

                  |

               +-----+                  +-----+

               |     | Router           |     | PCE

               |     |                  |     |

               +-----+                  +-----+

                  |                        |

                  |     Subnet Backbone    |

            +--------------------+------------------+

            |                    |                  |

         +-----+             +-----+             +-----+

         |     | Backbone    |     | Backbone    |     | Backbone

    o    |     | router      |     | router      |     | router

         +-----+             +-----+             +-----+

    o                  o                   o                 o   o

        o    o   o         o   o  o   o         o  o   o    o

   o             o        o  LLN      o      o         o      o

      o   o    o      o      o o     o  o   o    o    o     o


The backbone needs to be one broadcast domain for the ND proxy operations defines in draft-thubert-6lowpan-backbone-router-03 to work.

Now, let's consider a campus-wide deployment, where the requirements is that (1) all the nodes use the same IPv6 prefix, and (2) all are managed by the same PCE. Since BBRs are "far apart", they will not all live on the same (broadcast) backbone.

This is a very realistic scenario that I have come across multiple times, and which I believe 6TSCH group could/should address.

I can see the following options:

Option 1: "under-the-hood" tunneling

When installing the network, network administrators interconnect the different pieces of the BB using some VLAN solution, essentially recreating a broadcast domain.

pros:
- This option does not require any change the ND operation.
cons:
- IMO, in most multi-BBR deployments, the remote BBR case is the rule rather than the exception. Using tunnels looks more like a "patch" which might be seen as overly complex if it needs to be applied all the time.



               ---+------------------------

                  |      External Network

                  |

               +-----+                  +-----+

               |     | Router           |     | PCE

               |     |                  |     |

               +-----+                  +-----+

                  |                        |

                  |     Subnet Backbone    |    ==========

            +--------------------+--------------  TUNNEL  ----+

            |                    |              ==========    |

         +-----+             +-----+                       +-----+ (remote)

         |     | Backbone    |     | Backbone              |     | Backbone

    o    |     | router      |     | router                |     | router

         +-----+             +-----+                       +-----+

    o                  o                   o                 o   o

        o    o   o         o   o  o   o                  o  o   o    o

   o             o        o  LLN      o      o             o      o

      o   o    o      o      o o     o  o   o             o    o     o


Option 2: PCE responsible for forwarding to correct BBR

Each BBR establishes an explicit (and secure) connection to the PCE. Since the PCE is aware of the nodes connected through each BBR, it can forward some inbound packet to the appropriate BBR.

The functionality of the PCE and Router can be merged. The PCE/Router gets a packet for a particular node, and forwards it to the appropriate BBR over the explicit connection to that BBR.




                --------------+-------------------

                              |  External Network

                              |

                           +-----+

                           |     | PCE/Router

                           |     |

                           +-----+

                            ^ ^ ^

                            | | |





            +---------------+ | +----------------+

            |                 |                  |

            v                 v                  v

         +-----+           +-----+            +-----+

         |     | Backbone  |     | Backbone   |     | Backbone

    o    |     | router    |     | router     |     | router

         +-----+           +-----+            +-----+

    o                  o                   o                 o   o

        o    o   o         o   o  o   o         o  o   o    o

   o             o        o  LLN      o      o         o      o

      o   o    o      o      o o     o  o   o    o    o     o



Option 3: hybrid



This is the same as option 2, but the router and the PCE as separate. The PCE acts as the ND proxy for all the nodes attached to all the BBRs it is managing. The router is a regular router.



  ---+-----------------------





     |   External Network

     |

  +-----+                  +-----+

  |     | Router           |     | PCE

  |     |               +--|     |

  +-----+               |  +-----+

     |                  |   ^ ^ ^





     |                  |   | | |

  ------------------------  | | |

                            | | |

                            | | |





            +---------------+ | +----------------+

            |                 |                  |

            v                 v                  v

         +-----+           +-----+            +-----+

         |     | Backbone  |     | Backbone   |     | Backbone

    o    |     | router    |     | router     |     | router

         +-----+           +-----+            +-----+

    o                  o                   o                 o   o

        o    o   o         o   o  o   o         o  o   o    o

   o             o        o  LLN      o      o         o      o

      o   o    o      o      o o     o  o   o    o    o     o





Thoughts?





Thomas

_______________________________________________
6tsch mailing list
[e-mail]<mailto:[e-mail]>
https://www.ietf.org/mailman/listinfo/6tsch



--
Pascal