Re: Power consumption reduction in edge and core routers in the internet

[Curtis Villamizar <[e-mail]>]

In message <[e-mail]>
Balaji Venkat writes:
 
> Dear Curtis,
>  
> We used nroffedit for preparing these drafts and forgot to delete the
> default references that got stuck in there.
>  
> We intended these to be serious material.
>  
> If you could comment on the specifics of why you disagree with our
> proposals it would be most beneficial.
>  
> Regards
> Balaji


Without too lengthy a response I have to say that I don't think your
approach is going to yield much if any benefit.

First reducing power in core and edge are very different problems.

There are a few things in the core.  For example:

  1.  Constraining the number of forwarding entries to get the
      existing external TCAM back into the forwarding chip and doing
      the forwarding with a less power greedy method than TCAM.

  2.  If possible eliminating the very large buffer on one side of a
      switching fabric, making it small enough to use on chip RAM and
      putting the bulk of the buffering on one side.  Some think
      putting enough overspeed in the fabric and putting large buffer
      on the output side may be a solution.  Others favor a VOQ
      approach.  The extreme memory bandwidth needed to go to external
      DRAM is the culprit.  As on chip SRAM sizes increase over time
      this may be possible.  Putting external DRAM on both sides of a
      fabric is a big power drain.

The first above is supported by using MPLS with a BGP free core (no IP
forwarding except for the small number of prefixes needed for control
traffic and then just a set of MPLS labels to traverse the topology.
Comments on LDP vs RSVP-TE to /dev/null please (not important here).
Comments on IPFRR are also not relevant as IPFRR doesn't address this.

For edge, forwarding speeds are much lower (about 3 decimal orders of
magnitude give or take one) and buffering requirements are smaller.
On chip buffering of one side is clearly feasible but needs to be done
right.  At the very low end, on chip buffering on both ends might be
feasible.  Access is further out than edge and on chip buffer should
be sufficient.  With IP, if global routing is needed, then an external
TCAM is generally used, though some chips will use external DRAM
(realizing that multiple accesses would be needed for a trie
approach).  I'm far less familiar with where state-of-the-art is for
edge and access.

For large networks the parallel links provide a further potential for
power reduction.  Generally some traffic must be run for monitoring
even if the link is unused.  However in principle on-chip power
islands could be used to eliminate the leakage current that unused
power forwarding engines would draw, while keeping the minimal
circuitry needed for an OAM or link layer equivalent active.  I don't
know any chip that does this.  Generally that would be taking
component links out of a parallel set of links (for example a wide LAG
of 100Gb/s links).

Powering down the laser is very problematic in long haul (LH).  The LH
optical systems monitor and balance power levels on the optical fiber.
Lighting up additional wavelengths on a WDM can take minutes before
power levels are adjusted right and transmit and receive sides and all
the optical amplifiers along the way are sufficiently in sync to bring
bit error rates down.  LH optics and modulation is a black art,
particularly ultra LH (ULH) and subsea (no OEO for up to 6000km) and
right now shutting lasers down can be disruptive.  The optical
amplifier pump lasers have to be powered at about the same levels
regardless of how many optical channels are carried.

This does not apply to edge and access.  Between edge and access, the
physical topology is often a set of logical rings over WDM with ROADM
add/drop.  Taking down one side of each ring would not be viable.  If
there is parallel links or other redundancy, then shutting down lasers
in short haul (less than 40 or less than 100 km) is feasible.

Moving "high touch" functions such as deep packet inspection and
filtering to edge helps.  For core where a provider peering might
require filtering, keeping the rule set very small can keep it on
chip.  Adding still more TCAM or DRAM for filtering drive power up.
If a card must be all things, then it would help to allow those
components to be powered down if not needed and the forwarding chip
SERDES facing them to be powered down if not needed (internal to core
for example, rather than peering).

So I really think that you haven't started by stating a valid problem
other than recognizing that reducing power would be a good thing.

Maybe you should start over with one draft stating the problem.
Identify where the power goes in a provider core network or content
provider data center (or other extremely large data center) or in a
service provider edge and access.  Is it forwarding logic?  Is it chip
externals such as forwarding TCAM or DRAM, buffer SERDES/DRAM, filter
TCAM or DRAM, switching fabric, etc.  What is the contribution of
optical components driving the line?  (hint: don't look here).

Then once you know where the power goes, try to fix it.  IMHO you'll
end up with a very different approach.

Well - I suppose that wasn't as concise as it could be.

Curtis


> On 09-Sep-2012, at 1:32 AM, Curtis Villamizar <[e-mail]> wrote:
>  
> > 
> > In message <[e-mail]>
> > balaji venkat Venkataswami writes:
> > 
> >> Dear all,
> >> 
> >> After due consultation with the chairs and IRTF folks we have been
> >> instructed to ask for discussion on the following drafts that deal with
> >> power consumption reduction in edge and core devices in the internet.
> >> 
> >> We deal with the problem in a hierarchical approach. One within an
> >> Autonomous System and the other between Autonomous Systems and hence for the
> >> whole internet.
> >> 
> >> We duly submit to you this work that we have done for your kind review and
> >> consideration.
> >> 
> >> We deal with solutions for unicast and multicast at the AS level and the
> >> intra-AS level. Multicast solutions mostly in the intra-AS level.
> >> 
> >> We submit this work and request you to comment on it.
> >> 
> >> Multicast intra-AS level
> >> * mjsraman-pce-power-replic
> >> <http://tools.ietf.org/html/draft-mjsraman-pce-power-replic>  (timeline)
> >> <http://www.arkko.com/tools/lifecycle/draft-mjsraman-pce-power-replic-timing
> >> .html> 
> >> * mjsraman-pim-ecmp-redirect-power-replic-cap
> >> <http://tools.ietf.org/html/draft-mjsraman-pim-ecmp-redirect-power-replic-ca
> >> p>  (timeline) 
> >> <http://www.arkko.com/tools/lifecycle/draft-mjsraman-pim-ecmp-redirect-power
> >> -replic-cap-timing.html>
> >> Unicast AS level
> >> * mjsraman-rtgwg-bgp-power-path
> >> <http://tools.ietf.org/html/draft-mjsraman-rtgwg-bgp-power-path>  (timeline)
> >> <http://www.arkko.com/tools/lifecycle/draft-mjsraman-rtgwg-bgp-power-path-ti
> >> ming.html> 
> >> * mjsraman-rtgwg-inter-as-psp
> >> <http://tools.ietf.org/html/draft-mjsraman-rtgwg-inter-as-psp>  (timeline)
> >> <http://www.arkko.com/tools/lifecycle/draft-mjsraman-rtgwg-inter-as-psp-timi
> >> ng.html> 
> >> * mjsraman-rtgwg-inter-as-psp
> >> <http://tools.ietf.org/html/draft-mjsraman-rtgwg-inter-as-psp>  (timeline)
> >> <http://www.arkko.com/tools/lifecycle/draft-mjsraman-rtgwg-inter-as-psp-timi
> >> ng.html> 
> >> * mjsraman-rtgwg-inter-as-psp-protect
> >> <http://tools.ietf.org/html/draft-mjsraman-rtgwg-inter-as-psp-protect>
> >> (timeline) 
> >> <http://www.arkko.com/tools/lifecycle/draft-mjsraman-rtgwg-inter-as-psp-prot
> >> ect-timing.html> 
> >> Unicast intra-AS level
> >> * mjsraman-rtgwg-intra-as-psp-te-leak
> >> <http://tools.ietf.org/html/draft-mjsraman-rtgwg-intra-as-psp-te-leak>
> >> (timeline) 
> >> <http://www.arkko.com/tools/lifecycle/draft-mjsraman-rtgwg-intra-as-psp-te-l
> >> eak-timing.html> 
> >> * mjsraman-rtgwg-ospf-power-topo
> >> <http://tools.ietf.org/html/draft-mjsraman-rtgwg-ospf-power-topo>
> >> (timeline) 
> >> <http://www.arkko.com/tools/lifecycle/draft-mjsraman-rtgwg-ospf-power-topo-t
> >> iming.html> 
> >> Multicast intra-AS level
> >> * mjsraman-rtgwg-pim-power
> >> <http://tools.ietf.org/html/draft-mjsraman-rtgwg-pim-power>  (timeline)
> >> <http://www.arkko.com/tools/lifecycle/draft-mjsraman-rtgwg-pim-power-timing.
> >> html> 
> >> 
> >> Thanks again to the WG and IETF chairs for letting us share our work with
> >> this group.
> >> 
> >> Thanks and regards
> >> Shankar and team
> > 
> > 
> > My first reaction was that this was just a really bad idea from some
> > students who didn't understand very much about how provider networks
> > were built.
> > 
> > But looking at some of the citations I'm wondering if this whole thing
> > is just a joke.
> > 
> >  draft-mjsraman-rtgwg-ospf-power-topo-01
> > 
> >   [EVILBIT]  Bellovin, S., "The Security Flag in the IPv4 Header",
> >              RFC 3514, April 1 2003.
> > 
> >   [RFC5513]  Farrel, A., "IANA Considerations for Three Letter
> >              Acronyms", RFC 5513, April 1 2009.
> > 
> >   [RFC5514] Vyncke, E., "IPv6 over Social Networks", RFC 5514, April
> >             1 2009.
> > 
> > I'm really hoping the whole thing is a joke, but unfortunately I do
> > think that you are seriously proposing this.  Just confirming first.
> > 
> > Curtis