From rudolfvanderberg@gmail.com Sat Feb 17 13:53:53 2024 Return-Path: X-Original-To: e-impact@ietfa.amsl.com Delivered-To: e-impact@ietfa.amsl.com Received: from localhost (localhost [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4C32FC14F5ED for ; Sat, 17 Feb 2024 13:53:53 -0800 (PST) X-Virus-Scanned: amavisd-new at amsl.com X-Spam-Flag: NO X-Spam-Score: -7.104 X-Spam-Level: X-Spam-Status: No, score=-7.104 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001, HTML_MESSAGE=0.001, RCVD_IN_DNSWL_HI=-5, RCVD_IN_ZEN_BLOCKED_OPENDNS=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01, URIBL_BLOCKED=0.001, URIBL_DBL_BLOCKED_OPENDNS=0.001, URIBL_ZEN_BLOCKED_OPENDNS=0.001] autolearn=ham autolearn_force=no Authentication-Results: ietfa.amsl.com (amavisd-new); dkim=pass (2048-bit key) header.d=gmail.com Received: from mail.ietf.org ([50.223.129.194]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id NtlxTqYsy0At for ; Sat, 17 Feb 2024 13:53:48 -0800 (PST) Received: from mail-wm1-x334.google.com (mail-wm1-x334.google.com [IPv6:2a00:1450:4864:20::334]) (using TLSv1.3 with cipher TLS_AES_128_GCM_SHA256 (128/128 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by ietfa.amsl.com (Postfix) with ESMTPS id 7B228C14F5E7 for ; Sat, 17 Feb 2024 13:53:48 -0800 (PST) Received: by mail-wm1-x334.google.com with SMTP id 5b1f17b1804b1-412610fb006so495785e9.1 for ; Sat, 17 Feb 2024 13:53:48 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1708206826; x=1708811626; darn=ietf.org; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=+/9zZf3PWz4wdTflfXjlt0YLXyLn9Rzm7acF1Dc2xRg=; b=W/+yaCODzcQqUxhov/h3JihHHzlJveaqzp86ZN9qWRWuLmtD43rHR3KJs/YEQOywjs pSW0Rp2eBNdvagaksgLcWthd05Yip+spICP9a+969r+2Hu1XIANzSYx5HZzqVCSAM+kg 3LeatRBODUq2vjhUtlnyjNL+7UjyW7Q8pMfeCiAEbwAyrnZRzZMEsM5HMdHBZN8v6mW1 gNVajDa76rFfi7E8IcFYAQO82cl/TSXyPgFRke1km4/NxFRROWJQ2jXQ8iAkkHMTdW6o gVXrjSaQxatrPgFl/7XWdYF3PU1npMuitwcRVSpQ6P1cV1odEtfS8UpPgHSn6r2BIioY rMUA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1708206826; x=1708811626; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=+/9zZf3PWz4wdTflfXjlt0YLXyLn9Rzm7acF1Dc2xRg=; b=lPXt4To3dlafWetOhcuoC4iECPD5a3E9eOJ6GgTXMpE6cAdx6SAUAtQPEjr4Cq4ubK diSGf9sg4QbWhqIIngShfArP0WOHDpZQKcZAjTYhDibQ/q3KLfl4KVxtYLE4VsN/8zKC 6yKWl1m702m2WR1jLnmjFGV/s/yOxEsFJcxdB9y1xChSz/2Ah3KauwF9/AiozKXClNRB 2PxMQStol/LbzpH0kqh2WSTQQsB2CY1QWa2alXyuKdlNu6fAC7RbIBAdYQqogvhWuwmq U9gtmL95gU5ZRpwxQKKeKU3cqbjnAndcvQraDy/BBh01j+EhzRfzVV/ofAX9WVAVsR0I RgZA== X-Forwarded-Encrypted: i=1; AJvYcCXLpUhsVOFgpviEE+JmrZe9NaQELfgR1KH2utqB6NAZNMfC+ebSIO8+O4HzSRVFZZD5ftu4GggYGT9ro9GUOZ4RIA== X-Gm-Message-State: AOJu0Yx8p3HFKdXTAHuGMS/6mL15BizWW3LzhVzx4FyD5BkebpNh5sIL F1rzwhAKg9vVrm9NWgA3ADSw2zlOKWWIVbtoeFJA2HVsnS34L+PulcNxKWLtkZtvTseMmjwHHpV /v1DJJPjB/SlCbaT6n6XgFyaHjYx3rcTt X-Google-Smtp-Source: AGHT+IEVEXMmdSnBs18RT/xmoVu/PIsgjyslFqoO7BT1K7M2R6LU/40W0xmETy6SRYgEo3GwRV/xqHi+Yv5SSIgdRAY= X-Received: by 2002:a05:600c:19c7:b0:412:5296:9737 with SMTP id u7-20020a05600c19c700b0041252969737mr4229843wmq.12.1708206826130; Sat, 17 Feb 2024 13:53:46 -0800 (PST) MIME-Version: 1.0 References: <459643D6-2B47-4980-BA8D-88CABCE76982@ifi.uio.no> In-Reply-To: From: Rudolf van der Berg Date: Sat, 17 Feb 2024 22:53:33 +0100 Message-ID: To: Hesham ElBakoury Cc: Michael Welzl , Eve Schooler , E-Impact IETF Content-Type: multipart/alternative; boundary="00000000000013129706119ae570" Archived-At: Subject: [E-impact] Why carbon aware routing would break the Internet and emit more carbon Re: Carbon aware routing X-BeenThere: e-impact@ietf.org X-Mailman-Version: 2.1.39 Precedence: list List-Id: Environmental impacts of the Internet List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , X-List-Received-Date: Sat, 17 Feb 2024 21:53:53 -0000 --00000000000013129706119ae570 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable The very interesting paper on carbon aware routing by Sawsan yesterday, of which I only caught the last part of the presentation (sorry) inspired me to write a more general analysis of carbon aware routing, with some reference to the paper, but also a more fundamental critique of the idea of carbon aware routing. *Why carbon aware routing would break the Internet and emit more carbon * *Routing isn't computer science, it is economics* The science behind routing protocols is not computer science, it is economics. There are thousands of actors with varying levels of dependency upon each other. This is true within one network, but infinitely more so between thousands of networks. These actors are all boundedly rational and have varying levels of information on the state of their own and other's networks in the past and at this moment. To make things more complicated, their actions influence each other, can be mutually incompatible or mutually beneficial. They all have different assumptions, what caused their own actions and how this influenced the actions of others in the past and in the future. To completely mess things up, even if all data points and assumption were fully true until that point, that would not allow them to predict the future state of the network and routing of traffic (not even a second in the future!). It would be impossible to know whether their decision is "the most optimal", because routing is part of a chaotic system, called "the World". The state of the network and the traffic flowing through it, is not a closed system. It is influenced by everything that happens outside the network and that can be a lot, varying from war, natural disasters and pestilence to human error, Taylor Swift and people helping each other in times of need. That doesn't mean that the system is completely random, but it can't be relied on to be stable and it changes have to be expected and dealt with at any time. So the best routing system is the one that requires the least complete knowledge of the state of the network and can handle changes to that state at great speed and with great resilience. The routing of traffic within one network is determined by whoever is in control of that network. No two networks need to be the same, because the preferences and needs of those who control it may be different. Beautiful words can be spoken, about aligning the operation of the network and routing of traffic with lofty goals such as carbon awareness, customer needs etc. The reality is that there are many constraints and influences that will make the alignment of the daily operation of the network with those beautiful words and lofty goals a matter of opinion and not fact. There is therefore no most optimal way of designing a network or routing traffic over it. Each network is different and therefore operates differently. Interconnection of network makes it even more complex to operate a network optimally. When it comes to cross domain routing concepts such as path dependent (SCION), carbon based, cost based, QoS based routing were all tried in the 1980s and 1990s, failed and surpassed by BGP4. All those earlier and later failed routing protocols made the same flawed assumption as carbon free routing does today; if I know the state of the network, I can choose the most optimal route for me and the whole world. In the late eighties most routing protocols required that the state of the network was known and couldn't change too much. This almost broke the Internet. Fortunately advances allowed BGP to be developed. It requires remarkably little knowledge of the state of other networks. It can't see the difference between an Autonomous System Number of BT vs JANET or Google. It doesn't know what routers they use, what bandwidth there is, what physical distance there is etc. It knows how to get from one ASN to another ASN. And despite not knowing what a router, capacity, a telco, content provider, customer, supplier, carbon, money or a country border is, it works rather well. It turns out all those other variables may be very important to individuals who operate one network, but aren't relevant to inter-domain routing. A basic explanation I wrote of the economic transactions that take place,can be found at. https://arstechnica.com/features/2008/09/peering-and-transit/ (This model is different and the opposite of the more often cited, but flawed, two-sided models of interconnection of Laffont, Marcus, Rey and Tirole) *Carbon aware routing in one network needs an accurate model of the electricity network * I'm quite sceptical of carbon aware routing within one network and even more so when it happens across networks. For it to work, the routing algorithm would need a full understanding of how the exhaust of carbon is the result of electricity generation. The paper that was presented does not have such a model as part of the CATE algorithm. The paper says that national and European electricity markets are interconnected, but the algorithm doesn't derive the consequence from it: That the electricity network in Scotland is to a large extent the same one as in England and is interconnected with the electricity networks of EU countries. The total generation is related to the total consumption in the UK and that apart from losses in transport, import and export and congestion, the total electricity consumption of the UK is relevant and not the location. Routing traffic through Scotland doesn't make the wind blow faster in Scotland. It doesn't use less electricity in the UK. Most low carbon and zero carbon electricity sources aren't variable with demand. Their yield is driven by the intensity of the wind, water and the sun. Green electricity therefore has a priority over fossil fuel based electricity. Fossil fuel is the back-up for what can't be generated by renewable sources. (there are some renewable sources that may handle short peaks, eg water for TV-pick up in the UK) So shutting down an optic from London to Leeds may save electricity. This good, because it means a bit less electricity is needed in total and so a little bit less non-renewable fuel needs to be burnt. However, it will not lead to less carbon emitted, if shutting down a link to Leeds means that for traffic to Middlesborough a link from Manchester to Edinburgh and then to Middlesborough needs to be activated. The extra hop and longer distance for Edinburgh would ctually increase the total electricity consumption of the UK. This would require more fossil fuel to be burned. Scotland's green energy is a fixed quantity. Scotland doesn't become less green when traffic is routed through England. The UK does become less green if more electricity is needed in total to route traffic through Scotland. So a carbon aware routing protocol can only work if the model of the electricity network is complete and accurate for both how electricity is generated, consumed and distributed. *What if Scotland was independent from England, Wales and Northern Ireland, would that be better?* Theoretically it could be possible that the electricity grid of Scotland is completely independent of England, Wales and Northern Ireland. In that case the electricity consumption and the carbon footprint would be independent metrics. Scotland would need to have an over supplied and 100% green grid, completely independent of England, with no interdependence and no interconnection. If at the same time England has a network that still does emit CO2 in significant quantities to generate electricity, then there might be a form of arbitrage possible. In this case it might be possible that linking parts of BT's network in England, Wales and Northern Ireland through Scotland results in less carbon emissions by the combined electricity networks of Scotland, England, Wales and Northern Ireland. *Carbon aware routing in one network requires an accurate power profile of each device in the network* In the paper the authors assume that routers have a linear energy consumption profile above a base load. This is an academic oversimplification that is unusable in practice. Routers combine and split incoming data flows so that they become outgoing data flows. The complexity of a router is that incoming data flows and outgoing data flows are completely different from each other in size, distribution and compilation, though in aggregate they should be exactly the same. As a massive intersection of roads that should have no traffic jams or crashes. As a router reaches the maximum capacity of its ports and backplane its energy consumption is likely to increase in a non-linear fashion, heating it up more and requiring more cooling. At the same time it increasingly runs against its limits. Though the likes of Google have shown that they can get close to the technical limit of their networking equipment in the datacenter network for their services, it is very clear that this is a non-trivial task, that required a lot of upfront effort into knowing the behaviour of all their equipment and of the applications that run over it. https://cloud.google.com/blog/topics/systems/the-evolution-of-googles-jupit= er-data-center-network In the case of a telecom operator such as BT, the electricity consumption of the network is less deterministic. BT has a large number of very different types of users and usages. It has different agreements with those users. The equipment is at best of 2 different generations; the previous one and the one it is moving to. Its energy consumption is in part determined by physical distances, but also by weather! Certainly now climate change makes it warmer and sunnier in the UK too! The network changes every hour of every day, due to wear, tear, upgrades, downgrades, cable breaks, equipment failure etc. All of this isn't known to BT at any given moment and can take a while to percolate through to configuration databases at which point the state of the network is already different. What the CATE algorithm tries to do therefore falls far short of what would be needed, because it makes linear assumptions and assumes a stable state of BTs network, while not correctly incorporating the operation of the electricity network. *Carbon aware routing across independent networks will break the Internet* Routing based on BGP makes use of Autonomous Systems and how many hops there are between the source ASN and the target ASN. Less hops is better. BGP doesn't make assumptions about what an ASN represents. BGP doesn't know anything about how many routers, how much capacity, how many miles or how much traffic is in each ASN. Between the more than 75K ASNs there are continuous changes and BGP will announce them and algorithms will process them. It is a dumb, but scalable system, with a remarkably efficient and scalable network of interconnected networks as an emergent property. Carbon awareness would require that the routing algorithm actually knows and understands the electricity network that each piece of equipment in the entire Internet. Everything that should have been done in the BT network for the CATE algorithm to have reliable results, but then for every autonomous system and all the networks comprising those ASNs. What would really break the Internet however is that if carbon is used as a metric to guide routing decisions, than certain paths will always emerge as the most optimal from a carbon perspective. This is clear from the lament of the authors about the Geant network relying so much on Germany for its connectivity and Germany having such high carbon emissions. If carbon aware routing was a thing on the Internet, all traffic would want to evade networks that run through Germany (or not if they actually understand that Europe has an interconnected electricity market). What would happen is that traffic would seek the route with the lowest carbon emissions and all traffic would be sent to that route, without consideration for capacity, costs etc. That route would be overloaded, of which it would have to alert other networks, but those wouldn't be able to determine which network can and which network shouldn't send its traffic over the saturated, but lowest carbon, link. A similar issue was relayed to me by ISOC staff after the great earthquake in Japan, where most links had broken and therefore limited capacity was available. Japanese telecom firms had sold more "guaranteed" capacity than there was capacity available in the affected area. The result was that the routers wouldn't know what network had priority and therefore transmitted no traffic at all, despite total traffic being less than what little capacity was left. After the earthquake businesses were closed and most people were either in shelters hiding or doing practical things for the recovery and therefore the network was less in use. Carbon, QoS and other such metrics are of little use in routing across autonomous systems. Each network is different and has its own operations. The success of the Internet is that BGP doesn't have to factor it all in. *Electricity aware networking not carbon aware routing * The paper does show a more practical approach limiting the negative impact of use of resources by networks. By looking at the electricity consumption of links and interfaces and the amount of traffic, there can be a reason to turn off certain interfaces and equipment during moments of low use. Even this is non-trivial, because of non-linear effects, redundancy and the ever changing state of the network and the performance of equipment. Looking at operational electricity consumption is probably difficult enough for network staff to deal with. Embedded carbon of new versus old equipment might be too complex already. To illustrate, a network may supply connectivity to a customer through 2 times 2x100Gbps on a redundant route between two towns. The peaks exceed 100Gbps on the link, but not by much and traffic growth can be limited. 2 times 400Gbps will then use less electricity at any moment. This can be calculated and demonstrated. The embedded carbon of earlier replacement of interfaces, combined with the network being able to use 2 ports for another customer are much more complex, particularly when the 100Gbps is used elsewhere in the network. *Carbon aware placement of datacenters* Transport losses, congestion and other examples of inefficiency and scarcity in real world electric networks might make it preferable to build sites that consume a lot of electricity close to where the generation is greenest. That however applies to BT's peers and not to BT itself. BT is a nationwide network. Its electricity consumption primarily follows where people live and work. That typically is England and not Scotland. So placing a government or hyperscale datacenter in Glasgow might be preferred over Slough from a carbon aware electricity generation point of view. Transmitting photons over optical fibre uses less electricity than the losses of the electricity network over the same distance. *Conclusion* The impact of networking on the planet is significant. Every attempt should be made to decrease the impact whether it is from the operation of the network or building, upgrading and replacing it. Using less electricity in networking is good, because it means that less electricity needs to be generated, whether it is through sustainable or unsustainable generation. Making routing decisions depend on assumed carbon emissions is however not going to lead to less energy consumption, less carbon emessions or better routing and networking. It will likely break the Internet and emit more carbon. Better awareness of electricity consumption and the lifetime of equipment is what the internet community can work on. Op za 17 feb 2024 16:57 schreef Hesham ElBakoury : > Actually this Seyedali et al paper (from Adrian SCION group) is reference= d > by Salwa et al paper (from Noa group). > The comment made by Salwa's paper says that SCION paper "*takes advantage > of the SCION architecture in the context of path-aware networks. It deriv= es > an estimated* > *carbon footprint for every inter-domain path and then, end-domains choos= e > the paths with the least emissions. This should not be confused with the > network-wide carbon optimization problem that removes any overlapping > between end-domains*". > > There are many papers about green routing and green traffic engineering. > As Salwa pointed out in her presentation, each has its own limitations. > > Hesham > > On Sat, Feb 17, 2024, 6:21 AM Michael Welzl wrote: > >> =E2=80=A6 and then there was also this: >> https://netsec.ethz.ch/publications/papers/green_routing2023.pdf >> >> Cheers, >> Michael >> >> >> >> On Feb 17, 2024, at 1:37 AM, Eve Schooler wrote= : >> >> Thanks Hesham! >> The paper first appeared in HotCarbon'22, and was re-published along wit= h >> many other papers from that workshop in the ACM SIGEnergy Energy >> Informatics Review this past Oct 2023. >> --Eve >> >> On Fri, Feb 16, 2024 at 3:50=E2=80=AFPM Hesham ElBakoury >> wrote: >> >>> I also recall that Noa and Eve had a paper on carbon aware networking >>> https://dl.acm.org/doi/10.1145/3630614.3630618. It is also a good >>> paper. >>> >>> It is worth noting that section 8.6 of Sawdan and Noa paper provides a >>> comparison with the state of the art. "In summary, this paper looked at >>> practical current considerations, different to assumptions in previous >>> works. It estimated similar carbon savings while accounting for more >>> fine-grained carbon >>> intensity data, technical operating considerations of routers, without >>> assumptions on additional renewable energy deployments." >>> >>> Hesham >>> >>> On Fri, Feb 16, 2024, 12:47 PM Rudolf van der Berg < >>> rudolfvanderberg@gmail.com> wrote: >>> >>>> >>>> https://ora.ox.ac.uk/objects/uuid:504d0a87-6742-414f-b2c2-be74f2e5b579= /files/sc821gm64z >>>> >>>> Op vr 16 feb 2024 20:30 schreef Chris Adams < >>>> chris@thegreenwebfoundation.org>: >>>> >>>>> Hi folks, >>>>> >>>>> > We had good presentation today on carbon aware routing. More detail= s >>>>>> on >>>>>> > carbon aware routing are provided in this paper: >>>>>> > https://dl.acm.org/doi/10.1145/3629165 >>>>>> >>>>> >>>>> I=E2=80=99m not part of an institution with access to that paper at t= hat url - >>>>> would a kind soul point me to somewhere I can download it, or share a= copy >>>>> with me? I=E2=80=99m very interested in reading it. >>>>> >>>>> Thanks in advance. >>>>> >>>>> Chris Adams >>>>> >>>>> Executive Director >>>>> >>>>> w: thegreenwebfoundation.org >>>>> e: chris@thegreenwebfoundation.org >>>>> t: @mrchrisadams >>>>> >>>>> German Office >>>>> Naunynstrasse 40 >>>>> 10999 Berlin >>>>> Germany >>>>> >>>>> See our contact page for more details >>>>> https://www.thegreenwebfoundation.org/contact/ >>>>> >>>>> Book a short call with me to discuss something. >>>>> https://cal.com/mrchrisadams >>>>> >>>>> >>>>> On 16. Feb 2024, at 20:03, Dirk Trossen >>>> 40huawei.com@dmarc.ietf.org> wrote: >>>>> >>>>> CATS, although compute aware, could incorporate energy metrics in it= s >>>>> approach to traffic steering. >>>>> >>>>> In fact, we had discussed previously in CATS work that I published i= n >>>>> IFIP Networking 2021 that utilised a cardinal based WFQ at ingress po= ints >>>>> to dynamically steer traffic. We're currently looking into the right >>>>> utility function for energy to replace the compute units we used back= in >>>>> 2021. >>>>> >>>>> Dirk >>>>> >>>>> >>>>> >>>>> >>>>> *From:*Hesham ElBakoury >>>>> *To:*Toerless Eckert >>>>> *Cc:*E-Impact IETF >>>>> *Date:*2024-02-16 19:03:35 >>>>> *Subject:*Re: [E-impact] Carbon aware routing >>>>> >>>>> Tvr is my preference as we discussed upon tvr inception. But I think >>>>> in today's meeting IRTF was mentioned as the venue for this discussio= n. >>>>> >>>>> Hesham >>>>> >>>>> On Fri, Feb 16, 2024, 9:57 AM Toerless Eckert wrote: >>>>> >>>>>> IRTF not sure, but IETF TVR: >>>>>> >>>>>> >From charter: >>>>>> >>>>>> "Similarly, network traffic might be routed based on energy costs or >>>>>> expected user data volumes, which may vary predictably over time in >>>>>> networks prioritizing green computing and energy efficiency." >>>>>> >>>>>> Cheers >>>>>> Toerless >>>>>> >>>>>> On Fri, Feb 16, 2024 at 09:49:03AM -0800, Hesham ElBakoury wrote: >>>>>> > We had good presentation today on carbon aware routing. More >>>>>> details on >>>>>> > carbon aware routing are provided in this paper: >>>>>> > https://dl.acm.org/doi/10.1145/3629165 >>>>>> > >>>>>> > Which IRTF group is suitable to discuss this topic More? >>>>>> > >>>>>> > Thanks >>>>>> > Hesham >>>>>> >>>>>> > -- >>>>>> > E-impact mailing list >>>>>> > E-impact@ietf.org >>>>>> > https://www.ietf.org/mailman/listinfo/e-impact >>>>>> >>>>> -- >>>>> E-impact mailing list >>>>> E-impact@ietf.org >>>>> https://www.ietf.org/mailman/listinfo/e-impact >>>>> >>>>> >>>>> -- >>>>> E-impact mailing list >>>>> E-impact@ietf.org >>>>> https://www.ietf.org/mailman/listinfo/e-impact >>>>> >>>> -- >>> E-impact mailing list >>> E-impact@ietf.org >>> https://www.ietf.org/mailman/listinfo/e-impact >>> >> -- >> E-impact mailing list >> E-impact@ietf.org >> https://www.ietf.org/mailman/listinfo/e-impact >> >> >> -- > E-impact mailing list > E-impact@ietf.org > https://www.ietf.org/mailman/listinfo/e-impact > --00000000000013129706119ae570 Content-Type: text/html; charset="UTF-8" Content-Transfer-Encoding: quoted-printable
The very interesting paper on carbon aware routing by= Sawsan yesterday, of which I only caught the last part of the presentation= (sorry) inspired me to write a more general analysis of carbon aware routi= ng, with some reference to the paper, but also a more fundamental critique = of the idea of carbon aware routing.=C2=A0

= Why carbon aware routing would break the=C2=A0Internet and emit more car= bon=C2=A0

Routing isn't computer= science, it is economics
The science behind routing protocol= s is not computer science, it is economics. There are thousands of actors w= ith varying levels of dependency upon each other. This is true within one n= etwork, but infinitely more so between thousands of networks. These actors = are all boundedly rational and have varying levels of information on the st= ate of their own and other's networks in the past and at this moment. T= o make things more complicated, their actions influence each other, can be = mutually incompatible or mutually beneficial. They all have different assum= ptions, what caused their own actions and how this influenced the actions o= f others in the past and in the future. To completely mess things up, even = if all data points and assumption were fully true until that point, that wo= uld not allow them to predict the future state of the network and routing o= f traffic (not even a second in the future!). It would be impossible to kno= w whether their=C2=A0decision is "the most optimal", because rout= ing is part of a chaotic=C2=A0system, called "the World". The sta= te of the network and the traffic flowing through it, is not a closed=C2=A0= system. It is influenced by everything that happens outside the network and= that can be a lot,=C2=A0varying from war,=C2=A0natural disasters and pesti= lence to human=C2=A0error, Taylor Swift and people helping each=C2=A0other = in times of need. That doesn't mean that the system is completely rando= m, but it can't be relied on to be stable and it changes have to be exp= ected and dealt with at any time. So the best routing=C2=A0system is the on= e that requires the least complete knowledge of the state of the network an= d can handle changes to that state at great speed and with great resilience= .

The routing of traffic within one network is=C2= =A0 determined by whoever is in control of that network. No two networks ne= ed to be the same, because the preferences and needs of those who control i= t may be different. Beautiful words can be spoken, about aligning the opera= tion of the network and routing of traffic with lofty goals such as carbon = awareness, customer needs etc. The reality is that there are many constrain= ts and influences that will make the alignment of the daily operation of th= e network with those beautiful words and lofty goals a matter of opinion an= d not fact. There is therefore no most optimal way of designing a network o= r routing traffic over it. Each network is different and therefore operates= differently. Interconnection of network makes it even more complex to oper= ate a network optimally.=C2=A0

When it comes to cr= oss domain routing concepts such as path dependent (SCION),=C2=A0 carbon ba= sed, cost based, QoS based routing were all tried in the 1980s and 1990s, f= ailed and=C2=A0 surpassed by BGP4. All those earlier and later failed routi= ng protocols made the same flawed assumption as carbon free routing does to= day; if I know the state of the network, I can choose the most optimal rout= e for me and the whole world. In the late eighties most routing protocols r= equired that the state of the network was known and couldn't change too= much. This almost broke the Internet. Fortunately advances allowed BGP to = be developed. It requires remarkably little knowledge of the state of other= networks. It can't see the difference between an Autonomous System Num= ber of BT vs JANET or Google. It doesn't know what routers they use, wh= at bandwidth there is, what physical distance there is etc. It knows how to= get from one ASN to another ASN. And despite not knowing what a router, ca= pacity, a telco, content provider, customer, supplier, carbon, money or a c= ountry border is, it works rather well. It turns out all those other variab= les may be very important to individuals who operate one network, but aren&= #39;t relevant to inter-domain routing. A basic explanation I wrote of the = economic transactions that take place,can be found at. https://arstechnica.c= om/features/2008/09/peering-and-transit/ (This model is different and t= he opposite of the more often cited, but flawed, two-sided models of interc= onnection of Laffont, Marcus, Rey and Tirole)=C2=A0

Carbon aware routing in one network needs an=C2=A0acc= urate model of the electricity network=C2=A0
I&#= 39;m quite sceptical of carbon aware routing within one network and even mo= re so when it happens across networks. For it to work, the routing algorith= m would need a full understanding of how the exhaust of carbon is the resul= t of electricity generation. The paper that was presented does not have suc= h a model as part of the CATE algorithm. The paper says that national and E= uropean electricity markets are interconnected,=C2=A0but the algorithm does= n't=C2=A0 derive the=C2=A0consequence from it: That the electricity net= work in=C2=A0Scotland is to a=C2=A0large extent the same one as in England = and is interconnected with the electricity networks of EU countries. The to= tal generation is related to the total consumption in the UK and that apart= from losses in transport, import and export and=C2=A0 congestion, the tota= l electricity consumption of the UK is relevant and not the location.=C2=A0=

=C2=A0Routing traffic t= hrough Scotland doesn't make the wind blow faster in Scotland. It doesn= 't use less electricity in the UK.=C2=A0 Most low carbon and zero carbo= n electricity sources aren't variable with demand. Their=C2=A0yield is = driven by the intensity of the wind, water and the sun. Green electricity t= herefore has a priority over fossil fuel based electricity. Fossil fuel is = the back-up for what can't be generated by renewable sources. (there ar= e some renewable sources that may handle short peaks, eg water=C2=A0for TV-= pick up in the UK) So shutting down an optic from London to Leeds may save = electricity. This good, because it means a bit less electricity is needed i= n total and so a=C2=A0little bit less non-renewable fuel needs to be burnt.= However, it will not lead to less carbon emitted, if shutting down a link = to Leeds means that for traffic to Middlesborough a link from Manchester to= Edinburgh and then to Middlesborough needs to be activated. The extra hop = and longer distance for Edinburgh would ctually increase the total electric= ity consumption of the UK. This would require more fossil fuel to be=C2=A0b= urned. Scotland's green energy is a fixed quantity. Scotland doesn'= t become less green when traffic is routed through England. The UK does bec= ome less green if more electricity is needed in total to route traffic thro= ugh Scotland. So a carbon aware routing protocol can only work if the model= of the electricity network is complete and accurate for both how electrici= ty is generated, consumed and distributed.=C2=A0
What if Scotland was independent from=C2=A0 England, Wales a= nd Northern Ireland, would that be better?
Theoretically it c= ould be possible that the electricity grid of Scotland is completely indepe= ndent of England, Wales and Northern Ireland. In that case the electricity = consumption and the carbon footprint would be independent metrics. Scotland= would need to have an over supplied and 100% green grid,=C2=A0 completely = independent of England, with no interdependence and no interconnection. If = at the same time England has a network that still does emit CO2 in signific= ant quantities to generate electricity, then there might be a form of arbit= rage possible. In this=C2=A0case it might be possible that linking parts of= BT's network in England, Wales and Northern Ireland through Scotland r= esults in less carbon emissions by the combined electricity networks of Sco= tland,=C2=A0England, Wales and Northern Ireland.=C2=A0=C2=A0

=
Carbon aware routing in one network requires an accurate powe= r profile of each device in the network
In the paper the auth= ors assume that routers have a linear energy consumption profile above a ba= se load. This is an academic oversimplification that is unusable in practic= e. Routers combine and split incoming data flows so that they become outgoi= ng data flows. The complexity of a router is that incoming data flows and o= utgoing data flows are completely different from each other in size, distri= bution and compilation, though in aggregate they should be exactly the same= . As a massive intersection of roads that should have no traffic jams or cr= ashes. As a router reaches the maximum capacity of its ports and backplane = its energy consumption is likely to increase in a non-linear fashion, heati= ng it up more and requiring more cooling. At the same time it increasingly = runs against its limits. Though the likes of Google have shown that they ca= n get close to the technical limit of their networking equipment in the dat= acenter network for their services, it is very clear that this is a non-tri= vial task, that required a lot=C2=A0of upfront effort into knowing the beha= viour of all their equipment and of the applications that run over it.=C2= =A0https://cloud.google.com/blog/topics/= systems/the-evolution-of-googles-jupiter-data-center-network
=
In the case of a telecom operator such as BT, the electricit= y consumption of the network is less deterministic. BT has a large number o= f very different types of users and usages. It has different agreements wit= h those users. The equipment is at best of 2 different generations; the pre= vious one and the one it is moving to. Its energy consumption is in part de= termined by physical distances, but also by weather! Certainly now climate = change makes it warmer and sunnier in the UK too! The network changes every= hour of every day, due to wear, tear, upgrades, downgrades, cable breaks, = equipment failure etc. All of this isn't known to BT at any given momen= t and can take a while to percolate through to configuration databases at w= hich point the state of the network is already different.=C2=A0
<= br>
What the CATE algorithm tries to do therefore falls far short= of what would be needed, because it makes linear assumptions and assumes a= stable state of BTs network, while not correctly incorporating the operati= on of the electricity network.=C2=A0

Carbon awa= re routing across independent networks will break the Internet
Routing based on BGP makes use of Autonomous Systems and how many hops th= ere are between the source ASN and the target ASN. Less hops is better. BGP= doesn't make assumptions about what an ASN represents. BGP doesn't= know anything about how many routers, how much capacity, how many miles or= how much traffic is in each ASN. Between the more than 75K ASNs there are = continuous changes and BGP will announce them and algorithms will process t= hem. It is a dumb, but scalable system, with a remarkably efficient and sca= lable network of interconnected networks as an emergent property.=C2=A0

Carbon awareness would require that the routing algor= ithm actually knows and understands the electricity network that each piece= of equipment in the entire Internet. Everything that should have been done= in the BT network for the CATE algorithm to have reliable results, but the= n for every autonomous system=C2=A0and all the networks comprising those AS= Ns.=C2=A0

What would really break the Internet how= ever is that if carbon is used as a metric to guide routing decisions, than= certain paths will always emerge as the most optimal from a carbon perspec= tive. This is clear from the lament of the authors about the Geant network = relying so much on Germany for its connectivity and Germany having such hig= h carbon emissions. If carbon aware routing was a thing on the Internet, al= l traffic would want to evade networks that run through Germany (or not if = they actually understand that Europe has an interconnected electricity mark= et). What would happen is that traffic would seek the route with the lowest= carbon emissions and all traffic would be sent to that route, without cons= ideration for capacity, costs etc. That route would be overloaded, of which= it would have to alert other networks, but those wouldn't be able to d= etermine which network can and which network shouldn't send its traffic= over the saturated, but lowest carbon, link.=C2=A0

A similar issue was relayed to me by ISOC staff after the great earthquak= e in Japan, where most links had broken and therefore limited capacity was = available. Japanese telecom firms had sold more "guaranteed" capa= city than there was capacity available in the affected area. The result was= that the routers wouldn't know what network had priority and therefore= transmitted no traffic at all, despite total traffic being less than what = little capacity was left. After the earthquake businesses were=C2=A0closed = and most people were either in shelters hiding or doing practical things fo= r the recovery and therefore the network was less in use.=C2=A0
<= i>
Carbon, QoS and other such metrics are of little use i= n routing across autonomous systems. Each network is different and has its = own operations. The success of the Internet is that BGP doesn't have to= factor it all in.=C2=A0

Elec= tricity aware networking not carbon aware routing=C2=A0
The paper does show a more practical approach limiting the negative impact= of use of resources by networks. By looking at the electricity consumption= of links and interfaces and the amount of traffic, there can be a reason t= o turn off certain interfaces and equipment during moments of low use. Even= this is non-trivial, because of non-linear effects, redundancy and the eve= r changing state of the network and the performance of equipment. Looking a= t operational electricity consumption is probably difficult enough for netw= ork staff to deal with. Embedded carbon of new versus old equipment might b= e too complex already. To illustrate, a network may supply connectivity to = a customer through 2 times 2x100Gbps on a redundant route between two towns= . The peaks exceed 100Gbps on the link,=C2=A0but not by much and traffic gr= owth can be limited. 2 times 400Gbps will then use less electricity at any = moment. This can be calculated and demonstrated. The embedded carbon of ear= lier replacement of interfaces, combined with the network being able to use= 2 ports for another customer are much more complex, particularly when the = 100Gbps is used elsewhere in the network.=C2=A0=C2=A0

=
Carbon aware placement of datacenters
Transport losses, congestion and ot= her examples of inefficiency and scarcity in real world electric networks m= ight make it preferable to build sites that consume a lot of electricity cl= ose to where the generation is greenest. That however applies to BT's p= eers and not to BT itself. BT is a nationwide network. Its electricity cons= umption primarily follows where people live and work. That typically is Eng= land and not Scotland. So placing a government or hyperscale datacenter in = Glasgow might be preferred over Slough from a carbon aware electricity gene= ration point of view. Transmitting photons over optical fibre uses less ele= ctricity than the losses of the electricity network over the same distance.= =C2=A0

Conclusion
Th= e impact of networking on the planet is significant. Every attempt should b= e made to decrease the impact whether it is from the operation of the netwo= rk or building, upgrading and replacing it. Using less electricity in netwo= rking is good, because it means that less electricity needs to be generated= , whether it is through sustainable or unsustainable generation. Making rou= ting decisions depend on assumed carbon emissions is however not going to l= ead to less energy consumption, less carbon emessions or better routing and= networking. It will likely break the Internet and emit more carbon. Better= awareness of electricity consumption and the lifetime of equipment is what= the internet community can work on.=C2=A0



Op za 17 feb 202= 4 16:57 schreef Hesham ElBakoury <helbakoury@gmail.com>:
Actually this Seyedali et al paper (from A= drian SCION group) is referenced by Salwa et al paper (from Noa group).=C2= =A0
The comment made by Salwa's paper says that SCION = paper "takes advantage of the SCION architecture in the context of = path-aware networks. It derives an estimated
= carbon footprint for every inter-domain path and then, end-domains choose t= he paths with the least emissions. This should not be confused with the net= work-wide carbon optimization problem that removes any overlapping between = end-domains".

T= here are many papers about green routing and green traffic engineering. As = Salwa pointed out in her presentation, each has its own limitations.=C2=A0<= /div>

Hesham

On Sat, Feb = 17, 2024, 6:21 AM Michael Welzl <michawe@ifi.uio.no> wrote:
=E2=80=A6 and then there was also this:

Cheers,
Michael



On Feb 17, 2024, at 1:37 AM, Eve Schooler <eve.schooler@gmail.com> wrote:

Thanks Hesham!=C2=A0
The paper first appeared in HotCarbon'22, an= d was re-published along with many other papers from that workshop in the A= CM SIGEnergy Energy Informatics Review this past Oct 2023.
--Eve<= /div>

On Fri, Feb 16, 2024 at 3:50=E2=80=AFPM Hesham ElBakoury <helbakoury@gmail.com> wrote:
I also recall= that Noa and Eve had a paper on carbon aware networking
<= a href=3D"https://dl.acm.org/doi/10.1145/3630614.3630618" rel=3D"noreferrer= noreferrer noreferrer noreferrer noreferrer noreferrer" target=3D"_blank">= https://dl.acm.org/doi/10.1145/3630614.3630618. It is also a good paper= .=C2=A0

It is worth noti= ng that section 8.6 of Sawdan and Noa paper provides a comparison with the = state of the art. "In summary, this paper looked at practical current = considerations, different to assumptions in previous works. It estimated si= milar carbon savings while accounting for more fine-grained carbon=C2=A0
=C2=A0intensity data, technical oper= ating considerations of routers, without assumptions on additional renewabl= e energy deployments."

Hesham

On Fri, Feb 16, 2024, 12:47 PM Rudolf van der Berg &l= t;rudolfvand= erberg@gmail.com> wrote:

Op vr 16 feb 2024 20:30 schreef Chris Adams &= lt;chris@thegreenwebfoundation.org>:
Hi folks,=C2=A0

> We had go= od presentation today on carbon aware routing. More details on
> carb= on aware routing are provided in this paper:
>=C2=A0https://dl.acm.org/doi/10.1145/3629165

I=E2=80=99m not part= of an institution with access to that paper at that url - would a kind sou= l point me to somewhere I can download it, or share a copy with me? I=E2=80= =99m very interested in reading it.

Thanks in adva= nce.

Chris Adams

Executive Director

w:=C2=A0thegree= nwebfoundation.org
e:=C2=A0chris@thegreenwebfoundatio= n.org
t: @mrchrisadams

German Office
Naunynstrasse 40
1= 0999 Berlin
Germany

See our contact page for more details
https://www.thegreenwebfoundation.org/contact/

Book a short call with me to discuss=C2=A0something.
https= ://cal.com/mrchrisadams


On 16. Feb 2024, at 20:03, Dirk Tro= ssen <dirk.trossen=3D40huawei.com@dmarc.ietf.org> wro= te:

CATS, although compute aware,= =C2=A0 could incorporate energy metrics in its approach to traffic steering= .=C2=A0

In fact,=C2=A0 we had discussed previously= in CATS work that I published in IFIP Networking 2021 that utilised a card= inal based WFQ at ingress points to dynamically steer traffic. We're cu= rrently looking into the right utility function for energy to replace the c= ompute units we used back in 2021.=C2=A0

Dirk
<= br>


From:Hesham ElBakoury <= ;helbakoury@gmail.com>
= To:Toerless Eckert <tte@cs.fau.de>
Cc:E-Impact IETF <e-impact@ietf.org>
=
Date:2024-02-16 19:03:35
Subject:Re: [E-impact] Carbon awar= e routing

Tvr i= s my preference as we discussed upon tvr inception. But I think in today= 9;s meeting IRTF was mentioned as the venue for this discussion.

Hesham

On Fri, Feb 16, 2024, 9:57 A= M Toerless Eckert <tte@cs.fau.de> wrote:
IRTF not sure, but IETF TVR:

>Fr= om charter:

"Similarly, network traffic might be routed based o= n energy costs or expected user data volumes, which may vary predictably ov= er time in networks prioritizing green computing and energy efficiency.&quo= t;

Cheers
=C2=A0 =C2=A0 Toerless

On Fri, Feb 16, 2024 at 0= 9:49:03AM -0800, Hesham ElBakoury wrote:
> We had good presentation t= oday on carbon aware routing. More details on
> carbon aware routing = are provided in this paper:
>=C2=A0https://dl.acm.org/doi/10.1145/3629165
>=C2= =A0
> Which IRTF group is suitable to discuss this topic More?=
>=C2=A0
> Thanks
> Hesham

> --=C2=A0
> E-impact mailing list
>=C2=A0E-impact@ietf.org
>=C2=A0https://www.ietf.org/mailman/listinfo/e-impac= t
E-impact mailing list
E-impact@ietf.org
https://www.ietf= .org/mailman/listinfo/e-impact

--
E-impact mailing list
E-impact@ietf.org
https://www.ietf.org/mailman/listinfo/e-i= mpact
--
E-impact mailing list
E-impact@ietf.org
https://www.ietf.org/mailman/lis= tinfo/e-impact
--
E-impact mailing list
E-impact@ietf.org
https://www.ietf.org/mailman/listinfo/e-impact=

--
E-impact mailing list
E= -impact@ietf.org
https://www.ietf.org/mailman/listinfo/e-imp= act
--00000000000013129706119ae570--