Re: [E-impact] [OPSAWG] side meeting #119: Power Metrics: concrete usage example

[Michael Welzl <[e-mail]>]

Hi Hesham!

Answering below:

> On Mar 26, 2024, at 3:34 PM, Hesham ElBakoury <[e-mail]> wrote:
> 
> Good points to ponder Mike!
> 
> Another issue is related to the emerging machine learning applications. I think the reduction of network energy consumption and carbon emission that we can achieve in IETF by the instrumentations and tools you mentioned may prove to be insignificant compared to the huge amount of energy consumption and carbon emission of ML model (including LLM) training and inference.
> 
> Hugging Face researchers estimated that training BLOOM LLM training led to 25 metric tons of carbon dioxide emissions. The researchers found that this figure doubled when they took into account the emissions produced by the manufacturing of the computer equipment used for training, the broader computing infrastructure, and the energy required to actually run BLOOM once it was trained [1].
> 
> Researchers at the University of Massachusetts, Amherst, performed a life cycle assessment for training several common large AI models. They found that the process can emit more than 626,000 pounds of carbon dioxide which is equivalent to approximately five times the lifetime emissions of an average American car (includes manufacturing of the car itself) [2,3]

Up to here, I thought, that may well be, and these numbers are wild!  but why is this related to what we’re discussing?


> I think, yet again, we should consider the sociological issues. Our efforts to construct the most energy efficient machines possible may go in vain if there is profligate use for purposes that do not justify their use.

Aha, so this is akin to asking a group of road engineers: should we still build these roads, when they can be used by electrical cars and Diesel trucks alike?

The Internet is a general purpose infrastructure. It isn’t, and has never been, the role of the IETF to try to mandate what applications are running on top of it - for better or worse.
Hmmm. Perhaps that’s wrong (although “mandate” is surely too strong a word - we couldn’t if we wanted to). I briefly considered bringing in child pornography as an example here, but then realized that this is a tricky subject, as the IETF *has* taken a policy stance on “pervasive monitoring is an attack”, which probably has the downside of possibly getting in the way of *preventing* child pornography …   so it wouldn’t be completely right to say that we “just build the infrastructure”  - yes we do think about what it’s used for, and try to do what’s right, in general.

All this said, other than putting out a statement (“thou shalt not train AI until the planet melts, and if so, please set the evil bit in the IP header”), what could the IETF do?

Anyway, I think this point about LLMs is irrelevant to the discussion at hand: what could the IETF currently standardize, and is there something useful to standardize?

Cheers,
Michael



> 
> What do you think?
> 
> Hesham
> [1] Hugging Face’s work is published in a non-peer-reviewed paper <https://arxiv.org/pdf/2211.02001.pdf>. 
> [2] https://www.supermicro.com/en/article/ai-training-5-tips-reduce-environmental-impact
> [3] https://arxiv.org/abs/1906.02243
> 
> On Tue, Mar 26, 2024, 4:12 AM Michael Welzl <[e-mail] <mailto:[e-mail]>> wrote:
>> Hi everyone!
>> 
>> Many good points here. Just to bring one more view to the table:
>> 
>> I agree with George that it’s hard to see the actual "protocol work” that would help - but I think that this may just be because we’re used to think of algorithmic solutions.
>> 
>> E.g., what routing strategy is “better” - based on energy? based on carbon efficiency? based on a mix between these and “traditional” metrics?  (well of course - e.g. shortest path will never cease to be important, but how to mix these?)   People have already come up with example cases where these metrics *can* be useful. But how useful are they really, and how realistic is it to apply them at a larger scale? WOULD a larger scale be what we should aim for, or is this inherently limited-scope stuff?
>> 
>> It doesn’t seem like we have overall answers to these questions, and I don’t think we can expect to find them anytime soon; and should *we*, as engineers? Hesham has pointed at sociology and marketing; Rudolf has pointed at economics.
>> 
>> Concluding from all this, it seems to me that useful work can already be done in the IETF to *instrument* protocols to provide, or enable use of, sustainability-related information (at least energy, but perhaps also carbon efficiency or some such metric if we do have hope to come up with a useful definition…).  I.e., define a routing metric; offer management information (see the proposed YANG model and ICMP extension);  ... and then it would be up to others to see what they can do with it.
>> 
>> Here’s another way of looking at this: if we don’t instrument protocols with such information, nobody can even try to deploy any strategy because the tools are just not available - and nothing will happen.
>> 
>> So, IMO, such “instrumentation” work should indeed happen in the IETF, soon - and in addition, we should keep having this mailing list to inform the discussion on how the newly offered instruments could be brought to good use.
>> 
>> Cheers,
>> Michael
>> 
>> 
>>> On Mar 26, 2024, at 11:46 AM, Dom Robinson <[e-mail] <mailto:[e-mail]>> wrote:
>>> 
>>> 100%
>>> -- 
>>> Dom Robinson
>>> www.id3as.co.uk <http://www.id3as.co.uk/> 
>>> www.greeningofstreaming.org <http://www.greeningofstreaming.org/> 
>>> uk.linkedin.com/in/domrobinson <http://uk.linkedin.com/in/domrobinson>
>>> Meet >> https://calendly.com/id3as 
>>> 
>>>> On 26 Mar 2024, at 10:45, Dirk Trossen <[e-mail] <mailto:[e-mail]>> wrote:
>>>> 
>>>> Dom,
>>>>  
>>>> Yes, it is a slippery slope from dumb to aware to (artificially) intelligent 😉
>>>>  
>>>> But given that constraint-based routing already represents awareness (e.g., to latency), adding energy in a limited scope is not a big slide down that slope IMO. And we may find other opportunities on the dry end of that slope before slippery kicks in.
>>>>  
>>>> The reason for having this discussion in an engineering forum is to get the balance of views right between the crazy desires and the actual working things.
>>>>  
>>>> Dirk
>>>>  
>>>> From: Dom Robinson <[e-mail] <mailto:[e-mail]>> 
>>>> Sent: 26 March 2024 11:39
>>>> To: Dirk Trossen <[e-mail] <mailto:[e-mail]>>
>>>> Cc: George Michaelson <[e-mail] <mailto:[e-mail]>>; E-Impact IETF <[e-mail] <mailto:[e-mail]>>
>>>> Subject: Re: [E-impact] [OPSAWG] side meeting #119: Power Metrics: concrete usage example
>>>>  
>>>> Dirk
>>>>  
>>>> So I agree on being careful about the scope of energy considerations in protocol design (to avoid feature creep, to avoid raising new security considerations) but I cannot agree that it is not potentially substantive protocol work for the IETF.
>>>>  
>>>> More good points: better made than my own! 
>>>>  
>>>> I stand by giving the whole idea of giving the seemingly inexorable merge of energy operations with IP (as far as this group is concerned) a solid kick-test. Without that check and balance i do fear an unending scope creep in trying to bring ever-deeper decisioning / awareness into the routing layer. 
>>>>  
>>>> "Dumb-networks" have proven very successful for a long while and while it’s tempting to add (A)intelligence to everything these days it may take us toward using a particle accelerator to crack a nut :)
>>>>  
>>>> But i happily cede to the points you make about prior form for such ‘awareness' already being present as far as traffic flow is concerned.
>>>>  
>>>>  
>>>>  
>>>> -- 
>>>> Dom Robinson
>>>> www.id3as.co.uk <http://www.id3as.co.uk/> 
>>>> www.greeningofstreaming.org <http://www.greeningofstreaming.org/> 
>>>> uk.linkedin.com/in/domrobinson <http://uk.linkedin.com/in/domrobinson>
>>>> Meet >> https://calendly.com/id3as 
>>>> 
>>>> 
>>>> On 26 Mar 2024, at 08:41, Dirk Trossen <[e-mail] <mailto:[e-mail]>> wrote:
>>>>  
>>>> Dom, all,
>>>>  
>>>> Chiming in here as a passive observer so far on this list. 
>>>>  
>>>> When you say: “But should a data routing protocol confuse its purpose by routing based on its own ‘awareness’ of energy? I think that adds a burden to IP that would weaken it and be out of scope.” I wonder why you limit the scope of IP routing constraints (which do exist, right?) to NOT consider energy? 
>>>>  
>>>> Why is it that we cannot envision energy costs associated to links in the same manner we assign latency budgets to select one path over another? If one considers multi-constraint routing work (see https://www.semanticscholar.org/paper/Routing-on-Multiple-Optimality-Criteria-Sobrinho-Ferreira/8ce4650878576839d61d0be35a364e019c18318b for what is hopefully working as a non-paywall reference), even multiple constraints in combination with latency or bandwidth are foreseeable IMO. 
>>>>  
>>>> When you say: “l3 routing should be routing, and not try to scope creep into application / infrastructure. It should stick to the end to end principle and leave the ends to decide on energy actions, while facilitating the data exchange.”, it confuses me since I was under the impression that routing is exactly about infrastructure, while it is also about application requirements (highest BW or lowest latency, or maybe even lowest energy) to make the right data exchange decisions.
>>>>  
>>>> An example of work in the IETF, where we can see (down the line) an active consideration of energy is that of traffic steering, i.e., the selection of one of possibly several choices of network locations where traffic could go to. If you see several (e.g., virtualized) instances of a service residing at those different network locations, picking the ‘best’ of the choices is key here. Isn’t it a good thought to consider energy as a possible ‘best’ choice here?
>>>>  
>>>> The CATS WG is one such body of work in the IETF right now. Its very name limits it to ‘compute-awareness’ but isn’t the energy consumption of compute endpoints not one such possible awareness metric? Of course, works like CATS but also ALTO need to work out the signaling of such application/service metrics to network nodes to act upon them, but that’s (in part) why those WGs exist.
>>>>  
>>>> So I agree on being careful about the scope of energy considerations in protocol design (to avoid feature creep, to avoid raising new security considerations) but I cannot agree that it is not potentially substantive protocol work for the IETF.
>>>>  
>>>> Best,
>>>>  
>>>> Dirk
>>>>  
>>>> From: E-impact <[e-mail] <mailto:[e-mail]>> On Behalf Of Dom Robinson
>>>> Sent: 26 March 2024 07:11
>>>> To: George Michaelson <[e-mail] <mailto:[e-mail]>>
>>>> Cc: E-Impact IETF <[e-mail] <mailto:[e-mail]>>
>>>> Subject: Re: [E-impact] [OPSAWG] side meeting #119: Power Metrics: concrete usage example
>>>>  
>>>> George - you make some good points very well.
>>>>  
>>>> I am simply unconvinced there is substantive
>>>> protocol work in energy burden to be done in IETF protocol space at
>>>> this time.
>>>>  
>>>>  
>>>> I have to say I agree.
>>>>  
>>>> As an influencing discussion, e-impact is hugely valuable.
>>>>  
>>>> As an intermediary between layers 1/2 and layer 4 (where all the energy is consumed) IP can already carry information (as an ‘application’) that leads to energy saving infrastructure and workload management. In this regard it already does the job perfectly!
>>>>  
>>>> But should a data routing protocol confuse its purpose by routing based on its own ‘awareness’ of energy? I think that adds a burden to IP that would weaken it and be out of scope. 
>>>>  
>>>> It risks becoming ‘for the sake of it’ rather than because it is really improving the protocol IMHO. 
>>>>  
>>>> I think better to continue to focus on how the physical layers and link layers are efficient, and the application layer has information to steer decisions.
>>>>  
>>>> l3 routing should be routing, and not try to scope creep into application / infrastructure. It should stick to the end to end principle and leave the ends to decide on energy actions, while facilitating the data exchange.
>>>>  
>>>> IPs role is hugely important, but it needs to remain benign else it will become a political tool. Energy information becoming a component of routing data looks like a cyberattack waiting to happen to me. 
>>>>  
>>>> BGP tables going wrong causes chaos, but generally ‘only’ impact data access. A DDoS attack on our energy using IP networks could cause real-world problems with much deeper consequences than ‘some data loss’. 
>>>>  
>>>>  
>>>>  
>>>> -- 
>>>> Dom Robinson
>>>> www.id3as.co.uk <http://www.id3as.co.uk/> 
>>>> www.greeningofstreaming.org <http://www.greeningofstreaming.org/> 
>>>> uk.linkedin.com/in/domrobinson <http://uk.linkedin.com/in/domrobinson>
>>>> Meet >> https://calendly.com/id3as 
>>>> 
>>>> 
>>>> 
>>>> On 26 Mar 2024, at 02:31, George Michaelson <[e-mail] <mailto:[e-mail]>> wrote:
>>>> 
>>>> Without wanting to push too hard, I would personally push back on
>>>> E-impact becoming a WG or having any solidity beyond an experimental
>>>> ongoing activity. I am simply unconvinced there is substantive
>>>> protocol work in energy burden to be done in IETF protocol space at
>>>> this time.
>>>> 
>>>> * I absolutely believe there is an energy impact issue in DC, in long
>>>> distance communications and maintenance of RF carrier, for the IEEE,
>>>> for 3GPP, for economies with unreliable power facing the question to
>>>> house a CDN or long-line it to somewhere else. These problems lie in
>>>> other people's standards bodies.
>>>> 
>>>> * I absolutely do believe we can theorise physical and link layer
>>>> protocol changes which reduce energy burdens (in many cases with cost,
>>>> like no 0RTT responses because we have to re-establish link) -And that
>>>> things like delay tolerant networking or 6LOWPAN go into this space
>>>> sometimes.
>>>> 
>>>> * I also absolutely do believe there may in future be clearer drive to
>>>> do work on energy in IETF, in protocol design for DC or other contexts
>>>> where it has to be exposed through the protocol stack to the
>>>> application. Therefore I can say there COULD be work to be done here,
>>>> in the future.
>>>> 
>>>> About as far as I would go right now is a problem statement. And, a
>>>> watch-and-review status.
>>>> 
>>>> I'm not in charge here and I don't have some magic veto card. If
>>>> people wanted to continue, I wouldn't oppose a list being hosted, but
>>>> I would be uncomfortable with positions being taken in public implying
>>>> the IETF or the IAB have substantive work to do here: I don't see it.
>>>> 
>>>> Maybe the IAB differs.
>>>> 
>>>> -G
>>>> 
>>>> -- 
>>>> E-impact mailing list
>>>> [e-mail] <mailto:[e-mail]>
>>>> https://www.ietf.org/mailman/listinfo/e-impact
>>> 
>>> -- 
>>> E-impact mailing list
>>> [e-mail] <mailto:[e-mail]>
>>> https://www.ietf.org/mailman/listinfo/e-impact
>> 
>> -- 
>> E-impact mailing list
>> [e-mail] <mailto:[e-mail]>
>> https://www.ietf.org/mailman/listinfo/e-impact