Re: [E-impact] [OPSAWG] side meeting #119: Power Metrics: concrete usage example

[Dirk Trossen <[e-mail]>]

Hmm, is it really about mandating use of LLM training or about reducing the energy consumption of its use? That's how I read Hesham's email, at least initially,  but maybe the sociological reference pointed that way?

We can happily discuss the uses and misuses of Internet tech (crypto comes to my mind here as another fingerpointing) or simply go about developing tech that makes the use (or misuse, if you see it that way) more efficient. As engineers,  that may be the best use of our (work) time.

As for relevance of LLM to the discussion: I recall a draft to the transport area on collective communication and the objective to improve its efficiency (e.g. avoiding headend replication). It's to me an example of what the IETF could do in this LLM space in relation to sustainability.

Similar for crypto in providing alternatives to its inefficient iterative diffusion,  utilizing multicast instead (we had a draft on quantifying the inefficiency and are working on improving the situation).

Dirk






From:Michael Welzl <[e-mail]<mailto:[e-mail]>>
To:Hesham ElBakoury <[e-mail]<mailto:[e-mail]>>
Cc:Dom Robinson <[e-mail]<mailto:[e-mail]>>;Dirk Trossen <[e-mail]<mailto:[e-mail]>>;George Michaelson <[e-mail]<mailto:[e-mail]>>;E-Impact IETF <[e-mail]<mailto:[e-mail]>>
Date:2024-03-26 16:55:35
Subject:Re: [E-impact] [OPSAWG] side meeting #119: Power Metrics: concrete usage example

Hi Hesham!

Answering below:

On Mar 26, 2024, at 3:34 PM, Hesham ElBakoury <[e-mail]> wrote:

Good points to ponder Mike!

Another issue is related to the emerging machine learning applications. I think the reduction of network energy consumption and carbon emission that we can achieve in IETF by the instrumentations and tools you mentioned may prove to be insignificant compared to the huge amount of energy consumption and carbon emission of ML model (including LLM) training and inference.

Hugging Face researchers estimated that training BLOOM LLM training led to 25 metric tons of carbon dioxide emissions. The researchers found that this figure doubled when they took into account the emissions produced by the manufacturing of the computer equipment used for training, the broader computing infrastructure, and the energy required to actually run BLOOM once it was trained [1].

Researchers at the University of Massachusetts, Amherst, performed a life cycle assessment for training several common large AI models. They found that the process can emit more than 626,000 pounds of carbon dioxide which is equivalent to approximately five times the lifetime emissions of an average American car (includes manufacturing of the car itself) [2,3]

Up to here, I thought, that may well be, and these numbers are wild!  but why is this related to what we’re discussing?


I think, yet again, we should consider the sociological issues. Our efforts to construct the most energy efficient machines possible may go in vain if there is profligate use for purposes that do not justify their use.

Aha, so this is akin to asking a group of road engineers: should we still build these roads, when they can be used by electrical cars and Diesel trucks alike?

The Internet is a general purpose infrastructure. It isn’t, and has never been, the role of the IETF to try to mandate what applications are running on top of it - for better or worse.
Hmmm. Perhaps that’s wrong (although “mandate” is surely too strong a word - we couldn’t if we wanted to). I briefly considered bringing in child pornography as an example here, but then realized that this is a tricky subject, as the IETF *has* taken a policy stance on “pervasive monitoring is an attack”, which probably has the downside of possibly getting in the way of *preventing* child pornography …   so it wouldn’t be completely right to say that we “just build the infrastructure”  - yes we do think about what it’s used for, and try to do what’s right, in general.

All this said, other than putting out a statement (“thou shalt not train AI until the planet melts, and if so, please set the evil bit in the IP header”), what could the IETF do?

Anyway, I think this point about LLMs is irrelevant to the discussion at hand: what could the IETF currently standardize, and is there something useful to standardize?

Cheers,
Michael




What do you think?

Hesham
[1] Hugging Face’s work is published in a non-peer-reviewed paper<https://arxiv.org/pdf/2211.02001.pdf>.
[2] https://www.supermicro.com/en/article/ai-training-5-tips-reduce-environmental-impact
[3] https://arxiv.org/abs/1906.02243

On Tue, Mar 26, 2024, 4:12 AM Michael Welzl <[e-mail]<mailto:[e-mail]>> wrote:
Hi everyone!

Many good points here. Just to bring one more view to the table:

I agree with George that it’s hard to see the actual "protocol work” that would help - but I think that this may just be because we’re used to think of algorithmic solutions.

E.g., what routing strategy is “better” - based on energy? based on carbon efficiency? based on a mix between these and “traditional” metrics?  (well of course - e.g. shortest path will never cease to be important, but how to mix these?)   People have already come up with example cases where these metrics *can* be useful. But how useful are they really, and how realistic is it to apply them at a larger scale? WOULD a larger scale be what we should aim for, or is this inherently limited-scope stuff?

It doesn’t seem like we have overall answers to these questions, and I don’t think we can expect to find them anytime soon; and should *we*, as engineers? Hesham has pointed at sociology and marketing; Rudolf has pointed at economics.

Concluding from all this, it seems to me that useful work can already be done in the IETF to *instrument* protocols to provide, or enable use of, sustainability-related information (at least energy, but perhaps also carbon efficiency or some such metric if we do have hope to come up with a useful definition…).  I.e., define a routing metric; offer management information (see the proposed YANG model and ICMP extension);  ... and then it would be up to others to see what they can do with it.

Here’s another way of looking at this: if we don’t instrument protocols with such information, nobody can even try to deploy any strategy because the tools are just not available - and nothing will happen.

So, IMO, such “instrumentation” work should indeed happen in the IETF, soon - and in addition, we should keep having this mailing list to inform the discussion on how the newly offered instruments could be brought to good use.

Cheers,
Michael


On Mar 26, 2024, at 11:46 AM, Dom Robinson <[e-mail]<mailto:[e-mail]>> wrote:

100%
--
Dom Robinson
www.id3as.co.uk<http://www.id3as.co.uk/>
www.greeningofstreaming.org<http://www.greeningofstreaming.org/>
uk.linkedin.com/in/domrobinson<http://uk.linkedin.com/in/domrobinson>
Meet >> https://calendly.com/id3as

On 26 Mar 2024, at 10:45, Dirk Trossen <[e-mail]<mailto:[e-mail]>> wrote:

Dom,

Yes, it is a slippery slope from dumb to aware to (artificially) intelligent 😉

But given that constraint-based routing already represents awareness (e.g., to latency), adding energy in a limited scope is not a big slide down that slope IMO. And we may find other opportunities on the dry end of that slope before slippery kicks in.

The reason for having this discussion in an engineering forum is to get the balance of views right between the crazy desires and the actual working things.

Dirk

From: Dom Robinson <[e-mail]<mailto:[e-mail]>>
Sent: 26 March 2024 11:39
To: Dirk Trossen <[e-mail]<mailto:[e-mail]>>
Cc: George Michaelson <[e-mail]<mailto:[e-mail]>>; E-Impact IETF <[e-mail]<mailto:[e-mail]>>
Subject: Re: [E-impact] [OPSAWG] side meeting #119: Power Metrics: concrete usage example

Dirk

So I agree on being careful about the scope of energy considerations in protocol design (to avoid feature creep, to avoid raising new security considerations) but I cannot agree that it is not potentially substantive protocol work for the IETF.

More good points: better made than my own!

I stand by giving the whole idea of giving the seemingly inexorable merge of energy operations with IP (as far as this group is concerned) a solid kick-test. Without that check and balance i do fear an unending scope creep in trying to bring ever-deeper decisioning / awareness into the routing layer.

"Dumb-networks" have proven very successful for a long while and while it’s tempting to add (A)intelligence to everything these days it may take us toward using a particle accelerator to crack a nut :)

But i happily cede to the points you make about prior form for such ‘awareness' already being present as far as traffic flow is concerned.



--
Dom Robinson
www.id3as.co.uk<http://www.id3as.co.uk/>
www.greeningofstreaming.org<http://www.greeningofstreaming.org/>
uk.linkedin.com/in/domrobinson<http://uk.linkedin.com/in/domrobinson>
Meet >> https://calendly.com/id3as


On 26 Mar 2024, at 08:41, Dirk Trossen <[e-mail]<mailto:[e-mail]>> wrote:

Dom, all,

Chiming in here as a passive observer so far on this list.

When you say: “But should a data routing protocol confuse its purpose by routing based on its own ‘awareness’ of energy? I think that adds a burden to IP that would weaken it and be out of scope.” I wonder why you limit the scope of IP routing constraints (which do exist, right?) to NOT consider energy?

Why is it that we cannot envision energy costs associated to links in the same manner we assign latency budgets to select one path over another? If one considers multi-constraint routing work (see https://www.semanticscholar.org/paper/Routing-on-Multiple-Optimality-Criteria-Sobrinho-Ferreira/8ce4650878576839d61d0be35a364e019c18318b for what is hopefully working as a non-paywall reference), even multiple constraints in combination with latency or bandwidth are foreseeable IMO.

When you say: “l3 routing should be routing, and not try to scope creep into application / infrastructure. It should stick to the end to end principle and leave the ends to decide on energy actions, while facilitating the data exchange.”, it confuses me since I was under the impression that routing is exactly about infrastructure, while it is also about application requirements (highest BW or lowest latency, or maybe even lowest energy) to make the right data exchange decisions.

An example of work in the IETF, where we can see (down the line) an active consideration of energy is that of traffic steering, i.e., the selection of one of possibly several choices of network locations where traffic could go to. If you see several (e.g., virtualized) instances of a service residing at those different network locations, picking the ‘best’ of the choices is key here. Isn’t it a good thought to consider energy as a possible ‘best’ choice here?

The CATS WG is one such body of work in the IETF right now. Its very name limits it to ‘compute-awareness’ but isn’t the energy consumption of compute endpoints not one such possible awareness metric? Of course, works like CATS but also ALTO need to work out the signaling of such application/service metrics to network nodes to act upon them, but that’s (in part) why those WGs exist.

So I agree on being careful about the scope of energy considerations in protocol design (to avoid feature creep, to avoid raising new security considerations) but I cannot agree that it is not potentially substantive protocol work for the IETF.

Best,

Dirk

From: E-impact <[e-mail]<mailto:[e-mail]>> On Behalf Of Dom Robinson
Sent: 26 March 2024 07:11
To: George Michaelson <[e-mail]<mailto:[e-mail]>>
Cc: E-Impact IETF <[e-mail]<mailto:[e-mail]>>
Subject: Re: [E-impact] [OPSAWG] side meeting #119: Power Metrics: concrete usage example

George - you make some good points very well.

I am simply unconvinced there is substantive
protocol work in energy burden to be done in IETF protocol space at
this time.


I have to say I agree.

As an influencing discussion, e-impact is hugely valuable.

As an intermediary between layers 1/2 and layer 4 (where all the energy is consumed) IP can already carry information (as an ‘application’) that leads to energy saving infrastructure and workload management. In this regard it already does the job perfectly!

But should a data routing protocol confuse its purpose by routing based on its own ‘awareness’ of energy? I think that adds a burden to IP that would weaken it and be out of scope.

It risks becoming ‘for the sake of it’ rather than because it is really improving the protocol IMHO.

I think better to continue to focus on how the physical layers and link layers are efficient, and the application layer has information to steer decisions.

l3 routing should be routing, and not try to scope creep into application / infrastructure. It should stick to the end to end principle and leave the ends to decide on energy actions, while facilitating the data exchange.

IPs role is hugely important, but it needs to remain benign else it will become a political tool. Energy information becoming a component of routing data looks like a cyberattack waiting to happen to me.

BGP tables going wrong causes chaos, but generally ‘only’ impact data access. A DDoS attack on our energy using IP networks could cause real-world problems with much deeper consequences than ‘some data loss’.



--
Dom Robinson
www.id3as.co.uk<http://www.id3as.co.uk/>
www.greeningofstreaming.org<http://www.greeningofstreaming.org/>
uk.linkedin.com/in/domrobinson<http://uk.linkedin.com/in/domrobinson>
Meet >> https://calendly.com/id3as



On 26 Mar 2024, at 02:31, George Michaelson <[e-mail]<mailto:[e-mail]>> wrote:
Without wanting to push too hard, I would personally push back on
E-impact becoming a WG or having any solidity beyond an experimental
ongoing activity. I am simply unconvinced there is substantive
protocol work in energy burden to be done in IETF protocol space at
this time.

* I absolutely believe there is an energy impact issue in DC, in long
distance communications and maintenance of RF carrier, for the IEEE,
for 3GPP, for economies with unreliable power facing the question to
house a CDN or long-line it to somewhere else. These problems lie in
other people's standards bodies.

* I absolutely do believe we can theorise physical and link layer
protocol changes which reduce energy burdens (in many cases with cost,
like no 0RTT responses because we have to re-establish link) -And that
things like delay tolerant networking or 6LOWPAN go into this space
sometimes.

* I also absolutely do believe there may in future be clearer drive to
do work on energy in IETF, in protocol design for DC or other contexts
where it has to be exposed through the protocol stack to the
application. Therefore I can say there COULD be work to be done here,
in the future.

About as far as I would go right now is a problem statement. And, a
watch-and-review status.

I'm not in charge here and I don't have some magic veto card. If
people wanted to continue, I wouldn't oppose a list being hosted, but
I would be uncomfortable with positions being taken in public implying
the IETF or the IAB have substantive work to do here: I don't see it.

Maybe the IAB differs.

-G

--
E-impact mailing list
[e-mail]<mailto:[e-mail]>
https://www.ietf.org/mailman/listinfo/e-impact

--
E-impact mailing list
[e-mail]<mailto:[e-mail]>
https://www.ietf.org/mailman/listinfo/e-impact

--
E-impact mailing list
[e-mail]<mailto:[e-mail]>
https://www.ietf.org/mailman/listinfo/e-impact