Re: [v6ops] draft-cc-v6ops-wlcg-flow-label-marking-00.txt

[Brian E Carpenter <[e-mail]>]

On 30-Jun-23 21:02, Tim Chown wrote:
> Hi Brian,
> 
>> On 29 Jun 2023, at 23:50, Brian E Carpenter <[e-mail]> wrote:
>>
>> Hi Dale,
>>
>> Thanks for documenting this.
>>
>> As well as the impact on ECMP/LAG (where you might add a reference to
>> RFC 6438), could you also mention the impact on server load balancing
>> (RFC 7098)? The real issue here is that with only 5 random bits, the
>> probability of two very large flows hashing to the same path or server
>> is higher than we'd like.
> 
> We can add that pointer.
> 
>> But since the WLCG flows are *all* very large,
>> this presumably doesn't matter so much in your use case?
> 
> Well, actually they’re not. Many transfers are aggregates of a high number of lower throughout flows, moreso perhaps that there’s been a shift away from GridFTP to XRootD.

OK. But in that case I think you do need to discuss the impact on ECMP/LAG a bit more deeply. Is ECMP/LAG a factor in WLCG, and if so, is your traffic mixed in with other general purpose traffic that has standard flow labels? Does an effective 1/32 risk of flow label collision significantly affect the traffic distribution between ECMP/LAG paths?

As far as server load balancing goes, I assume that is either absent in WLCG or is explicitly under your control, which means you can decide what matters and what doesn't.

Perhaps there's a bit of background needed here, to explain whether WLCG traffic is on dedicated paths and what the traffic profiles look like. (How many people here know anything about GridFTP or XRootD?) This determines whether non-standard flow label usage is harmful, and whether this technique can be used safely in other use cases.

     Brian

> 
>> I'd never thought about using a Hamming code to generate flow labels.
>> But since a given flow must use the same flow label throughout, you can
>> only use the regular 5-tuple as input to the Hamming code. I don't
>> believe that a Hamming code will give you a discrete uniform distribution,
>> but I'm not an expert in that area.
> 
> This would be good to get some expert feedback on.
> 
> Note that traffic for a single excitement (community) and application will share 15 common bits, so it’s the other 5 bits we would like to see as effectively randomised and distributed as possible.
> 
>> An expert known as ChatGPT says:
>> "No, a Hamming code does not produce a discrete uniform distribution...
>> The non-uniformity of the distribution arises from the fact that Hamming
>> codes prioritize error detection and correction capabilities over
>> uniformity of codeword distribution."
> 
> :)
> 
>> So I think that simply using a stateless hash is the best way. These
>> days I like FNV (draft-eastlake-fnv) which is easy and cheap to implement.
> 
> This is all good discussion, more feedback very welcome.
> 
> We’d like to produce a -01 before the cutoff.
> 
> Thanks,
> Tim
> 
>> Regards
>>    Brian Carpenter
>>
>> On 30-Jun-23 07:52, Dale W. Carder wrote:
>>> Hi folks,
>>> We have submitted this draft documenting our experimental use of the
>>> IPv6 flow label for packet marking and the considerations we had made
>>> along the way that sort of got us to this point.
>>> I'm hoping we could get some initial discussion underway here before
>>> IETF 117.  Thanks!
>>> Dale
>>> ----- Forwarded message from [e-mail] -----
>>>> Date: Thu, 29 Jun 2023 12:18:53 -0700
>>>> From: [e-mail]
>>>> Subject: New Version Notification for draft-cc-v6ops-wlcg-flow-label-marking-00.txt
>>>>
>>>>
>>>> A new version of I-D, draft-cc-v6ops-wlcg-flow-label-marking-00.txt
>>>> has been successfully submitted by Dale W. Carder and posted to the
>>>> IETF repository.
>>>>
>>>> Name:		draft-cc-v6ops-wlcg-flow-label-marking
>>>> Revision:	00
>>>> Title:		Use of the IPv6 Flow Label for WLCG Packet Marking
>>>> Document date:	2023-06-29
>>>> Group:		Individual Submission
>>>> Pages:		15
>>>> URL:            https://www.ietf.org/archive/id/draft-cc-v6ops-wlcg-flow-label-marking-00.txt
>>>> Status:         https://datatracker.ietf.org/doc/draft-cc-v6ops-wlcg-flow-label-marking/
>>>> Html:           https://www.ietf.org/archive/id/draft-cc-v6ops-wlcg-flow-label-marking-00.html
>>>> Htmlized:       https://datatracker.ietf.org/doc/html/draft-cc-v6ops-wlcg-flow-label-marking
>>>>
>>>>
>>>> Abstract:
>>>>     This document describes an experimentally deployed approach currently
>>>>     used within the Worldwide Large Hadron Collider Computing Grid (WLCG)
>>>>     to mark packets with their project (experiment) and application.  The
>>>>     marking uses the 20-bit IPv6 Flow Label in each packet, with 15 bits
>>>>     used for semantics and 5 bits for entropy.  Alternatives, in
>>>>     particular use of IPv6 Extension Headers (EH), were considered but
>>>>     found to not be practical.  The WLCG is one of the largest worldwide
>>>>     research communities and has adopted IPv6 heavily for movement of
>>>>     many tens of PB of data annually, with the ultimate goal of running
>>>>     IPv6 only.
>>>>
>>>>                                                                                    
>>>>
>>>> The IETF Secretariat
>>>>
>>>>
>>> ----- End forwarded message -----
>>> _______________________________________________
>>> v6ops mailing list
>>> [e-mail]
>>> https://www.ietf.org/mailman/listinfo/v6ops
>>
>> _______________________________________________
>> v6ops mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/v6ops
>