Re: [sfc] Framework
"Jim Guichard (jguichar)" <jguichar@cisco.com> Fri, 14 February 2014 14:06 UTC
Return-Path: <jguichar@cisco.com>
X-Original-To: sfc@ietfa.amsl.com
Delivered-To: sfc@ietfa.amsl.com
Received: from localhost (ietfa.amsl.com [127.0.0.1]) by ietfa.amsl.com (Postfix) with ESMTP id 4B76F1A0273 for <sfc@ietfa.amsl.com>; Fri, 14 Feb 2014 06:06:52 -0800 (PST)
X-Virus-Scanned: amavisd-new at amsl.com
X-Spam-Flag: NO
X-Spam-Score: -10.049
X-Spam-Level:
X-Spam-Status: No, score=-10.049 tagged_above=-999 required=5 tests=[BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, RP_MATCHES_RCVD=-0.548, SPF_PASS=-0.001, USER_IN_DEF_DKIM_WL=-7.5] autolearn=ham
Received: from mail.ietf.org ([4.31.198.44]) by localhost (ietfa.amsl.com [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zHZsQQiE1zWx for <sfc@ietfa.amsl.com>; Fri, 14 Feb 2014 06:06:47 -0800 (PST)
Received: from alln-iport-5.cisco.com (alln-iport-5.cisco.com [173.37.142.92]) by ietfa.amsl.com (Postfix) with ESMTP id B193B1A026B for <sfc@ietf.org>; Fri, 14 Feb 2014 06:06:46 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=cisco.com; i=@cisco.com; l=48026; q=dns/txt; s=iport; t=1392386805; x=1393596405; h=from:to:subject:date:message-id:references:in-reply-to: content-id:content-transfer-encoding:mime-version; bh=P8Ld4AA3CgjKmbLXuYKyAjVkxvS1JUhB3B25tzDLxwM=; b=OC/4LEApKFPmUDQNVrPuAzdoIEmGxUgvyfAJinoi/QlXgVi+YZ/gciow L3gN8R/K988/JtxePObVLSMLwZzb4J/PgiMmQg2GN19hxmjN7JvzM4oIx WLVyzKwU+67Hve4ok348eog8rHIrOeb0mmRpY44PSc6oFZD0sexMB7Ss2 E=;
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: AicFAOoh/lKtJV2c/2dsb2JhbABQCYMGOFe/MoEUFnSCJQEBAQQBAQEXAUoCBxcEAgEIEQEDAQEBJwchBgsUAwYIAgQBEodxAxENwFENiDwXjF+BPygIMgIEhDIEiRCNMIFsgTKLLIVFgy2CKg
X-IronPort-AV: E=Sophos;i="4.95,845,1384300800"; d="scan'208";a="20478576"
Received: from rcdn-core-5.cisco.com ([173.37.93.156]) by alln-iport-5.cisco.com with ESMTP; 14 Feb 2014 14:06:43 +0000
Received: from xhc-rcd-x04.cisco.com (xhc-rcd-x04.cisco.com [173.37.183.78]) by rcdn-core-5.cisco.com (8.14.5/8.14.5) with ESMTP id s1EE6h5I029029 (version=TLSv1/SSLv3 cipher=AES128-SHA bits=128 verify=FAIL); Fri, 14 Feb 2014 14:06:43 GMT
Received: from xmb-rcd-x01.cisco.com ([169.254.1.57]) by xhc-rcd-x04.cisco.com ([fe80::200:5efe:173.37.183.34%12]) with mapi id 14.03.0123.003; Fri, 14 Feb 2014 08:06:43 -0600
From: "Jim Guichard (jguichar)" <jguichar@cisco.com>
To: "Joel M. Halpern" <jmh@joelhalpern.com>, Linda Dunbar <linda.dunbar@huawei.com>, "sfc@ietf.org" <sfc@ietf.org>
Thread-Topic: [sfc] Framework
Thread-Index: AQHPKA5UwnREFNrUG0aiDEicuLyqC5qyPG4AgAAqPQCAAAjqgIAAAfCAgAAIFwCAAAcWAIAAAVmAgAAECQCAAAGvgIAADUiAgAACHACAALUQgIAAOXoAgAALygCAAB5QgIAAAmMAgAADlgCAAAjLAIAAAu8AgAAiSYD//6Jp6oAAbhkAgAACIACAAAHIgIAAAcsAgAAEf4CAABClAIAAOgEAgAABwwCAAJG3AA==
Date: Fri, 14 Feb 2014 14:06:42 +0000
Message-ID: <CF238C76.156E2%jguichar@cisco.com>
References: <CF225A5B.90D3%repenno@cisco.com> <52FD1D03.9030905@joelhalpern.com> <245f4cf6d3824c1f8758ddeefdc7c16b@CO2PR05MB716.namprd05.prod.outlook.com> <52FD2249.9020802@joelhalpern.com> <0BF7E0211CA62B42AE3FD4020E41609884A079D0@SEAEMBX01.olympus.F5Net.com> <4A95BA014132FF49AE685FAB4B9F17F645C7FC87@dfweml701-chm.china.huawei.com> <52FD6262.1040605@joelhalpern.com>
In-Reply-To: <52FD6262.1040605@joelhalpern.com>
Accept-Language: en-US
Content-Language: en-US
X-MS-Has-Attach:
X-MS-TNEF-Correlator:
user-agent: Microsoft-MacOutlook/14.3.9.131030
x-originating-ip: [10.82.213.38]
Content-Type: text/plain; charset="iso-8859-1"
Content-ID: <F4D9DFBD59E7594B991EBEBEA766132A@emea.cisco.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
Archived-At: http://mailarchive.ietf.org/arch/msg/sfc/EmYHwX5SX6pkIoTSTB-Go1Nmj3M
Subject: Re: [sfc] Framework
X-BeenThere: sfc@ietf.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: Network Service Chaining <sfc.ietf.org>
List-Unsubscribe: <https://www.ietf.org/mailman/options/sfc>, <mailto:sfc-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www.ietf.org/mail-archive/web/sfc/>
List-Post: <mailto:sfc@ietf.org>
List-Help: <mailto:sfc-request@ietf.org?subject=help>
List-Subscribe: <https://www.ietf.org/mailman/listinfo/sfc>, <mailto:sfc-request@ietf.org?subject=subscribe>
X-List-Received-Date: Fri, 14 Feb 2014 14:06:52 -0000
Yes agreed Joel. Metadata is really ³context² that is consumable by any element of a given SFC. While our SFC encapsulation will of course carry information relevant to the steering of traffic through the desired set of service functions, I would not characterize that as metadata. On 2/13/14, 7:25 PM, "Joel M. Halpern" <jmh@joelhalpern.com> wrote: >I only consider the first of those to be metadata. The chain >identification in the service chaining header or the transport header >are not metadata. Treating fields which are needed for steering as >metadata induces massive confusion. can we please keep those two >concepts separate?! > >Yours, >Joel > >On 2/13/14, 7:18 PM, Linda Dunbar wrote: >> >> I see two types of "metadata" being discussed here: >> >> 1. The "flow metadata", like the transactions carried by http flows. >>They are inserted by applications, or inserted by a service function to >>pass some "cookies" to the next one. (many proxy based service functions >>have those capability) >> >> 2. The "Service Chain metadata", that are inserted by "Chain >>Classifier" or control plane to carry extra information (in additional >>to Chain ID) for the purpose of controlling the sequence of functions on >>a chain. >> >> Correct? >> >> IMHO, the first bullet above are specific to applications or service >>functions internal processing. Many of today's proxy based service >>functions make changes to data packets, some of them make those changes >>for other service functions. Those changes won't be in the Service Chain >>Header field. >> >> Linda >> >> >> >> >> >> -----Original Message----- >> From: sfc [mailto:sfc-bounces@ietf.org] On Behalf Of Sumandra Majee >> Sent: Thursday, February 13, 2014 2:51 PM >> To: Joel M. Halpern; Jerome Moisand; sfc@ietf.org >> Subject: Re: [sfc] Framework >> >> I believe most of us agree that an out of bad signalling will not >>address the majority of requirement. Also a typical http flow carries >>many transactions and each can have different or additional >>classification. So a metadata can not be simple connected with one flow >>either. Current implementations of proxies and load balancers has >>addressed this in many ways including custome header insertion. The >>custom header in this case equivalent of metadata vector. >> >> I think we need an efficient way annotate actual data with inline >>metadata. I also strongly believe in solving the 80% of the use case >> >> Regards >> Sumandra >> ________________________________________ >> From: sfc [sfc-bounces@ietf.org] on behalf of Joel M. Halpern >>[jmh@joelhalpern.com] >> Sent: Thursday, February 13, 2014 11:51 AM >> To: Jerome Moisand; sfc@ietf.org >> Subject: Re: [sfc] Framework >> >> I know very well what GTP does in terms of correlators, and why. >> If all you need is an identifier for the subscriber, carrying a short >>identifier, and using it to select the desired behavior that has been >>pre-populated when the subscribers session starts works really well. >> That is part of why I am not objecting to having provision for >>out-of-band metadata. >> >> However, claiming that a single such correlator is all that is needed >>for even 80% of SFC cases (and I very much supporting trying to focus on >>the 80% cases), just does not seem to work, given the broad range of >>requirements that we have seen. >> >> Yours, >> Joel >> >> On 2/13/14, 2:35 PM, Jerome Moisand wrote: >>> Joel, >>> >>> Protocols like GTP and L2TP work exactly that, with a form of session >>>correlator between the data plane and the control plane. This is very >>>handy for subscriber and service management. Also if a correlator is >>>defined as an opaque and unique number, then one can avoid some of the >>>pitfalls you described. Long-lived metadata is clearly useful (and >>>thanks for sharing, Reinaldo, will read), and there is clear >>>applicability to various service chaining needs. >>> >>> Now this is just one way. The I-D Bruno referred to was just listing >>>this approach as one possible way among others. I totally agree with >>>you that other forms of metadata (like an outcome of the classification >>>of a packet or a sequence of a few packets) may not be suitable for >>>out-of-band signaling. >>> >>> Bottomline seems to be that we have too many options to choose from, >>>and none of them solving it all... Tough. >>> >>> -----Original Message----- >>> From: sfc [mailto:sfc-bounces@ietf.org] On Behalf Of Joel M. Halpern >>> Sent: Thursday, February 13, 2014 2:29 PM >>> To: sfc@ietf.org >>> Subject: Re: [sfc] Framework >>> >>> As I understand it, the tsvwg (and aeon) work is about flow metadata. >>> That is long-lived metadata of use across many packets. That does >>>indeed seem well-suited to out-of-band cases. >>> >>> My concern is that in SFC, there are many cases which are at best >>>badly-addressed if we insist that they be solved using out-of-band >>>metadata. >>> >>> Yours, >>> Joel >>> >>> On 2/13/14, 2:22 PM, Reinaldo Penno (repenno) wrote: >>>> There is draft about transport independent metadata. >>>> >>>> http://tools.ietf.org/html/draft-choukir-tsvwg-flow-metadata-encoding >>>> - >>>> 01 >>>> >>>> The complexities you mention are discussed there: variable-encoding, >>>> direction, security of metadata, etc. >>>> >>>> Thanks, >>>> >>>> -RP >>>> >>>> >>>> On 2/13/14, 11:15 AM, "Joel M. Halpern" <jmh@joelhalpern.com> wrote: >>>> >>>>> While there are cases where out-of-band metadata makes sense, There >>>>> are many cases I would not want to try to cover that way. The best >>>>> examples are situations where the metadata changes from packet to >>>>> packet (such as the data produced by a DPI engine for consumption by >>>>> other applications in the chain.) >>>>> >>>>> More importantly, if you are putting correlators in the packet, it >>>>> is still very complex if you want to use one correlator to handle >>>>> metadata produced by several different entities (the initial >>>>> classifer, the DPI engine, ...) You quickly end up deciding that >>>>> you need several correlators. At which point we are back to >>>>>variable amounts of metadata. >>>>> >>>>> The alternative is to require exceedingly specific and strong >>>>> coupling to a mandated out-of-band mechanism. That seems to me to >>>>> be a very bad idea. >>>>> >>>>> Yours, >>>>> Joel >>>>> >>>>> On 2/13/14, 1:40 PM, Bruno Rijsman wrote: >>>>>> resend to avoid "too many recipients" warning >>>>>> >>>>>> >>>>>> On Thu, Feb 13, 2014 at 1:39 PM, Bruno Rijsman >>>>>> <brunorijsman@gmail.com <mailto:brunorijsman@gmail.com>> wrote: >>>>>> >>>>>> There are ways to signal variable length amounts of metadata >>>>>>without >>>>>> needing an additional variable-length header on each data >>>>>>packet. >>>>>> >>>>>> draft-rijsman-sfc-metadata-considerations-00 discusses some >>>>>>of the >>>>>> possible approaches: out-of-band signaling (congruent or >>>>>> non-congruent) or a hybrid approach using a fix-length >>>>>>in-band >>>>>> correlator and out-of-band additional metadata. See >>>>>>sections 3.3, >>>>>> 3.4 and 3.5 in the draft. >>>>>> >>>>>> The issue of fixed-length encoding versus variable length >>>>>>encoding >>>>>> is discussion in the challenges chapter in section 4.3. I >>>>>>should >>>>>> probably mention the MTU and segmentation issues as well in >>>>>>that >>>>>> section. >>>>>> >>>>>> >>>>>> On Thu, Feb 13, 2014 at 1:16 PM, Joel M. Halpern >>>>>> <jmh@joelhalpern.com <mailto:jmh@joelhalpern.com>> wrote: >>>>>> >>>>>> The variable length shim header is needed even if every >>>>>> individual piece of metadata has a fixed length. >>>>>>Different >>>>>> service chains will need different combinations of >>>>>>metadata. So >>>>>> the metadata portion of the header needs to be variable >>>>>>length. >>>>>> >>>>>> I think the approach of a fixed length initial part, and >>>>>>a >>>>>> variable length extension part, is the right model. >>>>>> >>>>>> Yours, >>>>>> Joel >>>>>> >>>>>> >>>>>> On 2/13/14, 11:13 AM, Jerome Moisand wrote: >>>>>> >>>>>> Yes, fragmentation and variable headers are usually >>>>>>a really >>>>>> bad thing for forwarding performance. Let's not >>>>>>forget that >>>>>> we live in a 100G-ish kind of world nowadays. >>>>>> >>>>>> Now the question should be: WHY would we want to do >>>>>>that >>>>>> (variable headers leading to fragmentation issues)? >>>>>> >>>>>> For example, the type of metadata that may require >>>>>> variable-length fields might be a better candidate >>>>>>for some >>>>>> out-of-band signaling mechanism. If this is service >>>>>> authorization data (e.g. a service profile/policy), >>>>>>this >>>>>> sounds likely. Not 100% sure this is true for all >>>>>>use cases >>>>>> though. Only a good understanding of use cases, >>>>>>grounded by >>>>>> reflecting on existing network architectures (notably >>>>>> broadband, fixed or mobile), would tell us if one >>>>>>approach >>>>>> or another is the most appropriate. >>>>>> >>>>>> Anyhoo, we're jumping way deep in the detailed >>>>>>protocol >>>>>> design here. This seems a tad premature. >>>>>> >>>>>> -----Original Message----- >>>>>> From: sfc [mailto:sfc-bounces@ietf.org >>>>>> <mailto:sfc-bounces@ietf.org>] On Behalf Of Dave >>>>>>Dolson >>>>>> Sent: Thursday, February 13, 2014 11:03 AM >>>>>> To: 'jmh@joelhalpern.com >>>>>><mailto:jmh@joelhalpern.com>'; >>>>>> 'Ron_Parker@affirmednetworks.__com >>>>>> <mailto:Ron_Parker@affirmednetworks.com>'; >>>>>> 'mohamed.boucadair@orange.com >>>>>> <mailto:mohamed.boucadair@orange.com>'__; >>>>>> 'Nicolas.BOUTHORS@qosmos.com >>>>>> <mailto:Nicolas.BOUTHORS@qosmos.com>'; >>>>>>'paulq@cisco.com >>>>>> <mailto:paulq@cisco.com>' >>>>>> Cc: 'S.Majee@F5.com'; 'sfc@ietf.org >>>>>><mailto:sfc@ietf.org>'; >>>>>> 'linda.dunbar@huawei.com >>>>>><mailto:linda.dunbar@huawei.com>' >>>>>> Subject: Re: [sfc] Framework >>>>>> >>>>>> it is overall more efficient to support PMTU >>>>>>discovery >>>>>> rather than fragmenting every large packet. The is >>>>>>why TCP >>>>>> adopted it so early on. >>>>>> >>>>>> The fragmenting also puts huge performance burden on >>>>>>the >>>>>> service functions. >>>>>> Fragmenting may also affect the ability of load >>>>>>balancers to >>>>>> get each fragment to the correct destination. >>>>>> >>>>>> So PMTU discovery SHOULD be used, in my opinion. By >>>>>>this I >>>>>> mean the client or server is informed that its >>>>>>packets are >>>>>> too big (for IPv6 or IPv4 with DF=1). >>>>>> >>>>>> Some operators may wish to incur the extra cost to >>>>>>hide the >>>>>> existence of the tunneling, when the elements of the >>>>>>chain >>>>>> can support it, so we could consider that as an >>>>>>optional >>>>>> mechanism. >>>>>> >>>>>> -Dave >>>>>> >>>>>> >>>>>> ----- Original Message ----- >>>>>> From: Joel M. Halpern [mailto:jmh@joelhalpern.com >>>>>> <mailto:jmh@joelhalpern.com>] >>>>>> Sent: Thursday, February 13, 2014 10:31 AM >>>>>> To: Ron Parker <Ron_Parker@affirmednetworks.__com >>>>>> <mailto:Ron_Parker@affirmednetworks.com>>; >>>>>> mohamed.boucadair@orange.com >>>>>> <mailto:mohamed.boucadair@orange.com> >>>>>> <mohamed.boucadair@orange.com >>>>>> <mailto:mohamed.boucadair@orange.com>>__; Dave >>>>>>Dolson; >>>>>> 'Nicolas.BOUTHORS@qosmos.com >>>>>> <mailto:Nicolas.BOUTHORS@qosmos.com>' >>>>>> <Nicolas.BOUTHORS@qosmos.com >>>>>> <mailto:Nicolas.BOUTHORS@qosmos.com>>; >>>>>>'paulq@cisco.com >>>>>> <mailto:paulq@cisco.com>' <paulq@cisco.com >>>>>> <mailto:paulq@cisco.com>> >>>>>> Cc: 'S.Majee@F5.com' <S.Majee@F5.com>; 'sfc@ietf.org >>>>>> <mailto:sfc@ietf.org>' <sfc@ietf.org >>>>>><mailto:sfc@ietf.org>>; >>>>>> 'linda.dunbar@huawei.com >>>>>><mailto:linda.dunbar@huawei.com>' >>>>>> <linda.dunbar@huawei.com >>>>>><mailto:linda.dunbar@huawei.com>> >>>>>> Subject: Re: [sfc] Framework >>>>>> >>>>>> I mostly agree with you Ron. I can probably come up >>>>>>with >>>>>> some other variations, but your second point is that >>>>>>the >>>>>> transport must deal with the issue of frame size / >>>>>>fit. >>>>>> >>>>>> I would note that if we assume that the carrying >>>>>>encaps >>>>>> (IPv4/v6 outer) is being fragmented, then we are >>>>>>assuming >>>>>> that the exit from service chaining can and will >>>>>>reassemble. >>>>>> >>>>>> Yours, >>>>>> Joel >>>>>> >>>>>> On 2/13/14, 10:18 AM, Ron Parker wrote: >>>>>> >>>>>> Joel, >>>>>> >>>>>> That is an excellent point to consider when >>>>>>choosing >>>>>> transport >>>>>> encapsulations. If the transport encapsulation >>>>>>is IPv4 >>>>>> or IPv6 >>>>>> based (i.e., IPx/UDP, IPx/GRE, IPx/UDP/VxLAN, >>>>>> etc.), then >>>>>> fragmentation and defragmentation are naturally >>>>>> supported. If the >>>>>> transport encapsulation is VLAN, MPLS, etc., >>>>>>then I >>>>>> believe one of the >>>>>> following must be true: >>>>>> >>>>>> * The end-to-end path is known to support the >>>>>>resulting >>>>>> larger frames >>>>>> * A path discovery mechanism is mandated by SFC >>>>>>when >>>>>> non-IP transports >>>>>> are used * An SFC-specific fragmentation header >>>>>>and >>>>>> mechanisms must be >>>>>> defined (i.e., >>>>>> >>>>>> SFC-transport/SFC-__fragmentation/SFC-service/__original-packet-or- >>>>>> f >>>>>> rame) >>>>>> >>>>>> Ron >>>>>> >>>>>> >>>>>> -----Original Message----- From: Joel M. Halpern >>>>>> [mailto:jmh@joelhalpern.com >>>>>> <mailto:jmh@joelhalpern.com>] Sent: Thursday, >>>>>>February >>>>>> 13, 2014 10:10 >>>>>> AM To: mohamed.boucadair@orange.com >>>>>> <mailto:mohamed.boucadair@orange.com>; Dave >>>>>>Dolson; >>>>>> 'Nicolas.BOUTHORS@qosmos.com >>>>>> <mailto:Nicolas.BOUTHORS@qosmos.com>'; Ron >>>>>>Parker; >>>>>> 'paulq@cisco.com <mailto:paulq@cisco.com>' Cc: >>>>>> 'S.Majee@F5.com'; 'sfc@ietf.org >>>>>><mailto:sfc@ietf.org>'; >>>>>> 'linda.dunbar@huawei.com >>>>>> <mailto:linda.dunbar@huawei.com>' Subject: >>>>>> Re: [sfc] Framework >>>>>> >>>>>> There is a related complexity. I hope that we >>>>>>expect to >>>>>> support IPv6. >>>>>> IPv6 explicitly prohibits network elements from >>>>>> fragmenting end user >>>>>> packets. >>>>>> >>>>>> Yours, Joel >>>>>> >>>>>> On 2/13/14, 8:21 AM, mohamed.boucadair@orange.com >>>>>> <mailto:mohamed.boucadair@orange.com> wrote: >>>>>> >>>>>> Re-, >>>>>> >>>>>> FWIW, one of the existing architecture drafts >>>>>> includes the following >>>>>> behavior >>>>>> >>>>>> (http://tools.ietf.org/html/__draft-boucadair-sfc-framework-__02#se >>>>>> c >>>>>> tion- >>>>>> 6 >>>>>> >>>>>> >>>>>><http://tools.ietf.org/html/draft-boucadair-sfc-framework-02#section- >>>>>>6>): >>>>>> >>>>>> >>>>>> >>>>>> " >>>>>> >>>>>> 6. Fragmentation Considerations >>>>>> >>>>>> If adding the Service Chaining Header would >>>>>>result >>>>>> in a fragmented >>>>>> packet, the classifier should include a >>>>>>Service >>>>>> Chaining Header in >>>>>> each fragment. Doing so would prevent SF >>>>>>Nodes to >>>>>> dedicate resource >>>>>> to handle fragments. " >>>>>> >>>>>> Cheers, Med >>>>>> >>>>>> >>>>>> -----Message d'origine----- De : sfc >>>>>> [mailto:sfc-bounces@ietf.org >>>>>> <mailto:sfc-bounces@ietf.org>] >>>>>> De la part de Dave Dolson Envoyé : >>>>>> jeudi 13 février 2014 13:39 À : >>>>>> 'Nicolas.BOUTHORS@qosmos.com >>>>>> <mailto:Nicolas.BOUTHORS@qosmos.com>'; >>>>>> 'Ron_Parker@affirmednetworks.__com >>>>>> >>>>>><mailto:Ron_Parker@affirmednetworks.com>'; >>>>>> 'jmh@joelhalpern.com >>>>>> <mailto:jmh@joelhalpern.com>'; >>>>>> 'paulq@cisco.com >>>>>><mailto:paulq@cisco.com>' Cc : >>>>>> 'S.Majee@F5.com'; 'sfc@ietf.org >>>>>> <mailto:sfc@ietf.org>'; >>>>>> 'linda.dunbar@huawei.com >>>>>> <mailto:linda.dunbar@huawei.com>' Objet >>>>>>: Re: >>>>>> [sfc] Framework >>>>>> >>>>>> Good point to consider fragmentation. >>>>>> >>>>>> We should design the encapsulation such >>>>>>that the >>>>>> forwarding >>>>>> functions do not need to reassemble. Each >>>>>> fragment should be >>>>>> independently forwardable; some SFs may >>>>>>choose >>>>>> to ignore these >>>>>> packets. >>>>>> >>>>>> Any layer 2.5 protocol like VLAN or MPLS >>>>>>would >>>>>> support this. >>>>>> Otherwise, a reassembly layer could be >>>>>>added >>>>>> after the forwarding >>>>>> coordinates. Think of something like the >>>>>>IPv6 >>>>>> reassembly option >>>>>> after a GRE header, for instance. >>>>>> >>>>>> IP | GRE | reassem | frag data >>>>>> >>>>>> We could alternatively mandate the inner >>>>>>IP be >>>>>> fragmented and that >>>>>> path-MTU discovery be supported. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> ----- Original Message ----- From: >>>>>> Nicolas BOUTHORS >>>>>> [mailto:Nicolas.BOUTHORS@__qosmos.com >>>>>> <mailto:Nicolas.BOUTHORS@qosmos.com>] >>>>>>Sent: >>>>>> Thursday, February 13, >>>>>> 2014 04:13 AM To: Ron Parker >>>>>> <Ron_Parker@affirmednetworks.__com >>>>>> >>>>>><mailto:Ron_Parker@affirmednetworks.com>>; >>>>>> Dave Dolson; Joel M. Halpern >>>>>> <jmh@joelhalpern.com >>>>>> <mailto:jmh@joelhalpern.com>>; Paul Quinn >>>>>> (paulq) <paulq@cisco.com >>>>>> <mailto:paulq@cisco.com>> Cc: Sumandra >>>>>>Majee >>>>>> <S.Majee@F5.com>; >>>>>> sfc@ietf.org <mailto:sfc@ietf.org> >>>>>><sfc@ietf.org >>>>>> <mailto:sfc@ietf.org>>; Linda Dunbar >>>>>> <linda.dunbar@huawei.com >>>>>> <mailto:linda.dunbar@huawei.com>> >>>>>> Subject: Re: [sfc] Framework >>>>>> >>>>>> If we do not enforce a fix header size >>>>>>we have >>>>>> other implication. >>>>>> >>>>>> There is the question of segmentation and >>>>>> reassembly responsibility >>>>>> as well. >>>>>> >>>>>> If adding length to the service header >>>>>>forces >>>>>> segmentation, then >>>>>> whose responsibility is it to reassemble >>>>>>the >>>>>> payload before passing >>>>>> it to the SF. >>>>>> >>>>>> Similar question for packet re-ordering. >>>>>> >>>>>> >>>>>> >>>>>>__________________________________________ From: >>>>>> Ron Parker >>>>>> [Ron_Parker@affirmednetworks.__com >>>>>> >>>>>><mailto:Ron_Parker@affirmednetworks.com>] Sent: >>>>>> Wednesday, February 12, >>>>>> 2014 11:25 PM To: Dave Dolson; Joel M. >>>>>>Halpern; >>>>>> Paul Quinn >>>>>> (paulq) Cc: Sumandra Majee; sfc@ietf.org >>>>>> <mailto:sfc@ietf.org>; Linda Dunbar >>>>>>Subject: >>>>>> Re: [sfc] Framework >>>>>> >>>>>> Dave, >>>>>> >>>>>> Yes, that is a good point if we logically >>>>>> separate the forwarding >>>>>> function from the SFC-aware service >>>>>>function, as >>>>>> we should. The >>>>>> forwarding function is concerned with >>>>>>only the >>>>>> inbound transport >>>>>> header, the fixed portion of the >>>>>>service header >>>>>> containing the >>>>>> chain information, and the outbound >>>>>>transport >>>>>> header. The >>>>>> service function may look at the >>>>>>entirety of the >>>>>> service header and >>>>>> would look at the encapsulated packet. >>>>>> >>>>>> Ron >>>>>> >>>>>> >>>>>> -----Original Message----- From: Dave >>>>>>Dolson >>>>>> [mailto:ddolson@sandvine.com >>>>>> <mailto:ddolson@sandvine.com>] Sent: >>>>>>Wednesday, >>>>>> February 12, 2014 >>>>>> 5:18 PM To: Joel M. Halpern; Ron Parker; >>>>>>Paul >>>>>> Quinn (paulq) Cc: >>>>>> Sumandra Majee; sfc@ietf.org >>>>>> <mailto:sfc@ietf.org>; Linda Dunbar >>>>>>Subject: RE: >>>>>> [sfc] >>>>>> Framework >>>>>> >>>>>> The forwarding plane might not even need >>>>>>to look >>>>>> at the encapsulated >>>>>> packet. >>>>>> >>>>>> For example, (if the SF Identifier is a >>>>>>VLAN >>>>>> tag), an Ethernet >>>>>> switch can forward packets of the form: >>>>>>Ether | >>>>>> VLAN | BLOB. >>>>>> >>>>>> >>>>>> >>>>>> -----Original Message----- From: sfc >>>>>> [mailto:sfc-bounces@ietf.org >>>>>> <mailto:sfc-bounces@ietf.org>] >>>>>> On Behalf Of Joel M. Halpern Sent: >>>>>> Wednesday, February 12, 2014 4:30 PM To: >>>>>>Ron >>>>>> Parker; Dave Dolson; >>>>>> Paul Quinn (paulq) Cc: Sumandra Majee; >>>>>> sfc@ietf.org <mailto:sfc@ietf.org>; >>>>>>Linda Dunbar >>>>>> Subject: Re: [sfc] Framework >>>>>> >>>>>> I agree as well. Yours, Joel >>>>>> >>>>>> On 2/12/14, 4:24 PM, Ron Parker wrote: >>>>>> >>>>>> Hi, Dave. >>>>>> >>>>>> I Agree with your statement. And >>>>>>if the >>>>>> total length of the >>>>>> header is >>>>>> >>>>>> contained in the mandatory portion, the >>>>>>hardware >>>>>> implementation can >>>>>> easily locate the encapsulated packet. >>>>>> >>>>>> >>>>>> Ron >>>>>> >>>>>> >>>>>> -----Original Message----- From: >>>>>>Dave Dolson >>>>>> [mailto:ddolson@sandvine.com >>>>>> <mailto:ddolson@sandvine.com>] Sent: >>>>>> Wednesday, February 12, >>>>>> 2014 4:10 PM To: Ron Parker; Paul >>>>>>Quinn >>>>>> (paulq) Cc: Joel M. >>>>>> Halpern; Sumandra Majee; sfc@ietf.org >>>>>> <mailto:sfc@ietf.org>; Linda Dunbar >>>>>>Subject: >>>>>> RE: [sfc] Framework >>>>>> >>>>>> I think a good approach would be: >>>>>> >>>>>> The information required for >>>>>>forwarding (the >>>>>> SF Identifier) should >>>>>> be in >>>>>> >>>>>> a mandatory fixed-size header. >>>>>> >>>>>> >>>>>> Optional information (if any) is NOT >>>>>>to be >>>>>> used for forwarding, but >>>>>> is >>>>>> >>>>>> for consumption by one or more of the >>>>>> applications in the chain. >>>>>> >>>>>> >>>>>> Then the forwarding plane, and even >>>>>>the PDP, >>>>>> can be agnostic about >>>>>> the >>>>>> >>>>>> meta-data. >>>>>> >>>>>> >>>>>> -Dave >>>>>> >>>>>> >>>>>> -----Original Message----- From: sfc >>>>>> [mailto:sfc-bounces@ietf.org >>>>>> <mailto:sfc-bounces@ietf.org>] >>>>>> On Behalf Of Ron Parker Sent: >>>>>> Wednesday, February 12, 2014 4:05 PM >>>>>>To: >>>>>> Paul Quinn (paulq) Cc: >>>>>> Joel M. Halpern; Sumandra Majee; >>>>>> sfc@ietf.org <mailto:sfc@ietf.org>; >>>>>> Linda Dunbar >>>>>> Subject: Re: [sfc] Framework >>>>>> >>>>>> Paul, >>>>>> >>>>>> That is why I am proposing a hybrid >>>>>>where >>>>>> extensions or options can >>>>>> be >>>>>> >>>>>> added but the total length is contained >>>>>>in the >>>>>> fixed portion. I >>>>>> can envision certain deployments (e.g., >>>>>> Enterprise) where perhaps >>>>>> extensions are not needed and the SFC >>>>>> functionality is realized >>>>>> in hardware. And, I can envision >>>>>>certain other >>>>>> deployments >>>>>> (e.g., 3gpp) where the extension headers >>>>>>add >>>>>> sufficient value to >>>>>> justify software based implementations. >>>>>> >>>>>> >>>>>> Ron >>>>>> >>>>>> >>>>>> -----Original Message----- From: >>>>>>Paul Quinn >>>>>> (paulq) >>>>>> [mailto:paulq@cisco.com >>>>>> <mailto:paulq@cisco.com>] Sent: >>>>>>Wednesday, >>>>>> February 12, 2014 >>>>>> 3:40 PM To: Ron Parker Cc: Sumandra >>>>>>Majee; >>>>>> Linda Dunbar; Joel M. >>>>>> Halpern; sfc@ietf.org >>>>>><mailto:sfc@ietf.org> >>>>>> Subject: Re: [sfc] Framework >>>>>> >>>>>> Hi, >>>>>> >>>>>> We certainly need to be very careful >>>>>>with >>>>>> variable length headers >>>>>> when >>>>>> >>>>>> hardware platform are in play. >>>>>> >>>>>> >>>>>> Ron, your examples of IP options and >>>>>>v6 EH >>>>>> have both suffered from >>>>>> >>>>>> significant implementation and deployment >>>>>> hurdles due to the >>>>>> complexity and cost associated with >>>>>>hardware >>>>>> support for the >>>>>> option/EH. >>>>>> >>>>>> >>>>>> Paul >>>>>> >>>>>> On Feb 12, 2014, at 3:10 PM, Ron >>>>>>Parker >>>>>> <Ron_Parker@affirmednetworks.__com >>>>>> >>>>>> <mailto:Ron_Parker@affirmednetworks.com>> >>>>>> >>>>>> wrote: >>>>>> >>>>>> >>>>>> Hi, Sumandra. >>>>>> >>>>>> Regarding service header >>>>>>flexibility, I >>>>>> completely agree. I >>>>>> might >>>>>> >>>>>> suggest a compromise between hardware >>>>>> friendliness and software >>>>>> flexibility. If the header had the >>>>>>ability to >>>>>> add extensions >>>>>> (perhaps similar to IPv6) but also had >>>>>>the >>>>>> header length encoded in >>>>>> the mandatory part (like IPv4), then a >>>>>>hardware >>>>>> implementation would >>>>>> be free to skip over the extensions (in >>>>>>cases >>>>>> where the deployment >>>>>> justifies ignoring the extensions). >>>>>> >>>>>> >>>>>> Ron >>>>>> >>>>>> >>>>>> -----Original Message----- From: >>>>>> Sumandra Majee >>>>>> [mailto:S.Majee@F5.com >>>>>> <mailto:S.Majee@F5.com>] Sent: >>>>>> Wednesday, February 12, 2014 >>>>>> 3:04 PM To: Ron Parker; Linda >>>>>>Dunbar; >>>>>> Joel M. Halpern; >>>>>> sfc@ietf.org >>>>>><mailto:sfc@ietf.org> >>>>>> Subject: Re: [sfc] Framework >>>>>> >>>>>> For all of those >>>>>>reasons, I >>>>>> strongly support a >>>>>> canonical service >>>>>> header that is >>>>>>independent of >>>>>> the transport >>>>>>methodology. >>>>>> >>>>>> >>>>>> >>>>>> I agree. However the format of >>>>>>service >>>>>> header should be >>>>>> standardized in >>>>>> >>>>>> a way that is flexible. Some of us would >>>>>>like to >>>>>> see variable length >>>>>> header that is also HW friendly. The >>>>>>service >>>>>> header can be >>>>>> represented as a mandotory fixed length >>>>>>header >>>>>> (like IP >>>>>> header) along with an optional variable >>>>>>length >>>>>> attribute field. >>>>>> Different services can be interested in >>>>>> different metadata, for >>>>>> example subscriber ID could be of >>>>>>interest in >>>>>> the mobile core >>>>>> service chain only. >>>>>> >>>>>> >>>>>> Sumandra >>>>>> >>>>>> On 2/12/14, 11:32 AM, "Ron >>>>>>Parker" >>>>>> >>>>>> <Ron_Parker@affirmednetworks.__com >>>>>> >>>>>> <mailto:Ron_Parker@affirmednetworks.com>> >>>>>> >>>>>> wrote: >>>>>> >>>>>> >>>>>> Linda, >>>>>> >>>>>> My interpretation of the >>>>>> architecture docs is that the >>>>>> service >>>>>> function chain is built in an >>>>>> abstract manner (i.e., SF-A >>>>>>then >>>>>> SF-B >>>>>> >>>>>> then SF-C). >>>>>> >>>>>> Separately, a locator table >>>>>>provides >>>>>> network location for >>>>>> each of those service >>>>>>functions. >>>>>> In this way, you can >>>>>> locate the service functions >>>>>> >>>>>> in >>>>>> >>>>>> a manner appropriate to the >>>>>>type of >>>>>> transport you are >>>>>> using. If you want that to >>>>>>be >>>>>> interface+VLAN, >>>>>> gateway-IP+MPLS label stack, >>>>>> IP >>>>>> >>>>>> address, >>>>>> >>>>>> GRE tunnel remote IP + key, >>>>>>etc., >>>>>> you can do that. You >>>>>> can even potentially locate >>>>>> different service functions >>>>>>that >>>>>> reside in the same chain with >>>>>> different flavors of >>>>>> network locators. Another >>>>>> justification to separate the >>>>>> identity of a service >>>>>>function from >>>>>> its network location is >>>>>> load balancing. If, for >>>>>>example, >>>>>> SF-A had 3 IP addresses, >>>>>> that would imply load >>>>>>balancing to 3 >>>>>> instances using IP as a >>>>>> transport layer. >>>>>> >>>>>> For all of those reasons, I >>>>>>strongly >>>>>> support a canonical service >>>>>> header that is independent >>>>>>of the >>>>>> transport >>>>>> methodology. At a >>>>>>particular >>>>>> node, the decision as to >>>>>> where to go next should be >>>>>>based >>>>>> solely on information >>>>>>carried in >>>>>> the canonical service header >>>>>>and not >>>>>> on the >>>>>> >>>>>> fields >>>>>> >>>>>> in the transport header. >>>>>>That is, >>>>>> the SFC node discards >>>>>> the transport header and >>>>>>extracts >>>>>> the chain ID from the >>>>>> service header. Through >>>>>> mechanisms we have not yet >>>>>>begun >>>>>> to discuss, the SFC node >>>>>>knows how >>>>>> to interpret the chain ID and >>>>>> ultimately how to progress >>>>>> the packet. >>>>>> >>>>>> Ron >>>>>> >>>>>> >>>>>> -----Original Message----- >>>>>>From: sfc >>>>>> [mailto:sfc-bounces@ietf.org >>>>>> >>>>>><mailto:sfc-bounces@ietf.org>] On >>>>>> Behalf Of Linda Dunbar >>>>>> Sent: Wednesday, February >>>>>>12, 2014 >>>>>> 12:01 PM To: Joel M. >>>>>> Halpern; sfc@ietf.org >>>>>> <mailto:sfc@ietf.org> >>>>>>Subject: Re: >>>>>> [sfc] Framework >>>>>> >>>>>> Agree with Joel's statement. >>>>>> >>>>>> I think a simple sentence >>>>>>below >>>>>> should be added Section 5.2 >>>>>>(SFC >>>>>> Classifier) to emphasize >>>>>>this point. >>>>>> >>>>>> "A Service Chain Classifier >>>>>>node can >>>>>> associate a unique Layer 2 >>>>>> or 3 Label (e.g. VLAN, MPLS >>>>>>label) >>>>>> to the packets in the flow. >>>>>> Those Layer 2 or 3 Label >>>>>>makes it >>>>>> easier for subsequent nodes >>>>>> along the flow path to steer >>>>>>the >>>>>> flow to the service functions >>>>>> specified by the flow's >>>>>> service chain." >>>>>> >>>>>> >>>>>> Linda -----Original >>>>>>Message----- >>>>>> From: sfc >>>>>> [mailto:sfc-bounces@ietf.org >>>>>> >>>>>><mailto:sfc-bounces@ietf.org>] On >>>>>> Behalf Of Joel M. Halpern >>>>>> Sent: Wednesday, February >>>>>>12, 2014 >>>>>> 10:20 AM To: >>>>>> sfc@ietf.org >>>>>><mailto:sfc@ietf.org> >>>>>> Subject: [sfc] Framework >>>>>> >>>>>> I was looking at the >>>>>>framework >>>>>> proposal. The proposal has a >>>>>>very >>>>>> specific model of the scope >>>>>>of the >>>>>> transport header (that it is >>>>>> derived from, and relates >>>>>>only to, >>>>>> the first service function to >>>>>> which the packet will be >>>>>>directed. >>>>>> That is clearly an >>>>>>operational >>>>>> model we need to support. >>>>>> >>>>>> However, I would like to >>>>>>allow other >>>>>> transport operational >>>>>> models, such as the use of a >>>>>>VLAN to >>>>>> direct traffic along a >>>>>> chain. Or the use of an >>>>>>MPLS label, >>>>>> or an MPLS label stack. >>>>>> >>>>>> Yours, Joel M. Halpern >>>>>> >>>>>> >>>>>> _________________________________________________ >>>>>> sfc mailing list >>>>>> sfc@ietf.org >>>>>> <mailto:sfc@ietf.org> >>>>>> >>>>>> https://www.ietf.org/mailman/__listinfo/sfc >>>>>> >>>>>> <https://www.ietf.org/mailman/listinfo/sfc> >>>>>> >>>>>> >>>>>> _________________________________________________ >>>>>> sfc mailing list >>>>>> sfc@ietf.org >>>>>> <mailto:sfc@ietf.org> >>>>>> >>>>>> https://www.ietf.org/mailman/__listinfo/sfc >>>>>> >>>>>> <https://www.ietf.org/mailman/listinfo/sfc> >>>>>> >>>>>> >>>>>> _________________________________________________ >>>>>> sfc mailing list >>>>>> sfc@ietf.org >>>>>> <mailto:sfc@ietf.org> >>>>>> >>>>>> https://www.ietf.org/mailman/__listinfo/sfc >>>>>> >>>>>> <https://www.ietf.org/mailman/listinfo/sfc> >>>>>> >>>>>> >>>>>> >>>>>> _________________________________________________ >>>>>> sfc mailing list >>>>>> sfc@ietf.org >>>>>> <mailto:sfc@ietf.org> >>>>>> >>>>>> https://www.ietf.org/mailman/__listinfo/sfc >>>>>> >>>>>> <https://www.ietf.org/mailman/listinfo/sfc> >>>>>> >>>>>> >>>>>> >>>>>> _________________________________________________ >>>>>> sfc mailing list >>>>>> sfc@ietf.org <mailto:sfc@ietf.org> >>>>>> >>>>>> https://www.ietf.org/mailman/__listinfo/sfc >>>>>> >>>>>> <https://www.ietf.org/mailman/listinfo/sfc> >>>>>> >>>>>> >>>>>> >>>>>> _________________________________________________ sfc >>>>>> mailing list >>>>>> sfc@ietf.org <mailto:sfc@ietf.org> >>>>>> >>>>>>https://www.ietf.org/mailman/__listinfo/sfc >>>>>> >>>>>> <https://www.ietf.org/mailman/listinfo/sfc> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> _________________________________________________ sfc >>>>>> mailing list >>>>>> sfc@ietf.org <mailto:sfc@ietf.org> >>>>>> >>>>>>https://www.ietf.org/mailman/__listinfo/sfc >>>>>> >>>>>> <https://www.ietf.org/mailman/listinfo/sfc> >>>>>> >>>>>> >>>>>> _________________________________________________ sfc >>>>>> mailing list >>>>>> sfc@ietf.org <mailto:sfc@ietf.org> >>>>>> >>>>>>https://www.ietf.org/mailman/__listinfo/sfc >>>>>> >>>>>> <https://www.ietf.org/mailman/listinfo/sfc> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> _________________________________________________ >>>>>> sfc mailing list >>>>>> sfc@ietf.org <mailto:sfc@ietf.org> >>>>>> https://www.ietf.org/mailman/__listinfo/sfc >>>>>> <https://www.ietf.org/mailman/listinfo/sfc> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> _________________________________________________ >>>>>> sfc mailing list >>>>>> sfc@ietf.org <mailto:sfc@ietf.org> >>>>>> https://www.ietf.org/mailman/__listinfo/sfc >>>>>> <https://www.ietf.org/mailman/listinfo/sfc> >>>>>> >>>>>> >>>>>> >>>>> >>>>> _______________________________________________ >>>>> sfc mailing list >>>>> sfc@ietf.org >>>>> https://www.ietf.org/mailman/listinfo/sfc >>>> >>>> >>> >>> _______________________________________________ >>> sfc mailing list >>> sfc@ietf.org >>> https://www.ietf.org/mailman/listinfo/sfc >>> >>> >>> >>> >> >> _______________________________________________ >> sfc mailing list >> sfc@ietf.org >> https://www.ietf.org/mailman/listinfo/sfc >> >> _______________________________________________ >> sfc mailing list >> sfc@ietf.org >> https://www.ietf.org/mailman/listinfo/sfc >> > >_______________________________________________ >sfc mailing list >sfc@ietf.org >https://www.ietf.org/mailman/listinfo/sfc
- [sfc] Framework Joel M. Halpern
- Re: [sfc] Framework Linda Dunbar
- Re: [sfc] Framework Ron Parker
- Re: [sfc] Framework Joel M. Halpern
- Re: [sfc] Framework Ron Parker
- Re: [sfc] Framework Joel M. Halpern
- Re: [sfc] Framework Sumandra Majee
- Re: [sfc] Framework Ron Parker
- Re: [sfc] Framework Paul Quinn (paulq)
- Re: [sfc] Framework Sumandra Majee
- Re: [sfc] Metadata structure Joel M. Halpern
- Re: [sfc] Framework Joel M. Halpern
- Re: [sfc] Framework Surendra Kumar (smkumar)
- Re: [sfc] Framework Ron Parker
- Re: [sfc] Framework Dave Dolson
- Re: [sfc] Framework Ron Parker
- Re: [sfc] Framework Joel M. Halpern
- Re: [sfc] Framework Dave Dolson
- Re: [sfc] Framework Ron Parker
- Re: [sfc] Framework Sumandra Majee
- Re: [sfc] Framework Ron Parker
- Re: [sfc] Framework Joel M. Halpern
- Re: [sfc] Framework Nicolas BOUTHORS
- Re: [sfc] Framework mohamed.boucadair
- Re: [sfc] Framework mohamed.boucadair
- Re: [sfc] Metadata structure mohamed.boucadair
- Re: [sfc] Framework Dave Dolson
- Re: [sfc] Framework Jerome Moisand
- Re: [sfc] Framework Ron Parker
- Re: [sfc] Framework mohamed.boucadair
- Re: [sfc] Framework Joel M. Halpern
- Re: [sfc] Framework Ron Parker
- Re: [sfc] Framework Reinaldo Penno (repenno)
- Re: [sfc] Framework Joel M. Halpern
- Re: [sfc] Framework Surendra Kumar (smkumar)
- Re: [sfc] Framework Ron Parker
- Re: [sfc] Framework Joel M. Halpern
- Re: [sfc] Framework Joel M. Halpern
- Re: [sfc] Framework Dave Dolson
- Re: [sfc] Framework Jerome Moisand
- Re: [sfc] Framework Ron Parker
- Re: [sfc] Framework Ron Parker
- Re: [sfc] Framework mikebianc@aol.com
- Re: [sfc] Framework Dave Dolson
- Re: [sfc] Framework Joel M. Halpern
- Re: [sfc] Framework Bruno Rijsman
- Re: [sfc] Framework Joel M. Halpern
- Re: [sfc] Framework Reinaldo Penno (repenno)
- Re: [sfc] Framework Joel M. Halpern
- Re: [sfc] Framework Sumandra Majee
- Re: [sfc] Framework Kevin J Ma
- Re: [sfc] Framework Ron Parker
- Re: [sfc] Framework Linda Dunbar
- Re: [sfc] Framework Joel M. Halpern
- Re: [sfc] Framework Ron Parker
- Re: [sfc] Framework Nicolas BOUTHORS
- Re: [sfc] Framework Bruno Rijsman
- Re: [sfc] Framework Ron Parker
- Re: [sfc] Framework Jim Guichard (jguichar)
- Re: [sfc] Framework Jerome Moisand
- Re: [sfc] Framework Jim Guichard (jguichar)
- Re: [sfc] Framework Linda Dunbar
- Re: [sfc] Framework David Allan I
- Re: [sfc] Framework Ron Parker
- Re: [sfc] Framework Jim Guichard (jguichar)