Re: [Forces-protocol] Data Encoding

Jamal Hadi Salim <hadi@znyx.com> Tue, 16 November 2004 14:21 UTC

Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA01805 for <forces-protocol-web-archive@ietf.org>; Tue, 16 Nov 2004 09:21:20 -0500 (EST)
Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CU4F8-0006Uy-Qa for forces-protocol-web-archive@ietf.org; Tue, 16 Nov 2004 09:23:39 -0500
Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CU49g-0001z0-AL; Tue, 16 Nov 2004 09:18:00 -0500
Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CU457-0000yd-LG for forces-protocol@megatron.ietf.org; Tue, 16 Nov 2004 09:13:27 -0500
Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id JAA01142 for <forces-protocol@ietf.org>; Tue, 16 Nov 2004 09:13:16 -0500 (EST)
Received: from znx208-2-156-007.znyx.com ([208.2.156.7] helo=lotus.znyx.com) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CU47K-0006Jv-JJ for forces-protocol@ietf.org; Tue, 16 Nov 2004 09:15:35 -0500
Received: from loopback ([208.2.156.2]) by lotus.znyx.com (Lotus Domino Release 5.0.11) with ESMTP id 2004111606164034:24480 ; Tue, 16 Nov 2004 06:16:40 -0800
Subject: Re: [Forces-protocol] Data Encoding
From: Jamal Hadi Salim <hadi@znyx.com>
To: Weiming Wang <wmwang@mail.hzic.edu.cn>
In-Reply-To: <009e01c4cb24$ba144bd0$020aa8c0@wwm1>
References: <4189F776.4080306@zurich.ibm.com> <00bd01c4c536$fb418ee0$020aa8c0@wwm1> <1099885892.2167.13.camel@jzny.localdomain> <132001c4c551$86023150$845c21d2@Necom.hzic.edu.cn> <1099911200.2169.29.camel@jzny.localdomain> <134f01c4c585$216584c0$845c21d2@Necom.hzic.edu.cn> <4191299F.4020809@zurich.ibm.com> <142a01c4c6d6$13569980$845c21d2@Necom.hzic.edu.cn> <1100100893.2210.24.camel@jzny.localdomain> <14fc01c4c79f$75231f20$845c21d2@Necom.hzic.edu.cn> <6.1.2.0.0.20041111091015.03bafec0@localhost> <155001c4c8c0$5eb73ec0$845c21d2@Necom.hzic.edu.cn> <1100269665.1106.30.camel@jzny.localdomain> <156501c4c8c5$1fb19220$845c21d2@Necom.hzic.edu.cn> <1100271071.1109.38.camel@jzny.localdomain> <1100307886.1110.20.camel@jzny.localdomain> <6.1.2.0.0.20041112220931.03c89ec0@localhost> <009a01c4c934$99cf7990$020aa8c0@wwm1> <6.1.2.0.0.20041112230835.03f0b0f0@localhost> <1100381538.2212.914.camel@jzny.localdomain> <6.1.2.0.0.20041113170557.03b05e00@localhost> <1100458281.2327.23.camel@jzny.l! ocaldomain> <009e01c4cb24$ba144bd0$020aa8c0@wwm1>
Organization: ZNYX Networks
Message-Id: <1100610279.1152.33.camel@jzny.localdomain>
Mime-Version: 1.0
X-Mailer: Ximian Evolution 1.2.2
Date: Tue, 16 Nov 2004 08:04:40 -0500
X-MIMETrack: Itemize by SMTP Server on Lotus/Znyx(Release 5.0.11 |July 24, 2002) at 11/16/2004 06:16:40 AM, Serialize by Router on Lotus/Znyx(Release 5.0.11 |July 24, 2002) at 11/16/2004 06:17:01 AM, Serialize complete at 11/16/2004 06:17:01 AM
Content-Transfer-Encoding: 7bit
Content-Type: text/plain
X-Spam-Score: 0.0 (/)
X-Scan-Signature: 963faf56c3a5b6715f0b71b66181e01a
Content-Transfer-Encoding: 7bit
Cc: "(Ram Gopal )" <ram.gopal@nokia.com>, Steve Blake <slblake@petri-meat.com>, "Joel M. Halpern" <joel@stevecrocker.com>, Dong Ligang <donglg@mail.hzic.edu.cn>, "Khosravi, Hormuzd M" <hormuzd.m.khosravi@intel.com>, Avri Doria <avri@acm.org>, forces-protocol@ietf.org, Patrick Droz <dro@zurich.ibm.com>, zsolt@modularnet.com, David.Putzolu@intel.com, Robert Haas <rha@zurich.ibm.com>
X-BeenThere: forces-protocol@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
Reply-To: hadi@znyx.com
List-Id: forces-protocol <forces-protocol.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/forces-protocol>, <mailto:forces-protocol-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/forces-protocol>
List-Post: <mailto:forces-protocol@ietf.org>
List-Help: <mailto:forces-protocol-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/forces-protocol>, <mailto:forces-protocol-request@ietf.org?subject=subscribe>
Sender: forces-protocol-bounces@ietf.org
Errors-To: forces-protocol-bounces@ietf.org
X-Spam-Score: 0.0 (/)
X-Scan-Signature: 43317e64100dd4d87214c51822b582d1
Content-Transfer-Encoding: 7bit

Weiming,

You introduce two things below:
1) content based access aka associative 
2) operations of such contents (AND, OR are the two i see you using)

We havent discussed those. But this is a good point as any to start.

Lets talk about one issue at a time

There is a TLV "KEY_INFO" Which has in its value a KEY that could be a
the ID and KEY_DATA to be the value. If we allow to have multiple of
these in a path-data then we can have complex operations.
With that in mind, let me move to the rest of your email and use this to
address your examples.

> 
On Mon, 2004-11-15 at 10:06, Weiming Wang wrote:
> Jamal,  Joel as well,
> 
> Good, I'l try to follow the example in the doc. I'm just trying to raise some
> question in the line actually trying to show my view further.
> 
> To summarize, I strongly believe the Data field should allow IDs other than pure
> values. I think in the examples Jamal also has shown some cases where Data are
> not pure values (including index, field ids).
> 

I think this should be available but optionaly i.e you could have
simple, BLOCK or KEY based accesses. It may also be ok to have a mixture
of all 3.

> Regarding using Data encoding without IDs,  I really have a feeling that it is
> more dangerous than with IDs. Note one bit error produced by programmer (e.g.,
> mistaken one u32 to u64 and another u64 to 32) will result in the total failure
> for the whole Data and the error is hard to be debugged (no IDs to verify it).
> One more question is the appearing sequence also become important with this
> approach, which makes the FE model schema sequence sensetive. This is really
> inconvenient.
> 

I dont see how. The path is still composed of IDs and subscripts/indices

> I'm trying to add Steve and Zsolt in the list (but seems Steve's server always
> takes my post as spam:( .  ).
> 
> Cheers,
> Weiming
> ---------------------
> OPER_TLV : = <PATH-DATA>
> PATH-DATA := flags IDcount <IDs> [BLOCK_OR_KEY_NOTATION] [DATA]
> [Weiming]Why don't we just set the PATH as a TLV?
> 
> -> IDs := a series of 32bit IDs (whose size is given by IDcount) defining the
> path.
> [Weiming] Obviously, we need a flag to indicate the ID being index(subscript) or
> field ID as
>     ID:= flag(1bit)  Index(31bit) | fieldID(31bit)
>     flag:=0(fieldID) | 1(index)
> [/Weiming]
> 

Lets defer your idea of decomposing the ID above for now. I think we can
still achieve the goals without it.

> Assume LFB with following attributes
> 
> foo1, type u32, ID = 1
> 
> foo2, type u32, ID = 2
> 
> table2: type array, ID = 4
>  elements are:
>  j1, type u32, ID = 1
>  j2, type u32, ID = 2
> [Weiming] Let me add one more field as
>  j3, type u32, ID=3
> 
> Then, very often, we may have operations as:
> 
> 1) delete(or get ) all entries with j1=100, and j2=200 (note j3 is not cared)

Can be done as follows:

Path = 4
TLV, T = KEY_INFO, Length = 4+8
KEY = 1 // j1
KEY_DATA = 100 
TLV, T = KEY_INFO, Length = 4+8
KEY = 2 // j2
KEY_DATA = 200  

The above is an && operation of the two keys. Not sure how to do ||
cleverly

Other 
Of course we could make KEY_DATA a TLV so we can have variable sized
DATA for a KEY, eg to repeat above:

Path=4
T=KEY_INFO, length = 4 + xx
   T=1,  Length= 4 + 4 // 1 is ID for j1
   V = 100
   T=2, L = 4 + 4
   V = 200

The above wastes 4 bytes per value because of TLV and is clearly more
complex to parse, but we could say thats the cost of doing this.
   

> 2) modify entry with index=10 to j1=100, and j2=200 (note j3 is not to be
> changed)
> 

Can use one of those two schemes above.

> If we allow field ID in Data, this can be expressed easily. I just see some
> difficulty using the encoding you and Joel proposed. For 2) you may say it can
> be done by using two separate Path-data, but I just think this way is very
> tedious, saying if we have 10 fields more. If we allow field ID in data, it can
> be tidily expressed.
> [/Weiming]
> 

I think what you have described is content based schemes - which is not
exclusive. Both schemes need to be supported.

> table1: type array, ID = 3
>  elements are:
>  t1, type u32, ID = 1
>  t2, type u32, ID = 2  // index into table 2
> 
> [Weiming] I think a compund structural type is very common, which should be
> instanced  in the example.  Let me show one as:
> 
> foo3, type structure, ID=5
> {
>     s1, type u32, ID=1
>     s2, type structure, ID=2
>     {
>         g1, type u32, ID=1
>         g2, type u32, ID=2
>     }
>  }
> 
> For this type, we then may also have very common operations as:
>         Modify s1 to 100 and g1 to 200.
> I also see some tidious problem if we don't allow field ID in the DataTLV.
> [/Weiming]

We could add that as another example; Lets come back to it when we have
solved some of the basic stuff

> 1) To get foo1
> Path IDs = 1
> 
> Result:
> Path = 1, DATA_TLV with value of foo1
> 
> 2) to set foo2 to 10
> Path IDs = 2
> DATA_TLV: L = 4+4, V=10
> 
> 3) To dump table2 (get)
> Path = 4
> 
> Result:
> Path = 4
> DATA_TLV, Length = 4 + (#of entries in table2)(8+4)
> This is followed by data of the form:
> subscript,j1value,j2value (subscript is 4 bytes, j1 + j2 is 8 bytes)
> 
> Note the above allows for holes in subscripts, i.e if subscript is
> not in use it doesnt take space
> [Weiming] Note that some tables may or may not have subscript, above format will
> then result in ambiguity in semantics. If we uniformly ask IDs in Data, no such
> problem then.

Forget about IDs for a second - I dont think we need them for this
specific case - and make a recommendation about how to word this.

Again, to reiterate both content and the other schemes need to be
supported. 

cheers,
jamal


_______________________________________________
Forces-protocol mailing list
Forces-protocol@ietf.org
https://www1.ietf.org/mailman/listinfo/forces-protocol