Re: [Forces-protocol] Data Encoding

"Joel M. Halpern" <joel@stevecrocker.com> Tue, 16 November 2004 20:16 UTC

Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA08017 for <forces-protocol-web-archive@ietf.org>; Tue, 16 Nov 2004 15:16:52 -0500 (EST)
Received: from megatron.ietf.org ([132.151.6.71]) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CU9nC-0006t7-PB for forces-protocol-web-archive@ietf.org; Tue, 16 Nov 2004 15:19:14 -0500
Received: from localhost.localdomain ([127.0.0.1] helo=megatron.ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CU9j5-0004Ny-LK; Tue, 16 Nov 2004 15:14:55 -0500
Received: from odin.ietf.org ([132.151.1.176] helo=ietf.org) by megatron.ietf.org with esmtp (Exim 4.32) id 1CU9d5-0001gX-LD for forces-protocol@megatron.ietf.org; Tue, 16 Nov 2004 15:08:45 -0500
Received: from ietf-mx.ietf.org (ietf-mx.ietf.org [132.151.6.1]) by ietf.org (8.9.1a/8.9.1a) with ESMTP id PAA06770 for <forces-protocol@ietf.org>; Tue, 16 Nov 2004 15:08:42 -0500 (EST)
Received: from ns.execdsl.net ([208.184.15.238] helo=EXECDSL.COM) by ietf-mx.ietf.org with esmtp (Exim 4.33) id 1CU9f2-0006fK-ET for forces-protocol@ietf.org; Tue, 16 Nov 2004 15:11:05 -0500
Received: from [64.254.114.114] (HELO JLaptop.stevecrocker.com) by EXECDSL.COM (CommuniGate Pro SMTP 3.3) with ESMTP id 7960945; Tue, 16 Nov 2004 15:08:15 -0500
Message-Id: <6.1.2.0.0.20041116150436.03b89ec0@localhost>
X-Sender: joel@stevecrocker.com@localhost
X-Mailer: QUALCOMM Windows Eudora Version 6.1.2.0
Date: Tue, 16 Nov 2004 15:07:54 -0500
To: hadi@znyx.com, Weiming Wang <wmwang@mail.hzic.edu.cn>
From: "Joel M. Halpern" <joel@stevecrocker.com>
Subject: Re: [Forces-protocol] Data Encoding
In-Reply-To: <1100610279.1152.33.camel@jzny.localdomain>
References: <4189F776.4080306@zurich.ibm.com> <00bd01c4c536$fb418ee0$020aa8c0@wwm1> <1099885892.2167.13.camel@jzny.localdomain> <132001c4c551$86023150$845c21d2@Necom.hzic.edu.cn> <1099911200.2169.29.camel@jzny.localdomain> <134f01c4c585$216584c0$845c21d2@Necom.hzic.edu.cn> <4191299F.4020809@zurich.ibm.com> <142a01c4c6d6$13569980$845c21d2@Necom.hzic.edu.cn> <1100100893.2210.24.camel@jzny.localdomain> <14fc01c4c79f$75231f20$845c21d2@Necom.hzic.edu.cn> <6.1.2.0.0.20041111091015.03bafec0@localhost> <155001c4c8c0$5eb73ec0$845c21d2@Necom.hzic.edu.cn> <1100269665.1106.30.camel@jzny.localdomain> <156501c4c8c5$1fb19220$845c21d2@Necom.hzic.edu.cn> <1100271071.1109.38.camel@jzny.localdomain> <1100307886.1110.20.camel@jzny.localdomain> <6.1.2.0.0.20041112220931.03c89ec0@localhost> <009a01c4c934$99cf7990$020aa8c0@wwm1> <6.1.2.0.0.20041112230835.03f0b0f0@localhost> <1100381538.2212.914.camel@jzny.localdomain> <6.1.2.0.0.20041113170557.03b05e00@localhost> <1100458281.2327.23.camel@jzny.l! ocaldomain> <009e01c4cb24$ba144bd0$020aa8c0@wwm1> <1100610279.1152.33.camel@jzny.localdomain>
Mime-Version: 1.0
Content-Type: text/plain; charset="us-ascii"; format=flowed
X-Spam-Score: 0.0 (/)
X-Scan-Signature: bc6181926481d86059e678c9f7cb8b34
Cc: "Khosravi, Hormuzd M" <hormuzd.m.khosravi@intel.com>, "\(Ram Gopal \)" <ram.gopal@nokia.com>, Avri Doria <avri@acm.org>, forces-protocol@ietf.org, Patrick Droz <dro@zurich.ibm.com>, zsolt@modularnet.com, Steve Blake <slblake@petri-meat.com>, David.Putzolu@intel.com, Dong Ligang <donglg@mail.hzic.edu.cn>, Robert Haas <rha@zurich.ibm.com>
X-BeenThere: forces-protocol@ietf.org
X-Mailman-Version: 2.1.5
Precedence: list
List-Id: forces-protocol <forces-protocol.ietf.org>
List-Unsubscribe: <https://www1.ietf.org/mailman/listinfo/forces-protocol>, <mailto:forces-protocol-request@ietf.org?subject=unsubscribe>
List-Archive: <http://www1.ietf.org/pipermail/forces-protocol>
List-Post: <mailto:forces-protocol@ietf.org>
List-Help: <mailto:forces-protocol-request@ietf.org?subject=help>
List-Subscribe: <https://www1.ietf.org/mailman/listinfo/forces-protocol>, <mailto:forces-protocol-request@ietf.org?subject=subscribe>
Sender: forces-protocol-bounces@ietf.org
Errors-To: forces-protocol-bounces@ietf.org
X-Spam-Score: 0.0 (/)
X-Scan-Signature: 4b66a1e94d7d92973ece9e5da449ff80

The more I think about it, the more I like three sets of information,
first the base path (length, sequence of identifiers)
then one or more TLVs for the subscripts, where the TLV wil contain an 
index identifier and the data to be used as a subscript.  This may be the 
base subscript, or some other field.  (The identifier is a field identifier 
rather than a type identifier since there might be two key fields for the 
same table with the same type, but differing values.)
Then for modify operations (and GET responses?) a TLV for the actual data.
If we want to allow block references, then the structure of the subscript 
TLV can be expanded to indicate ranges somehow.
This keeps the notions of "what structural thing", "which instance", and 
"what value" cleanly separated.

Yours,
Joel

At 08:04 AM 11/16/2004, Jamal Hadi Salim wrote:
>Weiming,
>
>You introduce two things below:
>1) content based access aka associative
>2) operations of such contents (AND, OR are the two i see you using)
>
>We havent discussed those. But this is a good point as any to start.
>
>Lets talk about one issue at a time
>
>There is a TLV "KEY_INFO" Which has in its value a KEY that could be a
>the ID and KEY_DATA to be the value. If we allow to have multiple of
>these in a path-data then we can have complex operations.
>With that in mind, let me move to the rest of your email and use this to
>address your examples.
>
> >
>On Mon, 2004-11-15 at 10:06, Weiming Wang wrote:
> > Jamal,  Joel as well,
> >
> > Good, I'l try to follow the example in the doc. I'm just trying to 
> raise some
> > question in the line actually trying to show my view further.
> >
> > To summarize, I strongly believe the Data field should allow IDs other 
> than pure
> > values. I think in the examples Jamal also has shown some cases where 
> Data are
> > not pure values (including index, field ids).
> >
>
>I think this should be available but optionaly i.e you could have
>simple, BLOCK or KEY based accesses. It may also be ok to have a mixture
>of all 3.
>
> > Regarding using Data encoding without IDs,  I really have a feeling 
> that it is
> > more dangerous than with IDs. Note one bit error produced by programmer 
> (e.g.,
> > mistaken one u32 to u64 and another u64 to 32) will result in the total 
> failure
> > for the whole Data and the error is hard to be debugged (no IDs to 
> verify it).
> > One more question is the appearing sequence also become important with this
> > approach, which makes the FE model schema sequence sensetive. This is 
> really
> > inconvenient.
> >
>
>I dont see how. The path is still composed of IDs and subscripts/indices
>
> > I'm trying to add Steve and Zsolt in the list (but seems Steve's server 
> always
> > takes my post as spam:( .  ).
> >
> > Cheers,
> > Weiming
> > ---------------------
> > OPER_TLV : = <PATH-DATA>
> > PATH-DATA := flags IDcount <IDs> [BLOCK_OR_KEY_NOTATION] [DATA]
> > [Weiming]Why don't we just set the PATH as a TLV?
> >
> > -> IDs := a series of 32bit IDs (whose size is given by IDcount) 
> defining the
> > path.
> > [Weiming] Obviously, we need a flag to indicate the ID being 
> index(subscript) or
> > field ID as
> >     ID:= flag(1bit)  Index(31bit) | fieldID(31bit)
> >     flag:=0(fieldID) | 1(index)
> > [/Weiming]
> >
>
>Lets defer your idea of decomposing the ID above for now. I think we can
>still achieve the goals without it.
>
> > Assume LFB with following attributes
> >
> > foo1, type u32, ID = 1
> >
> > foo2, type u32, ID = 2
> >
> > table2: type array, ID = 4
> >  elements are:
> >  j1, type u32, ID = 1
> >  j2, type u32, ID = 2
> > [Weiming] Let me add one more field as
> >  j3, type u32, ID=3
> >
> > Then, very often, we may have operations as:
> >
> > 1) delete(or get ) all entries with j1=100, and j2=200 (note j3 is not 
> cared)
>
>Can be done as follows:
>
>Path = 4
>TLV, T = KEY_INFO, Length = 4+8
>KEY = 1 // j1
>KEY_DATA = 100
>TLV, T = KEY_INFO, Length = 4+8
>KEY = 2 // j2
>KEY_DATA = 200
>
>The above is an && operation of the two keys. Not sure how to do ||
>cleverly
>
>Other
>Of course we could make KEY_DATA a TLV so we can have variable sized
>DATA for a KEY, eg to repeat above:
>
>Path=4
>T=KEY_INFO, length = 4 + xx
>    T=1,  Length= 4 + 4 // 1 is ID for j1
>    V = 100
>    T=2, L = 4 + 4
>    V = 200
>
>The above wastes 4 bytes per value because of TLV and is clearly more
>complex to parse, but we could say thats the cost of doing this.
>
>
> > 2) modify entry with index=10 to j1=100, and j2=200 (note j3 is not to be
> > changed)
> >
>
>Can use one of those two schemes above.
>
> > If we allow field ID in Data, this can be expressed easily. I just see some
> > difficulty using the encoding you and Joel proposed. For 2) you may say 
> it can
> > be done by using two separate Path-data, but I just think this way is very
> > tedious, saying if we have 10 fields more. If we allow field ID in 
> data, it can
> > be tidily expressed.
> > [/Weiming]
> >
>
>I think what you have described is content based schemes - which is not
>exclusive. Both schemes need to be supported.
>
> > table1: type array, ID = 3
> >  elements are:
> >  t1, type u32, ID = 1
> >  t2, type u32, ID = 2  // index into table 2
> >
> > [Weiming] I think a compund structural type is very common, which should be
> > instanced  in the example.  Let me show one as:
> >
> > foo3, type structure, ID=5
> > {
> >     s1, type u32, ID=1
> >     s2, type structure, ID=2
> >     {
> >         g1, type u32, ID=1
> >         g2, type u32, ID=2
> >     }
> >  }
> >
> > For this type, we then may also have very common operations as:
> >         Modify s1 to 100 and g1 to 200.
> > I also see some tidious problem if we don't allow field ID in the DataTLV.
> > [/Weiming]
>
>We could add that as another example; Lets come back to it when we have
>solved some of the basic stuff
>
> > 1) To get foo1
> > Path IDs = 1
> >
> > Result:
> > Path = 1, DATA_TLV with value of foo1
> >
> > 2) to set foo2 to 10
> > Path IDs = 2
> > DATA_TLV: L = 4+4, V=10
> >
> > 3) To dump table2 (get)
> > Path = 4
> >
> > Result:
> > Path = 4
> > DATA_TLV, Length = 4 + (#of entries in table2)(8+4)
> > This is followed by data of the form:
> > subscript,j1value,j2value (subscript is 4 bytes, j1 + j2 is 8 bytes)
> >
> > Note the above allows for holes in subscripts, i.e if subscript is
> > not in use it doesnt take space
> > [Weiming] Note that some tables may or may not have subscript, above 
> format will
> > then result in ambiguity in semantics. If we uniformly ask IDs in Data, 
> no such
> > problem then.
>
>Forget about IDs for a second - I dont think we need them for this
>specific case - and make a recommendation about how to word this.
>
>Again, to reiterate both content and the other schemes need to be
>supported.
>
>cheers,
>jamal


_______________________________________________
Forces-protocol mailing list
Forces-protocol@ietf.org
https://www1.ietf.org/mailman/listinfo/forces-protocol