Re: [vwrap] Status and future of the VWRAP working group

[Meadhbh Hamrick <[e-mail]>]

actually protobufs says quite a bit about transfer syntax. as does LLSD.

here's a link to an overview of ProtoBuf line encoding.

if you read the document, the entire back half of the document
describes serialization schemes used to concretize abstract data for
transmission over the network. since we use JSON and XML for 2/3 of
our wire format specification, we reference the ECMAscript and XML
standards rather than replicate them in the LLSD drafts.

LLSD and ProtoBufs ARE both transport agnostic, though the only
defined interoperability profile for LLSD was transporting messages
over HTTP. some people said they were interested in XMPP, but they
never produced any documentation for it.

LLSD defines multiple transfer syntaxes because there was a
requirement that it work with existing XML and JSON tools. The binary
format was something added to save space on disk and in packets,
though honestly, transferring XML or JSON with gzip content encoding
gets you to encoding sizes that are pretty close to binary encoding.

and yes, if you use JSON, you have to deal with JSON's flaws. but
guess what? if you use XML, you have to deal with XML's flaws. but at
the end of the day, some people use XML and some people use JSON.
we're not going to tell you which to use, because ultimately it's
pretty insignificant compared to the semantics of messages that flow
between services.

but sure, if you're down with having an argument about why we should
all use ProtocolBuffers, be my guest. I'll check in with you in a
couple years and see how close you are to consensus.

-cheers
-meadhbh
--
meadhbh hamrick * it's pronounced "maeve"
@OhMeadhbh * http://meadhbh.org/ * [e-mail]



On Sat, Mar 26, 2011 at 11:57 AM, Izzy Alanis <[e-mail]> wrote:
> I certainly am confused about what you want LLSD to be.
>
> Many protocols need mechanisms for both describing structured data in a language/technology neutral way (via interface definition language) and a concrete serialized form or forms -- primarily used for transfer, but also for storage/retrieval, archival purposes, etc, etc, etc.
>
> Protobuffs doesn't say anything about transfer protocols (neither does LLSD), protobuffs doesn't say anything about language/technology stack requirements (neither does LLSD).
>
>> my experience with
>> large systems and protobufs is you have to make everything optional to
>> cover the use case of two systems running slightly different versions
>> of an interface.
>
> How was LLSD's extension mechanism different? LLSD and protobuff are not that different except that LLSD allows for multiple serialization forms, which (IMHO) adds confusion and complexity.
>
> For example, serializing LLSD over JSON means you need a specialized LLSD-JSON serializer/deserializer. Why? Because the serializer needs to follow the LLSD imposed rules about default values, required data fields, etc. Explain that to a programmer who thinks, "yay it's JSON, I know how to parse that!" Well, you can use that JSON parser, but you'll end up with incomplete data structs.
>
> - Izzy
>
> On Mar 26, 2011, at 1:31 PM, Meadhbh Hamrick <[e-mail]> wrote:
>
>> fwiw.. i think this discussion indicates a little confusion over what LLSD is.
>>
>> LLSD is not a transfer syntax. it is an abstract type system and
>> interface description language that defines three transfer syntaxes.
>> ProtoBufs is not an abstract type system as it's type semantics are
>> not defined. The objective of LLSD was to allow us to describe
>> services independent of any particular technology.
>>
>> sadly, that was never made abundantly clear in the docs. but in the
>> list discussion, many of us who had used LLSD mentioned several times
>> that we were focusing on LLSD serialized with JSON or XML over
>> HTTP(S), but there was no reason it couldn't be carried over XMPP or
>> ProtoBufs. it's just that no one seemed to think those use cases were
>> interesting enough to sit down with a text editor and whip up some
>> code.
>>
>> so... even if you dump LLSD moving forward, i would encourage you to
>> not use ProtoBufs to define service interfaces. my experience with
>> large systems and protobufs is you have to make everything optional to
>> cover the use case of two systems running slightly different versions
>> of an interface.
>>
>> that being said, producing a map from LLSD to ProtoBufs should be
>> pretty straight forward.
>>
>> but that's just my 2 cents.
>> --
>> meadhbh hamrick * it's pronounced "maeve"
>> @OhMeadhbh * http://meadhbh.org/ * [e-mail]
>>
>>
>>
>> On Sat, Mar 26, 2011 at 8:54 AM, Izzy Alanis <[e-mail]> wrote:
>>> +1 to: put aside the existing stuff
>>>
>>> For the purposes of the Intro doc, I think it would be possible to
>>> table the decision between LLSD/LLIDL and something else (protobuf)
>>> for the abstract type system / interface descriptions *if* the doc was
>>> couched in terms of a VWRAP-IDL that we then specify/decide on in the
>>> following IDL/type system doc.
>>>
>>> One thing we should make a decision on, and that I think is fuzzy in
>>> the draft-hamrick-vwrap-intro-01 doc, is whether the serializations
>>> are negotiated between services, or whether VWRAP mandates LLSD as the
>>> only approved serialization.
>>>
>>> Sections 2.3.1:
>>>  "Web-based applications may choose to use JSON or XML.
>>> Server-to-server interactions may use the VWRAP specific binary
>>> serialization scheme..."
>>>
>>> Seems to conflict with (2.3.3):
>>>  " VWRAP uses the LLSD abstract type system and the LLIDL interface
>>>       description language to describe the structure and type semantics
>>>       of elements in messages sent between systems.  Because LLSD makes
>>>       extensive use of variable width, clearly delineated data fields,
>>>       services which consume protocol messages may identify and extract
>>>       only those message elements they know how to handle."
>>>
>>> I'd like to take this opportunity to point out that protobuff supports
>>> the same variable width, extensibility, default values, yada yada
>>> yada.
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> we could couch that in terms of only the IDL, and leave serialization
>>> of messages up to content negotiation between services involved.
>>>
>>> On Sat, Mar 26, 2011 at 11:03 AM, Patnad Babii <[e-mail]> wrote:
>>>> What we should focus on, is this I believe, are we going to keep LLSD (is it
>>>> good enought). Is it really needed or shall we start from scratch, with the
>>>> aim of using some new viewer / browser.
>>>>
>>>> Opensim (open source virtual platform with alot of grids already) is using
>>>> LLSD, especially because the viewer LL made is using LLSD.
>>>>
>>>> To be honest I keep saying since the beginning of these working groups (the
>>>> various ones, ogpx, vwrap, mmox..) we should put aside the existing stuff,
>>>> work more with Opensim, since they are the key to this new journey that is
>>>> open virtual worlds, its good they have HyperGrid already which does some
>>>> amazing stuff!
>>>>
>>>> What we need is a consensus, something we all agree on and go forward with
>>>> the docs.
>>>>
>>>> -----Message d'origine----- From: Barry Leiba
>>>> Sent: Saturday, March 26, 2011 10:40 AM
>>>> To: Carlo Wood
>>>> Cc: [e-mail]
>>>> Subject: Re: [vwrap] Status and future of the VWRAP working group
>>>>
>>>> Hi, Carlo, Vaughn, Izzy, Dzonatas, and everyone else.
>>>> I'm glad to see some discussion going.  Izzy is right when he says this:
>>>>>
>>>>> Of course this all may not matter unless somebody actually addresses
>>>>> the underlying issue: That the group needs to start producing documents.
>>>>
>>>> Indeed.  We, the chairs, need to see real progress on the documents...
>>>> particularly the introduction document.
>>>>
>>>> This discussion is a start, and, as I say, I'm pleased to see it, but
>>>> it has to turn into real document editing very soon.
>>>>
>>>> I want to say one other thing:
>>>>
>>>>> Oops, no I said 'we'... but count me out. I still see the same people
>>>>> around here and it's still going to fail just as bad as last time (or,
>>>>> as I expected the first time around... some "standard" is going to
>>>>> be produced that sucks; and that is either going to be ignored, or
>>>>> adopted by a few large "players" who then all get major head aches
>>>>> and problems that they can't fix; and in the end it will be the users
>>>>> who suffer most from having a bad protocol of course :/.
>>>>
>>>> Carlo, Meadhbh is still around, thought she (note gender) has given up
>>>> the editorship of the documents.  Your input is welcome -- encouraged
>>>> -- though, of course, if you choose not to participate for whatever
>>>> reason, that's your choice.  I think choosing not to participate
>>>> because some particular person is also participating is a poor choice,
>>>> but it's your choice to make.  I would like to see you reconsider
>>>> that, if you're willing to do the work.
>>>>
>>>> What I do *not* want to see, and what we won't tolerate, are personal
>>>> attacks on any participant here.  Do not engage in name-calling, do
>>>> not question people's integrity, do not malign the companies they work
>>>> for, and do not accuse people of malfeasance because they disagree
>>>> with you.  If you present your ideas and others agree with what you
>>>> say, those ideas will make it forward.  That's how we aim to work,
>>>> here, and if this working group can continue and make progress, that's
>>>> indeed how we'll work.
>>>>
>>>> So... will we make some progress on the intro document?  Can we get
>>>> some real discussion on it, and a draft that shows some level of
>>>> consensus within the nest few weeks?
>>>>
>>>> Barry, as chair
>>>> _______________________________________________
>>>> vwrap mailing list
>>>> [e-mail]
>>>> https://www.ietf.org/mailman/listinfo/vwrap
>>>> _______________________________________________
>>>> vwrap mailing list
>>>> [e-mail]
>>>> https://www.ietf.org/mailman/listinfo/vwrap
>>>>
>>> _______________________________________________
>>> vwrap mailing list
>>> [e-mail]
>>> https://www.ietf.org/mailman/listinfo/vwrap
>>>
>