Re: [vwrap] [wvrap] Simulation consistency

[Morgaine <[e-mail]>]

We have no "figure-of-speech services" in VWRAP.  Our services are running
processes, they have network endpoints, and they talk a network protocol.

Concepts can be very useful, and I regularly draw fluffy clouds that
represent the concept behind something concrete.

Where concepts are not useful is when they denote something that doesn't
actually exist, like say phlogiston, God, or Agent Domain, because this just
makes them a hindrance to further progress.  Humanity suffers from this a
lot.

Philosophy aside, we want our Agent Service to be totally concrete, with a
network API and well defined semantics.  The domain concept does not get us
there.


Morgaine.




==================

On Sun, Apr 3, 2011 at 6:38 PM, Dzonatas Sol <[e-mail]> wrote:

> I prefer the visualization of "domains" because those fluffy clouds really
> help those that don't do well with figure-of-speech "services."
>
>
> Morgaine wrote:
>
>> "Agent Domain" more or less ceased to exist in practice when David pointed
>> out very eloquently that the emperor had no clothes.  (Same for "Region
>> Domain".)
>>
>> I think we mostly talk about the Agent Service and Region Service these
>> days.  The "domain" was just a fluffy cloud that someone once drew on a
>> whiteboard, but which never actually existed.
>>
>>
>> Morgaine.
>>
>>
>>
>>
>> ====================
>>
>> On Sun, Apr 3, 2011 at 6:20 PM, Dzonatas Sol <[e-mail] <mailto:
>> [e-mail]>> wrote:
>>
>>    Probably easy as suggested in other terms here on this list, as
>>    how the client contacts the asset services now in the regions. The
>>    newer issue is to unitize that asset services. Since their is
>>    proprietary (legacy) code then we can't expect that to change, and
>>    some form of proxy is of need. Whatever works best, I tried to
>>    narrow it down to suggestions here.
>>
>>    Eventually, the agent domain is ideal to handle the direction of
>>    the asset services. This concept, unfortunately, ended support
>>    awhile ago with changes in LL.
>>    Also see; http://wiki.secondlife.com/wiki/Agent_Domain
>>    And: http://wiki.secondlife.com/wiki/User:Dzonatas_Sol/AWG_Asset
>>    (warn: unstructured collaborative notes, dumped on me and I tried
>>    to fix)
>>
>>    I tried to find previous visuals.
>>
>>    I'd imagine the agent domain could grow out of unitized versions
>>    of asset services. Despite that, I think that concept helps view
>>    where we were at in discussion and what didn't happen.
>>
>>    Vaughn Deluca wrote:
>>
>>        Hi�Dzonatas
>>
>>        Can you expand on that, what would be needed for legacy
>>        support in VWAP terms�?,
>>        If i want to read up on how the�asset server may proxy the
>>        simulator, what would you recommend me to read?
>>
>>        -- Vaughn
>>
>>        On Sun, Apr 3, 2011 at 5:51 AM, Dzonatas Sol
>>        <[e-mail] <mailto:[e-mail]>
>>        <mailto:[e-mail] <mailto:[e-mail]>>> wrote:
>>
>>           Some stated the proxy-to-asset-server is built into the sim;
>>           however, keep in mind possible legacy support where the asset
>>           server may proxy the simulator.
>>
>>
>>           Dzonatas Sol wrote:
>>
>>               Somehow I feel the basic asset server being able to
>>        login and
>>               download assets is now priority, yet I also wondered
>>        the best
>>               way to patch this into the current mode of viewers.
>>
>>               Maybe offer (1) by proxy (sim-side) and (2) by patch
>>               (viewer-side) that either of these two are optional and
>>               neither are mandatory for now. Thoughts?
>>
>>               Israel Alanis wrote:
>>
>>
>>                   > when designing for scalability, the model to bear in
>>                   mind is ...
>>
>>                   Well, there are a lot of different models to keep
>>        in mind,
>>                   and many different use cases. One particular use
>>        case to
>>                   keep in mind is: "User acquires new outfit, and
>>        wants to
>>                   'show it off' in a highly populated region".
>>
>>                   > Both worlds and asset services may include
>>        commercial,
>>                   community, and personal services
>>
>>                   Yes, yes and yes. I'm particularly concerned about
>>        how the
>>                   model affects the ability to host personal asset
>>        services.
>>
>>                   > a proxying region, which would get slammed for every
>>                   asset worn by every avatar present.
>>
>>                   Granted the collection of services that are provided by
>>                   the region need to be scaled to meet the demands of
>>        that
>>                   region. That's all part of capacity planning.
>>
>>                   > regions run many different CPU-intensive tasks,
>>                   including physics simulation and server-side scripting,
>>                   and absolutely cannot afford to serve assets too
>>                   Well... who said the same CPU's have to do proxying,
>>                   physics simulation and server-side scripting? Asset
>>                   proxying is a different service than physics simulation
>>                   and can be on separate hardware, could make use of
>>                   geographically distributed caching, and in certain
>>                   deployment patterns, the same caching services could be
>>                   shared by different regions. (Server-side scripting
>>        is a
>>                   discussion for another day).
>>
>>                   > This is why we have to go parallel...
>>
>>                   Totally agree, and a proxying model could and
>>        should also
>>                   take advantage of parallelism.
>>
>>                   > I think you're wrong that it has to cost much
>>        money. ?vs?
>>                   > It costs money to host a high performance and
>>        scalable
>>                   asset service and a high bandwidth network to
>>        handle the
>>                   traffic. �A *lot* of money.
>>                   I think what you're saying is: "It costs a lot of
>>        money to
>>                   build a scalable asset service, but if assets are
>>        spread
>>                   throughout the internet they don't have to be
>>        scalable."
>>                   But that's not quite right. You're opening up every
>>        asset
>>                   server to the VW equivalent of being slashdotted,
>>        so are
>>                   you sure you're not forcing *every* asset service to be
>>                   scalable and handle a lot of bandwith and network
>>        traffic?
>>                   It's the exact opposite of your intention, but I think
>>                   that's the result, all the same.
>>
>>                   This particular design decision has a big effect on the
>>                   economics of the VW infrastructure. I'd rather the
>>                   economics to work out such that a region provider who
>>                   wishes to build a region that supports a small
>>        population,
>>                   can do so economically. A region that wants to host a
>>                   *large* population has to bear that cost of
>>        providing that
>>                   scalable asset service.
>>                   I want the economics of hosting a small asset
>>        service to
>>                   be a non-issue (as to best promote creation and
>>                   creativity). Creating a high bar to provide asset
>>        services
>>                   will mean that service will cost money and people
>>                   shouldn't have to pay money just to create or own VW
>>                   objects (I'm using 'own' here to refer to maintaining
>>                   their existence, I'm not trying to make a
>>                   'leftist'/'communist' statement about ownership ;)
>>
>>                   - Izzy
>>
>>
>>                   On Apr 2, 2011, at 3:58 PM, Morgaine wrote:
>>
>>                       Izzy, when designing for scalability, the model to
>>                       bear in mind is that of seasoned virtual world
>>                       travelers whose inventories contain assets from
>>        many
>>                       different worlds, those assets being served by many
>>                       different asset services. �Both worlds and asset
>>                       services may include commercial, community, and
>>                       personal services, and as the metaverse grows, that
>>                       set is highly likely to become progressively less
>>                       clustered and more diverse.
>>
>>                       When those seasoned travelers click on an
>>        advertised
>>                       VW link and perform an inter-world teleport to one
>>                       particular world's region to share an experience,
>>                       their "worn" assets (the only ones of interest
>>        to the
>>                       region) will contain references to asset services
>>                       spread widely across the Internet. �The fetches
>>        by the
>>                       travelers' clients occur over many parallel
>>        paths from
>>                       clients to asset services, so one can reasonably
>>                       expect reduced network contention and reduced asset
>>                       server loading because they are both spread out
>>        over
>>                       however many asset services are being referenced by
>>                       the overall set of assets in the region.
>>
>>                       This is very different to the case of a proxying
>>                       region, which would get slammed for every asset
>>        worn
>>                       by every avatar present. �In our current
>>        architecture,
>>                       regions run many different CPU-intensive tasks,
>>                       including physics simulation and server-side
>>                       scripting, and absolutely cannot afford to serve
>>                       assets too unless your scalability requirements are
>>                       very low indeed, ie. just a few dozen avatars of
>>                       today's kind. �We've hit the ceiling already on
>>        region
>>                       scalability done that way. �There is nowhere to
>>        go in
>>                       that direction at all beyond improving the code
>>        like
>>                       Intel demonstrated, and that work is subject to
>>        a law
>>                       of diminishing returns.
>>
>>                       This is why we have to go parallel, and I think
>>        you're
>>                       wrong that it has to cost much money. �As we spread
>>                       the load across more and more asset services,
>>        we are
>>                       simply better utilizing all the hardware that's
>>                       already out there on the Internet, at least in
>>        respect
>>                       of community and private resources. �But add to the
>>                       community and private resources the commercial
>>        asset
>>                       services that are likely to appear to exploit this
>>                       opportunity, and not only will the number of asset
>>                       services leap, but the power of each one will
>>        rocket
>>                       too, because, after all, these businesses will be
>>                       heavily optimized for the job.
>>
>>                       As to why a world would want clients to access
>>                       external asset services instead of providing
>>        its own
>>                       implementation, that's an easy question. �It costs
>>                       money to host a high performance and scalable asset
>>                       service and a high bandwidth network to handle the
>>                       traffic. �A *lot* of money. �In contrast, it
>>        costs a
>>                       world nothing to let others serve the assets to
>>                       clients. �And that matters to the bottom line.
>>
>>
>>                       Morgaine.
>>
>>
>>
>>
>>                       ======================
>>
>>                       On Sat, Apr 2, 2011 at 7:05 PM, Izzy Alanis
>>                       <[e-mail]
>>        <mailto:[e-mail]> <mailto:[e-mail]
>>        <mailto:[e-mail]>>
>>                       <mailto:[e-mail]
>>        <mailto:[e-mail]>
>>
>>                       <mailto:[e-mail]
>>        <mailto:[e-mail]>>>> wrote:
>>
>>                       � �> As always though, it's a trade-off, since the
>>                       proxied design
>>                       � �has very poor scalability compared to the
>>                       distributed one.
>>
>>                       � �I don't agree with that... If a user enters a
>>                       highly populated
>>                       � �region,
>>                       � �every other client is going to (could and
>>        should be
>>                       trying to)
>>                       � �hit the
>>                       � �asset server(s) for the assets that the user is
>>                       wearing (assuming
>>                       � �they're not cached locally). �Every asset server
>>                       has to be scaled up
>>                       � �to the point that it can handle that load
>>        from all
>>                       over...
>>
>>                       � �If I'm hosting a region that supports 10s of
>>                       thousands of
>>                       � �simultaneous
>>                       � �users (thinking of the future), I already
>>        have to
>>                       scale to meet that
>>                       � �demand. If the region is proxying the
>>        assets, then,
>>                       yes the
>>                       � �region has
>>                       � �to be scaled to meet that asset demand too,
>>        but it
>>                       already has to be
>>                       � �scaled to meet other demands of being a region
>>                       server... and why is
>>                       � �scaling those asset proxy services hard? �It's
>>                       going to cost $,
>>                       � �but is
>>                       � �not technically challenging. So, if I want
>>        to host
>>                       a region like
>>                       � �that... sure it will cost me, but the simulation
>>                       will be consistent
>>                       � �and users will be able to participate equally,
>>                       regardless of the
>>                       � �capabilities of their individual asset services.
>>
>>
>>
>>
>>                       � �On Fri, Apr 1, 2011 at 11:55 PM, Morgaine
>>                       � �<[e-mail]
>>        <mailto:[e-mail]>
>>                       <mailto:[e-mail]
>>        <mailto:[e-mail]>>
>>                       � �<mailto:[e-mail]
>>
>>        <mailto:[e-mail]>
>>
>>                       <mailto:[e-mail]
>>        <mailto:[e-mail]>>>> wrote:
>>                       � �> Every design choice results in a trade-off,
>>                       Vaughn, improving
>>                       � �one thing at
>>                       � �> the expense of something else. �If every
>>        time we
>>                       offered a
>>                       � �service we had to
>>                       � �> inform its users about the downsides of
>>        all the
>>                       trade-offs we
>>                       � �have made,
>>                       � �> they would have an awful lot to read. ;-)
>>                       � �>
>>                       � �> The specific trade-off that you are
>>        discussing is no
>>                       � �different. �A region
>>                       � �> that proxies all content has the "benefit" of
>>                       acquiring control
>>                       � �from the
>>                       � �> asset servers over the items in the region, so
>>                       that it can
>>                       � �ensure that
>>                       � �> everyone in the region not only obtains
>>        the items
>>                       but obtains
>>                       � �the same items
>>                       � �> as everyone else. �That does indeed provide a
>>                       greater guarantee of
>>                       � �> consistency than a deployment in which the
>>        region
>>                       only passes
>>                       � �asset URIs to
>>                       � �> clients who then fetch the items from
>>        asset services
>>                       � �separately. �As always
>>                       � �> though, it's a trade-off, since the proxied
>>                       design has very
>>                       � �poor scalability
>>                       � �> compared to the distributed one.
>>                       � �>
>>                       � �> If we're going to warn users of the
>>        potential for
>>                       inconsistency
>>                       � �in the
>>                       � �> distributed deployment as you suggest, are we
>>                       also going to
>>                       � �warn them of
>>                       � �> non-scalability in the proxied one? �I really
>>                       don't see much
>>                       � �merit in the
>>                       � �> idea of warning about design choices.
>>        �Many such
>>                       choices are
>>                       � �technical, and
>>                       � �> the issues are quite likely to be of little
>>                       interest to
>>                       � �non-technical users
>>                       � �> anyway. �In any case, the better services are
>>                       likely to provide
>>                       � �such
>>                       � �> information in their online documentation, I
>>                       would guess.
>>                       � �>
>>                       � �> You mentioned users "voting with their
>>        feet" or
>>                       choosing to
>>                       � �accept the risk
>>                       � �> of inconsistency. �Well that will happen
>>        anyway,
>>                       when services
>>                       � �fail and
>>                       � �> users get annoyed. �If some asset services
>>        refuse
>>                       to send the
>>                       � �requested
>>                       � �> items to some users, those services will get a
>>                       bad reputation
>>                       � �and people
>>                       � �> will choose different asset services instead.
>>                       �Likewise, if a
>>                       � �world service
>>                       � �> proxies everything and so it can't handle
>>        a large
>>                       number of
>>                       � �assets or of
>>                       � �> people, users will get annoyed at the lag
>>        and will go
>>                       � �elsewhere. �This user
>>                       � �> evaluation and "voting with their feet"
>>        happens
>>                       already with
>>                       � �online services
>>                       � �> all over the Internet, and I am sure that this
>>                       human process
>>                       � �will continue
>>                       � �> to work when the services are asset and region
>>                       services.
>>                       � �>
>>                       � �> Back in September 2010, I wrote this post
>>        which
>>                       proposes that
>>                       � �we use in
>>                       � �> VWRAP a form of asset addressing that provides
>>                       massive
>>                       � �scalability at the
>>                       � �> same time as a very high degree of
>>        resilience --
>>                       � �>
>>                       �
>>                              �
>> http://www.ietf.org/mail-archive/web/vwrap/current/msg00463.html
>>                       � �. �It is
>>                       � �> based on the concept of the URI containing
>>        a host
>>                       part and a
>>                       � �hash part,
>>                       � �> where the hash is generated (once, at the
>>        time of
>>                       storage to
>>                       � �the asset
>>                       � �> service) using a specified digest
>>        algorithm over
>>                       the content of
>>                       � �the asset
>>                       � �> being referenced. �You may wish to note
>>        that if
>>                       this design
>>                       � �were used, the
>>                       � �> failure of an asset service to deliver a
>>                       requested item would
>>                       � �result in a
>>                       � �> failover request for the item to one or more
>>                       backup services,
>>                       � �using the same
>>                       � �> hash part but with a different host address.
>>                       � �>
>>                       � �> This can go some way towards overcoming the
>>                       problem that you
>>                       � �think might
>>                       � �> occur when assets are fetched by clients from
>>                       asset services
>>                       � �directly.
>>                       � �> Although it won't help when the missing
>>        item is
>>                       available from
>>                       � �only a single
>>                       � �> asset service, it will help in many other
>>        cases,
>>                       and it will
>>                       � �compensate for
>>                       � �> service failures and network outages
>>                       automatically at the same
>>                       � �time.
>>                       � �>
>>                       � �> PS. This design for hash-based asset
>>        addressing
>>                       is already
>>                       � �being tested by
>>                       � �> Mojito Sorbet in her experimental world and
>>                       client. �It would give
>>                       � �> VWRAP-based worlds an improved level of
>>        service
>>                       availability,
>>                       � �so I think it
>>                       � �> should be a core feature of our protocol.
>>                       � �>
>>                       � �>
>>                       � �> Morgaine.
>>                       � �>
>>                       � �>
>>                       � �>
>>                       � �>
>>                       � �> ===========================
>>                       � �>
>>                       � �> On Fri, Apr 1, 2011 at 11:17 PM, Vaughn Deluca
>>                       � �<[e-mail]
>>        <mailto:[e-mail]>
>>                       <mailto:[e-mail]
>>        <mailto:[e-mail]>>
>>                       <mailto:[e-mail]
>>        <mailto:[e-mail]>
>>                       <mailto:[e-mail]
>>        <mailto:[e-mail]>>>>
>>                       � �> wrote:
>>                       � �>>
>>                       � �>> This is a question i discussed with Morgaine
>>                       off-list a while
>>                       � �ago (I
>>                       � �>> intended to send it to the list but
>>        pushed the
>>                       wrong button...) I
>>                       � �>> think we need to address this problem, and
>>                       decide how to deal
>>                       � �with it.
>>                       � �>>
>>                       � �>> �In Davids deployment draft, section
>>        7.3.1.1 an
>>                       overview is
>>                       � �given van
>>                       � �>> ways to deliver content to the region.
>>        One way
>>                       is only passing a
>>                       � �>> capability that allows access to (part
>>        of) the
>>                       resource:
>>                       � �>>
>>                       � �>> � � � � � 7.3.1.1. �Content delivery models
>>                       � �>> � � � � � A range of possible
>>        represenations can
>>                       be passed to
>>                       � �a region for
>>                       � �>> � � � � � simulation. [...] The other end
>>        of the
>>                       delivery spectrum
>>                       � �>> involves passing
>>                       � �>> � � � � � only a URI or capability used to
>>                       access the rendering
>>                       � �>> information and a
>>                       � �>> � � � � � collision mesh,and related data for
>>                       physical simulation.
>>                       � �>> � � � � � In such a model, the client is
>>                       responsible for
>>                       � �fetching the
>>                       � �>> additional
>>                       � �>> � � � � � information needed to render the
>>                       item's visual
>>                       � �presence from a
>>                       � �>> separate
>>                       � �>> � � � � � service. �This fetch can be done
>>                       *under the
>>                       � �credentials of the
>>                       � �>> end user*
>>                       � �>> � � � � � viewing the material [my
>>        emphasis--VD]
>>                       , and
>>                       � �divorces the
>>                       � �>> simulation from
>>                       � �>> � � � � � the trust chain needed to manage
>>                       content. �Any
>>                       � �automation
>>                       � �>> is done on a
>>                       � �>> � � � � � separate host which the content
>>                       creator or owner trusts,
>>                       � �>> interacting with the
>>                       � �>> � � � � � object through remoted interfaces.
>>                       � �>>
>>                       � �>> �I can see the need for such a setup,
>>        however, i
>>                       feel we are
>>                       � �>> unpleasantly close to a situation were the
>>                       coherence of the
>>                       � �simulation
>>                       � �>> falls apart.
>>                       � �>> In this deployment pattern the region
>>        advertises
>>                       the presence
>>                       � �of the
>>                       � �>> asset, and *some* clients will be able to
>>        get it
>>                       as expected,
>>                       � �while
>>                       � �>> -based on the arbitrary whims of the asset
>>                       service- others
>>                       � �might not.
>>                       � �>>
>>                       � �>> My hope would be that after the asset server
>>                       provides the
>>                       � �region with
>>                       � �>> the capability to get the asset, it gives up
>>                       control. That
>>                       � �would mean
>>                       � �>> that if the client finds the inventory
>>        server is
>>                       unwilling to
>>                       � �serve
>>                       � �>> the content - in spire of the region
>>        saying it
>>                       is present-,
>>                       � �the client
>>                       � �>> should be able to turn around ask the
>>        *region*
>>                       for the asset,
>>                       � �(and get
>>                       � �>> is after all).
>>                       � �>>
>>                       � �>> �If that is not the case, -and there are
>>                       probably good reasons
>>                       � �for the
>>                       � �>> deployment pattern as described-
>>        �shouldn't we
>>                       *warn* clients
>>                       � �that the
>>                       � �>> region might be inconsistent, so the users
>>                       behind the client
>>                       � �can vote
>>                       � �>> with their feet, (or take the risk)?
>>                       � �>>
>>                       � �>> --Vaughn
>>                       � �>>
>>        _______________________________________________
>>                       � �>> vwrap mailing list
>>                       � �>> [e-mail] <mailto:[e-mail]>
>>        <mailto:[e-mail] <mailto:[e-mail]>>
>>                       <mailto:[e-mail] <mailto:[e-mail]>
>>        <mailto:[e-mail] <mailto:[e-mail]>>>
>>
>>                       � �>> https://www.ietf.org/mailman/listinfo/vwrap
>>                       � �>
>>                       � �>
>>                       � �>
>>        _______________________________________________
>>                       � �> vwrap mailing list
>>                       � �> [e-mail] <mailto:[e-mail]>
>>        <mailto:[e-mail] <mailto:[e-mail]>>
>>                       <mailto:[e-mail] <mailto:[e-mail]>
>>        <mailto:[e-mail] <mailto:[e-mail]>>>
>>
>>                       � �> https://www.ietf.org/mailman/listinfo/vwrap
>>                       � �>
>>                       � �>
>>
>>
>>
>>
>>  ------------------------------------------------------------------------
>>
>>                   _______________________________________________
>>                   vwrap mailing list
>>                   [e-mail] <mailto:[e-mail]>
>>        <mailto:[e-mail] <mailto:[e-mail]>>
>>                   https://www.ietf.org/mailman/listinfo/vwrap
>>                   �
>>
>>
>>
>>
>>
>>           --     --- https://twitter.com/Dzonatas_Sol ---
>>           Web Development, Software Engineering, Virtual Reality,
>>        Consultant
>>
>>           _______________________________________________
>>           vwrap mailing list
>>           [e-mail] <mailto:[e-mail]>
>>        <mailto:[e-mail] <mailto:[e-mail]>>
>>           https://www.ietf.org/mailman/listinfo/vwrap
>>
>>
>>
>>
>>    --     --- https://twitter.com/Dzonatas_Sol ---
>>    Web Development, Software Engineering, Virtual Reality, Consultant
>>
>>    _______________________________________________
>>    vwrap mailing list
>>    [e-mail] <mailto:[e-mail]>
>>    https://www.ietf.org/mailman/listinfo/vwrap
>>
>>
>> ------------------------------------------------------------------------
>>
>>
>> _______________________________________________
>> vwrap mailing list
>> [e-mail]
>> https://www.ietf.org/mailman/listinfo/vwrap
>>
>>
>
>
> --
> --- https://twitter.com/Dzonatas_Sol ---
> Web Development, Software Engineering, Virtual Reality, Consultant
>
>