Re: [vwrap] [wvrap] Simulation consistency

[Vaughn Deluca <[e-mail]>]

Hi Dzonatas

Can you expand on that, what would be needed for legacy support in VWAP
terms ?,
If i want to read up on how the asset server may proxy the simulator, what
would you recommend me to read?

-- Vaughn

On Sun, Apr 3, 2011 at 5:51 AM, Dzonatas Sol <[e-mail]> wrote:

> Some stated the proxy-to-asset-server is built into the sim; however, keep
> in mind possible legacy support where the asset server may proxy the
> simulator.
>
>
> Dzonatas Sol wrote:
>
>> Somehow I feel the basic asset server being able to login and download
>> assets is now priority, yet I also wondered the best way to patch this into
>> the current mode of viewers.
>>
>> Maybe offer (1) by proxy (sim-side) and (2) by patch (viewer-side) that
>> either of these two are optional and neither are mandatory for now.
>> Thoughts?
>>
>> Israel Alanis wrote:
>>
>>>
>>> > when designing for scalability, the model to bear in mind is ...
>>>
>>> Well, there are a lot of different models to keep in mind, and many
>>> different use cases. One particular use case to keep in mind is: "User
>>> acquires new outfit, and wants to 'show it off' in a highly populated
>>> region".
>>>
>>> > Both worlds and asset services may include commercial, community, and
>>> personal services
>>>
>>> Yes, yes and yes. I'm particularly concerned about how the model affects
>>> the ability to host personal asset services.
>>>
>>> > a proxying region, which would get slammed for every asset worn by
>>> every avatar present.
>>>
>>> Granted the collection of services that are provided by the region need
>>> to be scaled to meet the demands of that region. That's all part of capacity
>>> planning.
>>>
>>> > regions run many different CPU-intensive tasks, including physics
>>> simulation and server-side scripting, and absolutely cannot afford to serve
>>> assets too
>>> Well... who said the same CPU's have to do proxying, physics simulation
>>> and server-side scripting? Asset proxying is a different service than
>>> physics simulation and can be on separate hardware, could make use of
>>> geographically distributed caching, and in certain deployment patterns, the
>>> same caching services could be shared by different regions. (Server-side
>>> scripting is a discussion for another day).
>>>
>>> > This is why we have to go parallel...
>>>
>>> Totally agree, and a proxying model could and should also take advantage
>>> of parallelism.
>>>
>>> > I think you're wrong that it has to cost much money. ?vs?
>>> > It costs money to host a high performance and scalable asset service
>>> and a high bandwidth network to handle the traffic.  A *lot* of money.
>>> I think what you're saying is: "It costs a lot of money to build a
>>> scalable asset service, but if assets are spread throughout the internet
>>> they don't have to be scalable." But that's not quite right. You're opening
>>> up every asset server to the VW equivalent of being slashdotted, so are you
>>> sure you're not forcing *every* asset service to be scalable and handle a
>>> lot of bandwith and network traffic? It's the exact opposite of your
>>> intention, but I think that's the result, all the same.
>>>
>>> This particular design decision has a big effect on the economics of the
>>> VW infrastructure. I'd rather the economics to work out such that a region
>>> provider who wishes to build a region that supports a small population, can
>>> do so economically. A region that wants to host a *large* population has to
>>> bear that cost of providing that scalable asset service.
>>> I want the economics of hosting a small asset service to be a non-issue
>>> (as to best promote creation and creativity). Creating a high bar to provide
>>> asset services will mean that service will cost money and people shouldn't
>>> have to pay money just to create or own VW objects (I'm using 'own' here to
>>> refer to maintaining their existence, I'm not trying to make a
>>> 'leftist'/'communist' statement about ownership ;)
>>>
>>> - Izzy
>>>
>>>
>>> On Apr 2, 2011, at 3:58 PM, Morgaine wrote:
>>>
>>>  Izzy, when designing for scalability, the model to bear in mind is that
>>>> of seasoned virtual world travelers whose inventories contain assets from
>>>> many different worlds, those assets being served by many different asset
>>>> services.  Both worlds and asset services may include commercial, community,
>>>> and personal services, and as the metaverse grows, that set is highly likely
>>>> to become progressively less clustered and more diverse.
>>>>
>>>> When those seasoned travelers click on an advertised VW link and perform
>>>> an inter-world teleport to one particular world's region to share an
>>>> experience, their "worn" assets (the only ones of interest to the region)
>>>> will contain references to asset services spread widely across the Internet.
>>>>  The fetches by the travelers' clients occur over many parallel paths from
>>>> clients to asset services, so one can reasonably expect reduced network
>>>> contention and reduced asset server loading because they are both spread out
>>>> over however many asset services are being referenced by the overall set of
>>>> assets in the region.
>>>>
>>>> This is very different to the case of a proxying region, which would get
>>>> slammed for every asset worn by every avatar present.  In our current
>>>> architecture, regions run many different CPU-intensive tasks, including
>>>> physics simulation and server-side scripting, and absolutely cannot afford
>>>> to serve assets too unless your scalability requirements are very low
>>>> indeed, ie. just a few dozen avatars of today's kind.  We've hit the ceiling
>>>> already on region scalability done that way.  There is nowhere to go in that
>>>> direction at all beyond improving the code like Intel demonstrated, and that
>>>> work is subject to a law of diminishing returns.
>>>>
>>>> This is why we have to go parallel, and I think you're wrong that it has
>>>> to cost much money.  As we spread the load across more and more asset
>>>> services, we are simply better utilizing all the hardware that's already out
>>>> there on the Internet, at least in respect of community and private
>>>> resources.  But add to the community and private resources the commercial
>>>> asset services that are likely to appear to exploit this opportunity, and
>>>> not only will the number of asset services leap, but the power of each one
>>>> will rocket too, because, after all, these businesses will be heavily
>>>> optimized for the job.
>>>>
>>>> As to why a world would want clients to access external asset services
>>>> instead of providing its own implementation, that's an easy question.  It
>>>> costs money to host a high performance and scalable asset service and a high
>>>> bandwidth network to handle the traffic.  A *lot* of money.  In contrast, it
>>>> costs a world nothing to let others serve the assets to clients.  And that
>>>> matters to the bottom line.
>>>>
>>>>
>>>> Morgaine.
>>>>
>>>>
>>>>
>>>>
>>>> ======================
>>>>
>>>> On Sat, Apr 2, 2011 at 7:05 PM, Izzy Alanis <[e-mail]<mailto:
>>>> [e-mail]>> wrote:
>>>>
>>>>    > As always though, it's a trade-off, since the proxied design
>>>>    has very poor scalability compared to the distributed one.
>>>>
>>>>    I don't agree with that... If a user enters a highly populated
>>>>    region,
>>>>    every other client is going to (could and should be trying to)
>>>>    hit the
>>>>    asset server(s) for the assets that the user is wearing (assuming
>>>>    they're not cached locally).  Every asset server has to be scaled up
>>>>    to the point that it can handle that load from all over...
>>>>
>>>>    If I'm hosting a region that supports 10s of thousands of
>>>>    simultaneous
>>>>    users (thinking of the future), I already have to scale to meet that
>>>>    demand. If the region is proxying the assets, then, yes the
>>>>    region has
>>>>    to be scaled to meet that asset demand too, but it already has to be
>>>>    scaled to meet other demands of being a region server... and why is
>>>>    scaling those asset proxy services hard?  It's going to cost $,
>>>>    but is
>>>>    not technically challenging. So, if I want to host a region like
>>>>    that... sure it will cost me, but the simulation will be consistent
>>>>    and users will be able to participate equally, regardless of the
>>>>    capabilities of their individual asset services.
>>>>
>>>>
>>>>
>>>>
>>>>    On Fri, Apr 1, 2011 at 11:55 PM, Morgaine
>>>>    <[e-mail]
>>>>    <mailto:[e-mail]>> wrote:
>>>>    > Every design choice results in a trade-off, Vaughn, improving
>>>>    one thing at
>>>>    > the expense of something else.  If every time we offered a
>>>>    service we had to
>>>>    > inform its users about the downsides of all the trade-offs we
>>>>    have made,
>>>>    > they would have an awful lot to read. ;-)
>>>>    >
>>>>    > The specific trade-off that you are discussing is no
>>>>    different.  A region
>>>>    > that proxies all content has the "benefit" of acquiring control
>>>>    from the
>>>>    > asset servers over the items in the region, so that it can
>>>>    ensure that
>>>>    > everyone in the region not only obtains the items but obtains
>>>>    the same items
>>>>    > as everyone else.  That does indeed provide a greater guarantee of
>>>>    > consistency than a deployment in which the region only passes
>>>>    asset URIs to
>>>>    > clients who then fetch the items from asset services
>>>>    separately.  As always
>>>>    > though, it's a trade-off, since the proxied design has very
>>>>    poor scalability
>>>>    > compared to the distributed one.
>>>>    >
>>>>    > If we're going to warn users of the potential for inconsistency
>>>>    in the
>>>>    > distributed deployment as you suggest, are we also going to
>>>>    warn them of
>>>>    > non-scalability in the proxied one?  I really don't see much
>>>>    merit in the
>>>>    > idea of warning about design choices.  Many such choices are
>>>>    technical, and
>>>>    > the issues are quite likely to be of little interest to
>>>>    non-technical users
>>>>    > anyway.  In any case, the better services are likely to provide
>>>>    such
>>>>    > information in their online documentation, I would guess.
>>>>    >
>>>>    > You mentioned users "voting with their feet" or choosing to
>>>>    accept the risk
>>>>    > of inconsistency.  Well that will happen anyway, when services
>>>>    fail and
>>>>    > users get annoyed.  If some asset services refuse to send the
>>>>    requested
>>>>    > items to some users, those services will get a bad reputation
>>>>    and people
>>>>    > will choose different asset services instead.  Likewise, if a
>>>>    world service
>>>>    > proxies everything and so it can't handle a large number of
>>>>    assets or of
>>>>    > people, users will get annoyed at the lag and will go
>>>>    elsewhere.  This user
>>>>    > evaluation and "voting with their feet" happens already with
>>>>    online services
>>>>    > all over the Internet, and I am sure that this human process
>>>>    will continue
>>>>    > to work when the services are asset and region services.
>>>>    >
>>>>    > Back in September 2010, I wrote this post which proposes that
>>>>    we use in
>>>>    > VWRAP a form of asset addressing that provides massive
>>>>    scalability at the
>>>>    > same time as a very high degree of resilience --
>>>>    >
>>>>    http://www.ietf.org/mail-archive/web/vwrap/current/msg00463.html
>>>>    .  It is
>>>>    > based on the concept of the URI containing a host part and a
>>>>    hash part,
>>>>    > where the hash is generated (once, at the time of storage to
>>>>    the asset
>>>>    > service) using a specified digest algorithm over the content of
>>>>    the asset
>>>>    > being referenced.  You may wish to note that if this design
>>>>    were used, the
>>>>    > failure of an asset service to deliver a requested item would
>>>>    result in a
>>>>    > failover request for the item to one or more backup services,
>>>>    using the same
>>>>    > hash part but with a different host address.
>>>>    >
>>>>    > This can go some way towards overcoming the problem that you
>>>>    think might
>>>>    > occur when assets are fetched by clients from asset services
>>>>    directly.
>>>>    > Although it won't help when the missing item is available from
>>>>    only a single
>>>>    > asset service, it will help in many other cases, and it will
>>>>    compensate for
>>>>    > service failures and network outages automatically at the same
>>>>    time.
>>>>    >
>>>>    > PS. This design for hash-based asset addressing is already
>>>>    being tested by
>>>>    > Mojito Sorbet in her experimental world and client.  It would give
>>>>    > VWRAP-based worlds an improved level of service availability,
>>>>    so I think it
>>>>    > should be a core feature of our protocol.
>>>>    >
>>>>    >
>>>>    > Morgaine.
>>>>    >
>>>>    >
>>>>    >
>>>>    >
>>>>    > ===========================
>>>>    >
>>>>    > On Fri, Apr 1, 2011 at 11:17 PM, Vaughn Deluca
>>>>    <[e-mail] <mailto:[e-mail]>>
>>>>    > wrote:
>>>>    >>
>>>>    >> This is a question i discussed with Morgaine off-list a while
>>>>    ago (I
>>>>    >> intended to send it to the list but pushed the wrong button...) I
>>>>    >> think we need to address this problem, and decide how to deal
>>>>    with it.
>>>>    >>
>>>>    >>  In Davids deployment draft, section 7.3.1.1 an overview is
>>>>    given van
>>>>    >> ways to deliver content to the region. One way is only passing a
>>>>    >> capability that allows access to (part of) the resource:
>>>>    >>
>>>>    >>           7.3.1.1.  Content delivery models
>>>>    >>           A range of possible represenations can be passed to
>>>>    a region for
>>>>    >>           simulation. [...] The other end of the delivery spectrum
>>>>    >> involves passing
>>>>    >>           only a URI or capability used to access the rendering
>>>>    >> information and a
>>>>    >>           collision mesh,and related data for physical simulation.
>>>>    >>           In such a model, the client is responsible for
>>>>    fetching the
>>>>    >> additional
>>>>    >>           information needed to render the item's visual
>>>>    presence from a
>>>>    >> separate
>>>>    >>           service.  This fetch can be done *under the
>>>>    credentials of the
>>>>    >> end user*
>>>>    >>           viewing the material [my emphasis--VD] , and
>>>>    divorces the
>>>>    >> simulation from
>>>>    >>           the trust chain needed to manage content.  Any
>>>>    automation
>>>>    >> is done on a
>>>>    >>           separate host which the content creator or owner trusts,
>>>>    >> interacting with the
>>>>    >>           object through remoted interfaces.
>>>>    >>
>>>>    >>  I can see the need for such a setup, however, i feel we are
>>>>    >> unpleasantly close to a situation were the coherence of the
>>>>    simulation
>>>>    >> falls apart.
>>>>    >> In this deployment pattern the region advertises the presence
>>>>    of the
>>>>    >> asset, and *some* clients will be able to get it as expected,
>>>>    while
>>>>    >> -based on the arbitrary whims of the asset service- others
>>>>    might not.
>>>>    >>
>>>>    >> My hope would be that after the asset server provides the
>>>>    region with
>>>>    >> the capability to get the asset, it gives up control. That
>>>>    would mean
>>>>    >> that if the client finds the inventory server is unwilling to
>>>>    serve
>>>>    >> the content - in spire of the region saying it is present-,
>>>>    the client
>>>>    >> should be able to turn around ask the *region* for the asset,
>>>>    (and get
>>>>    >> is after all).
>>>>    >>
>>>>    >>  If that is not the case, -and there are probably good reasons
>>>>    for the
>>>>    >> deployment pattern as described-  shouldn't we *warn* clients
>>>>    that the
>>>>    >> region might be inconsistent, so the users behind the client
>>>>    can vote
>>>>    >> with their feet, (or take the risk)?
>>>>    >>
>>>>    >> --Vaughn
>>>>    >> _______________________________________________
>>>>    >> vwrap mailing list
>>>>    >> [e-mail] <mailto:[e-mail]>
>>>>    >> https://www.ietf.org/mailman/listinfo/vwrap
>>>>    >
>>>>    >
>>>>    > _______________________________________________
>>>>    > vwrap mailing list
>>>>    > [e-mail] <mailto:[e-mail]>
>>>>    > https://www.ietf.org/mailman/listinfo/vwrap
>>>>    >
>>>>    >
>>>>
>>>>
>>>>
>>> ------------------------------------------------------------------------
>>>
>>> _______________________________________________
>>> vwrap mailing list
>>> [e-mail]
>>> https://www.ietf.org/mailman/listinfo/vwrap
>>>
>>>
>>
>>
>>
>
> --
> --- https://twitter.com/Dzonatas_Sol ---
> Web Development, Software Engineering, Virtual Reality, Consultant
>
> _______________________________________________
> vwrap mailing list
> [e-mail]
> https://www.ietf.org/mailman/listinfo/vwrap
>