Re: [vwrap] Technical basis for VW client in a web browser?

[Joshua Bell <[e-mail]>]

On Thu, Dec 16, 2010 at 3:01 AM, Morgaine <[e-mail]>wrote;wrote:

> So yes, what I'm looking for here are some substantive technical reasons
> for pursuing the browser route for VW clients beyond small, cut-down
> introductory ones.  So far I've not seen any good technical reasons for it
> at all, and in contrast, technical reasons why it could be a *bad* host
> platform are abundant.


Lisa Dusseault and I wrote a paper (linked from here) last year outlining
some of the technical challenges. The good news is that most of the
challenges are being actively addressed by browser vendors and the Web
community in general.

At a high level, a 3D immersive VW client needs these subsystems:

   - User Interface
   - 3D Rendering
   - Networking
   - Audio and Voice
   - Logic
   - Application Host/Framework

The Second Life Viewer, for example, uses:

   - UI: custom toolkit on top of OpenGL (with declarative definitions
   called XUI)
   - 3D: custom rendering engine on top of OpenGL
   - Net: custom messaging library using UDP plus HTTP libraries like CURL
   - Audio: FMOD/OpenAL (general) and Vivox (voice)
   - Logic: application logic in C++
   - App: custom app host/framework for windowing/OS integration

Looking at the Web as a platform, the following are available:

   - UI: HTML/CSS/JS; Flash; Silverlight; Unity3D
   - 3D: Flash; Unity3D; WebGL; custom plug-ins; server-side rendering to
   video
   - Net: XHR; Flash; WebSockets
   - Audio: Flash; HTML5 Audio; Vivox plugin
   - Logic: JavaScript; Flash ActionScript; Unity3D C#
   - App: Web browser

Obviously, that's a mish-mash of different technologies; one of the pushes
of the "HTML5" effort - using that term in it's broad marketing sense and
covering the various technologies like WebGL, WebSockets, WebWorkers, CSS3,
ES5, etc etc - is to create a more unified, Standards-based platform for
application development.

Efforts such as KataLabs KataSpace (http://www.sirikata.com/blog/?p=184) -
which is built using "HTML5" tech only - shows that this is feasible today,
to some degree.

To drill into the technical impediments and how they will need to be
addressed:

   - UI: none, really; HTML for UI is a mature technology. It's true that
   many implementations are poor, but this is not a blocker.
   - 3D: effectively 0% of users have WebGL now, but that will change
   significantly within 12 months; Flash 3D is currently weak but Molehill is
   due in late 2011 and should have fast uptake. Unity is available now but is
   a plugin which turns away many users. (Yes, technically, Flash is a plugin
   too, but it's a special case both technically and socially); server-side
   rendering is available now but there are many open variables
   - Net: WebSockets is not stabilized yet and does not offer UDP (which may
   be necessary for some low-latency scenarios); further, cross-platform
   - Audio: HTML5 audio for presentation is progressing well, but streaming
   voice capture has not yet been tackled and I haven't seen streaming-down of
   realtime sessions yet. Vivox has a plugin available today, however, that can
   be driven by..
   - Logic: JavaScript appears adequate for most application logic, but the
   amount of data that needs to be processed per frame in a complex world like
   SL is significant - far more than most games. Most WebGL demos so far have
   extremely simple requirements.
   - App: obviously, in the web an app lives in a browser tab. This implies
   far less control over system resources and the user's attention, and affects
   the implied user model - fast startup, tear-down with little warning,
   integration into navigation stack (e.g. back/forward), simultaneous
   sessions, etc.

Like most applications, attempting to simply recreate the experience as-is
within the browser is probably not the right approach, and will lead to a
poor experience. That said, I don't think there are significant technical
barriers, and certainly we already have examples where functional VW clients
implemented in the browser have been created.

-- Josh