This post, and my previous post about integration of OpenSim into Web 2.0, explore how immersive virtual worlds, through a full architectural integration into Web 2.0, will become part of the fabric of everyday computing.
The diagram above shows where OpenSim sits in Web 2.0 (click on the diagram to see a readable enlarged version!). The following interview with OpenSim developer, Teravus Ousley, describes some of the work being done to create documented protocols that will make OpenSim fit seamlessly into Web 2.0 architecture.
OpenSim is in the news a lot these days, explicitly as in the case of the announcement last week by 3Di of their “3Di OpenSim” Standard (for more see here and here), and implicitly with the launch of ChinaQ. Adam Frisby, OpenSim, pointed out to me if you download the ChinaQ client that it is based on OpenSim, it connects nicely to OSGrid too. There is speculation the client is a rebranded version of the realXtend viewer (which is based on the open source Second Life viewer) as all the version numbers are the same.
So OpenSim is not only attracting the interest of business giants like IBM, Microsoft and Intel, it is becoming the architecture of choice for virtual world initiatives from Chinese and Japanese telecoms (see here and here for more). Also, see the press release about Nokia and the City of Oulu, Finland, joining as supporters of realXtend.
But, as Raph Koster in his post commenting on 3Di’s OpenSim announcement notes, the question how immersive virtual worlds can go from strong niche or enterprise markets to mass adoption in consumer markets must be answered. As Raph points out, Lively, Whirled, SmallWorlds, Vivaty, and yes, Metaplace have a very different architecture that they hope will attract broad consumer markets. (I did a long interview with Raph on this at The Virtual Worlds Conference and Expo in LA which I will post as soon as it is transcribed, so more on this soon!).
Architectural integration into the heart of Web 2.0, I would argue, is the key to mass adoption for immersive virtual worlds. While architecture alone will not guarantee the necessary breakthroughs in usability for widespread consumer adoption, it will create the ideal conditions for the innovation through which usability obstacles will be overcome, and the enormous potential for immersive, real time interaction over the internet will be realized.
Interview with Teravus Ousley
Tish: What has been the most fundamental problem re virtual world architecture that has kept immersive virtual worlds isolated from web 2.0 to date?
Teravus: a lack of standardization, licensing issues, and the difficulty of entry into the industry.
1) Standardization
Tish: In order of importance what in your view are the priorities for standardization?
Teravus: Probably the same order that OpenSimulator was tackled in, basic connect (current state of OGP – Open Grid Protocol). Basic Service (interaction standards). Advanced connect/mashup/aggregate extensions. Preferably people will have working code in the various spaces there to use freely under various licenses..
Tish: Can you show me where OpenSim will fit in this drawing of Web 2.0 architecture? [Teravus makes some modifications on the drawing I send him from Dion Hinchcliffe’s presentation from his Web 2.0 Expo workshop, see original here]
Teravus: The modified diagram [now opening this post] is a great view of how it will look.
Tish: Why is the TCP stream left out of the original drawing? [For more about Transmission Control Protocol (TCP) is one of the core protocols of the Internet Protocol Suite see here.
Teravus: It is left out because the person who made this diagram had web pages in mind. Static large files, or small changing files. In the the drawing the fact that TCP streams are smaller then HTTP is on purpose.
Tish: I have heard different opinions on the percentage of the communications for virtual worlds that can be done over HTTP?
Teravus: The fact is that the biggest usage of communications in virtual worlds is transmitting images that’s the number one bandwidth usage. So, if we’re counting by ‘usage’ I say 91%. If we’re counting by services that use http. I say probably 75% I definitely think that http should be evaluated for use on new things ‘first’. But, there are a few places where HTTP doesn’t shine.
I am skeptical about replacing things in the UDP with HTTP thinking that they’ll ‘perform better. [For more about User Datagram Protocol (UDP) another of the core protocols of the Internet Protocol Suite see here.]
I think there’s been a huge test going on now and for the last 5 or six years with regards to the UDP protocol and it really has performed admirably. In the last year and a half, I’ve seen attempts to convert several things to HTTP that have failed, and failed somewhat spectacularly sometimes. In the end the items get reverted back to the UDP protocol. One such item that sticks out in my mind is CAPS(HTTP) based inventory retrieval. The capability to do that in the client has been available since before February. And, it’s been turned on and off on ‘Agni’ at least once in the process. Additionally, we (OpenSimulator) enabled http inventory, and, the inventory failures rose pretty steeply.
I think some services are really just not ‘right’ for HTTP.. . particularly where a ‘poll’ methodology is used, or, the data is significantly dynamic enough that it makes caching useless.
Anyway, as far as the future is concerned, I do want to see some services over HTTP. Other services, it would be more appropriate to have a TCP stream. Stock market data, for example, uses a TCP stream. The Scalability of the stock market, is just one example of a scalable TCP stream.
Tish: So you see TCP as the communications protocol that would do the work for the parts of virtual worlds not suitable for HTTP. At least that is how you have shown it in our Web 2.0 architecture drawing. But should there also be a UDP stream?
Teravus: For the virtual world of tomorrow? .. probably not.
Tish: Why not?
Teravus: You have less control over the quality of service when it’s delivered over UDP then TCP.
Tish: What is the exact relation between TCP and UDP. My understanding is UDP a lower level protocol.
Teravus: TCP offers guaranteed delivery through flow control, while UDP does not. One of the failures of UDP, is the ‘resend’ technology we’ve put on top of it to try and make it reliable. TCP does this automatically and better then we could at a lower level but it does also cost up to twice the bandwidth depending on what is being sent. HTTP is a layer on top of TCP.
Tish: So just like the HTTP/TCP discussion there has to be a TCP/UDP boundary discussion …so it is HTTP then TCP then UDP and the boundaries have to be worked on.
Teravus: Those are the orderings in my mind… probably if UDP uses any.. it should use less then 0.5%.
Tish: And the current Second Life architecture what does it use if it isn’t using HTTP? [to see the work of the Architecture Working Group on the future Second Life architecture here]
Teravus: UDP or HTTP
Tish: and TCP?
Teravus: Well, TCP is a layer under HTTP. As far as I know, SL doesn’t use TCP streams directly
Teravus: Instead, it uses HTTP polling. This is one of the places, that I’ve highlighted where it doesn’t shine.
Tish: Polling does sound slow?
Teravus: Polling is essentially.. (connect) Got any data for me? No?(disconnect), (connect) Got any data for me? No?(disconnect).
Tish: So what is the path to standards for this then?
Teravus: Distilling what we know works and what we actually intend on supporting as far as adoption under these standards.
Tish: Where does MPEG-V fit in? Have you read their document yet?
Tervavus: MPEG-V is interesting reading… but is there any working example? I have just the overview. But I’ll read it over to have a better determination of how to ‘keep it in mind’ for the future. It looks like they’ve only really defined the requirements of the MPEG-V spec. The MPEG-V spec looks quite far reaching.. but the documents so far are requirements and marketing talk aimed toward business people – obviously intended to get more people interested in working on them.
But I have a feeling that any format with MPEG before it will be onerous to support. ..for me it’s too early to tell. It’s quite far reaching…it isn’t anything like ’signal processing’ which the MPEG group is most famous for.
Tish: The whole top down approach of the MPEG-V initiative seems counter to Web 2.0 principles to me.
Teravus: Well, remember.. that even if there’s a virtual world format war (reference to DVD-HD vs BlueRay) we still need to win over the rest of the web.
Tish: Yes and don’t you think the way to win over the web is to use as many existing standards as possible?
Teravus: Well, it’s to use as many existing standards as ‘fit’ though.. KISS, as always (K)eep (I)t (S)imple (S)tupid if we have 30 different internet standards.. people looking at it will @.@
Tish: But it is just lack of documented protocols that has created isolation from Web 2.0? And really doesn’t it boil down to standardizing that small percentage that is outside HTTP – the TCP and UDP stream that we talked about earlier where the real time stuff that virtual worlds bring to the web happens?
Teravus: no.. actually the HTTP standardization is just as important.
Tish: You mean even though SL used HTTP it isn’t standardized?
Teravus: Not documented specifically.
Tish: And OpenSim is that documented?
Teravus: Not well enough probably to define a standard.
Tish: Is AWG (Architecture Working Group) doing the documentation?
Teravus: working on it..
2) Licensing Issues
Tish: It sounds like some of this work has to go on across client and server. Are we running into the issue of BSD for OpenSim and GPL for the Second Life viewer?
Tervaus: Well, some of the issue here is license choice. One of the reasons that libOMV was able to achieve what they did was they did it /before/ the client was open sourced.
Tish: So open sourcing the client actually became an obstacle!!???
Teravus: I don’t think so in a whole. I think it was great for the community. I do, however think that C++ UDP stacks will be scrutinized more for GPL license violations because, of course, the client is GPL and C++ .
Tish: It is my understanding that Linden Lab is open to discussions on making the licensing more efficient for the open source community?
Teravus: Well, the client, in a whole, should not be changed as far as the license. JUST the things that they expect people to adopt should be made more open. If they expect people to adopt PRIMs, then there should be an efficient implementation available for anyone to use.. at the very least, in LGPL format. Otherwise, the die hards are forced to re-implement them from scratch, and most people will just choose something more open.
Tish: Has anyone ever put together a list of the parts that need to be LGPLed?
Teravus: Well, I think it’s there in a few places. There is at least one jira open on it.
Teravus: A few that come to mind for me.. is the UDP stack and the prim to mesh/UV code. I think there are some things that can definitely be improved about the UDP Stack. There are some things, (images come to mind), that would be better over HTTP
Tish: Do you think if the UDP stack were L GPLed that would be a significant help to integrating OpenSim better with the web?
Teravus: Well, it would certainly be adopted by more clients. GPL + (your own code) = GPL Licensed client. LGPL linked library + (your own code) = Your own license.
You still need to mention that you used LL’s UDP stack, and provide the source code for it at request.
The general client itself should remain GPL, it’s better for LL that way. Just the items that they want people to ’standardize’ on. It would help.. if it was at least LGPL
Tish: And the value to LL on LGPLing these parts is it helped spread their basic technology while protecting the rest of their viewer?
Teravus: It furthers their goal of standardization on their systems because it allows more people to adopt it for their own uses without worrying about GPL-ing their own client.
Tish: It is hard to standardize without access to the low level parts of the client right?
Teravus: The general population of Developers.. will want a libX that they can plug into their application for communicating.. . libY to deal with object data..
Tish: Hence your requests for LGPL were UDP stack and the prim->mesh/UV
Teravus nods
Tish: and at the moment they only have openmv?
Teravus: That’s the only ‘truly’ open standard right now as far as the LL technology is concerned. OpenSimulator’s use of that data.. could also be seen as a standard..
Teravus: But we have not published anything beyond code.. neither have they really.. technically.. but their organization of the way things work is very very clear
Tish: What are the most significant limitations of openmv?
Teravus: Probably.. just it not being in c++. c++ has it’s benefits and it’s pitfalls. Changes in c++ usually take longer then ones in C#. But, of course c++ is always faster. With libOMV It isn’t always clear about what packet is used when. However, with some experimentation, you can figure it out in 30 minutes or less..
Usability
We didn’t spend much time discussing some of the innovation in usability that this architectural integration into Web 2.0 will enable (more to come on that!). But, Teravus mentioned one interesting use case he is working on.
Teravus: You might also stick a ‘cloud renderer’ into the graphic [Tervaus was looking at the diagram (from Dion Hinchcliffe) that opened my previous post on "Web 2.0 to OpenSim Made easy" click on the thumbnail below].
Some people have discussed having a ‘video stream’ that’s rendered on the cloud and providing that to flash clients would be the best solution to it for them.
The cloud renderer is for organizations that have large pools of servers with GPUs so would allow for very powerful rendering. The servers can render the scenes and stream them to the low end browsers. It would allow extremely high quality rendering for really low end browsers.. such as ‘cell phones.’
Tish: Is that possible now on OpenSim?
Teravus: Nope. But it’s something that in the future, I intend on working on. It would essentially be a video [streamed to low end browsers].
Tish: Is that different from what Vollee is doing? The mobile client for SL?
Teravus: It appears that they are, indeed, pre-rendering the client’s view and streaming it to the mobile device
