It’s been a number of months since I last looked at the State of Open Viewers back in december.

Since then there has been some coalescence on Idealist as the platform-of-choice for the future viewer. While Xenki has some definete nice features (browser access, reasonable UI), it has some problems too (32-bit only, security sandbox issues, etc)*. I’m hoping we’ll be able to take some of the UI handling code (camera control, etc) from Xenki and import it into Idealist to get a best-of-both result.

Right now, the Idealist renderer is pretty good – it’s rendering things close to 90% accuracy, the remaining 10% include avatars (a somewhat complex topic), some sculpted parameters, and special effects (glow, bump maps, etc.). The following two screenshots should give you a good approximation of the rendering capabilities of Idealist right now.

Fig 1.1 - Idealist Viewer

By contrast, the same scene in the Second Life viewer is fairly similar

Fig 1.2 - Second Life Viewer (1.22)

You can view this scene on OSGrid.org, it’s in the region “DTL Welcome Center” (a side project to build a new Orientation Island for osgrid.) – it makes a good test of a renderer since it uses almost every graphics feature on offer right now.

The most noticable differences between the viewers in terms of accuracy come down to a couple of minor but fixable points

• Texture alignment – Idealist doesnt process extra texture parameters; rotation, scale, offset, etc. Most of these are fairly simple to implement, so this should not pose any long term difficulties.
• Mirror X / Invert Sculpt Parameters – Idealist doesnt process these flags, so some of the sculpts are inverted (noticably the orange signs).
• Alpha smoothing – Idealist produces ‘rough edges’ on alpha surfaces, this is fixable with some tweaking of the renderer parameters.
• Texture Downloads – Not a bug in the renderer, however the texture download routine suffers issues with download speed and incomplete downloads. This is somewhat fixed in MXP due to MXP using plain HTTP for asset distribution – but under LLUDP some additional work needs to go into libomv to handle asset transmission better.

While that is not an exhaustive list – the current feature state is pretty good. Where work needs to go right now is in user interface (doing cross platform UI with 3D surfaces under .NET is interesting, let me assure you.) – one of the UI proposals I have been working on is pictured below.

Designing a Viewer UI

Fig 2.1 - Idealist UI Mockup

The main idea being to reduce clutter where possible – things such as Inventory are needed often, but arent needed permanently. Implementing these as a docking toolbox allows you to expand the inventory dock quickly and easily – while not having it consume significant screen real estate unnessecarily. As mentioned above – getting a design like this to work cross platform is difficult, infact it may even be easier to write a UI toolkit for Irrlicht than to attempt to embed Irrlicht into a UI toolkit while maintaining cross platform support.

MXP

One of the more interesting features of Idealist is the ability to connect a client on the newly implemented MXP handler. While getting it running requires a lot of technical knowledge right now – the MXP protocol has a lot more potential than the LLUDP one does for expansion and interactivity. There are also a number of improvements MXP makes to the transport layer which should improve scalability for the server.

Fig 3.1 - Idealist/OpenSim via MXP (source)

You can play with the MXP version by using the MXP branch of IdealistViewer, use the following settings when doing an SVN checkout

SVN URL: http://forge.opensimulator.org/svn/idealistviewer/branches/idealistviewer_mxp_0_0_1
SVN Username: anonymous
SVN Password: anonymous

There are some other benefits visible on LLUDP too, including a fixed water shader, better terrain handling and other renderer improvements which have not yet been migrated back to Idealist Trunk. When connecting, you will need to connect to the region in the format “mxp://ip:1253/<region uuid>/Test/” – note that on public grids exposing your region UUID can be a risk (some work being done on improving this).

Contributing

Finally, there’s a call for some contributors to Idealist – right now the major developers are also major OpenSim developers. We’re looking for some graphics specialists to come in and help with the design of Idealist – there’s a number of bugs and features which could do with patches, so if you are interested, please hop in #idealist-dev on Freenode.

I’ll repeat the call for Xenki if anyone is interested, I’m currently looking for a new maintainer to take over the role. Shoot me an email (see contact page for details) if you are interested in taking over the lead developer role on the project. (Update: Maintainer has been found.)

Sources are availible from the usual location:

http://forge.opensimulator.org/gf/project/xenki/frs/?action=FrsReleaseBrowse&frs_package_id=6

This is the rewritten version of the XBAP Viewer known as Xenki – this release has support for IRendering-style meshers, Terrain rendering and more.

I was reading some of the trackbacks about the original Xenki article that I wrote and came across this particular piece, with the truthful accusation that the original post was rather fuzzy unless you are already familiar with a lot of the technology.

Xenki (Snowcrash reference?) is a browser add-on able to show the OpenSim environments inside the browser. The original post is amazingly fuzzy at explaining what it does exactly (just technobabble to anyone not deep into coding) but its the very basics of getting 3D inside a browser in a similar manner OpenSim renders the same content inside its client.

Well, it’s time to correct that with an introduction to the background technologies I’ve used in Xenki, their benefits and finally a summary of what the hell it is.

An Introduction to XBAP

XBAP is a browser technology that Microsoft recently developed – it first appeared on the scene with .NET 3.0, along with WPF and a host of other new technologies. XBAP is basically short hand for “XAML Browser APplication” – in english gibberish this means “Browser hosted .NET WPF Application”.

Which in turn means “Your standard Desktop Application, inside a tiny little browser space, but not quite.”, there’s a few major differences – first

You dont need to install anything – XBAP’s run the same way Java Applets do – they download the program code onto your computer, then launch it in a tightly secured “sandbox” so it cannot affect the rest of your computer.

No-click installation of a desktop application that cannot access the local system is actually a big benefit here – since we dont actually need anything on the local system to be able to run it, all our content is being loaded remotely anyway.

We can however for caching purposes install the application which will give it access to the local system – but this isnt required. It will display just fine on it’s own without.

An Introduction to WPF

The other acronym pushed above is WPF – WPF is short for Windows Presentation Foundation, this is a new graphics display system introduced in .NET 3.0 and Windows Vista, although it’s been backported to 2K/XP if you install .NET3.

It has some very cool features – including a well thought out 3D API which I have been using for the rendering thus far. XBAPs require that you use WPF for rendering purposes – but otherwise let you access the Windows API just fine.

And Xenki?

Xenki is basically an XBAP 3D application that works like a Java Applet. It embeds libomv for networking – so we can connect to Second Life / OpenSim style world without any issues. It does not need to be installed and runs just from within the browser by visiting a page. We can put it in a ‘portal’ on the webpage by using an IFRAME (similar to how Lively operates, except it uses an ActiveX control for rendering).

Why this approach is a good idea

The answer here is simple – we dont need to install anything, so web designers can embed Xenki controls on their website and expect people running Windows to be able to see them without any installations required. All you need is to be running a OpenSim, or have a space in Second Life that your users can be sent to, and be using either Internet Explorer or Firefox.

Finally the name – Snowcrash reference?

Yep – actualy the name was suggested by an employee at DeepThink, the fabulous Cichli Azure. Cichli was suprised to hear that only a single person has gotten the reference so far.

I hope that clears it up for people.

There’s been a rush of discussion recently about embedding virtual world technologies in the browser, I have seen two Java3D based implementations (which are very clever and deserve a closer look – especially if either is Open Source).

Tish has written up an interview with Avi who I mentioned earlier, on 3D in the Browser, the technologies and where things should be going – he’s very eloquent and I agree with pretty much everything he has to say. Avi added a companion piece with just a table of definitions and a small commentary on the article, which leads me to add my own comments on things.

First – the best browser is one that we dont need to install much onto the clients system, because the hurdle of making a plugin popular is a big one. XBAP is great for this since you dont need to install it to run it – it runs on a sandbox on it’s own just fine.

The downside to this is that we lose things like Local Caches which dramatically increase the load times when frequenting common areas – it would be nice to be able to optionally “Preview, then install” something we might be able to do via the XBAPs (one mode restricts to sandbox, other presumes it’s installed). Some experimentation in installing easily is going to need to be done.

I’ve been doing some further experimentations with XBAP (and yes, I’ll be posting the latest Xenki code shortly), and discovered some interesting things. First – 3D performance is no worse than a standalone WPF application, Second – it runs on Windows 2000/XP and up – I previously assumed it was Vista only, and was pleasantly suprised to find out this is not the case. Third – getting security certificates needed to automatically launch directly without install actually wont be too painful afterall (as long as libopenmv can get signed, we’re good.).

Cross Platform, Java3D?

The cross platform issue still remains elusive, which is why Java3D has caught my eye recently – given the similarities between C# and Java, it strikes me as potentially useful to be able to take large chunks of Xenki, run it through a special compiler and produce something that will run on Linux and Mac too.

The question becomes: why not use Java directly and skip the C#/WPF bit? Well the answer here is somewhat a personal one, first is that WPF is a lot easier to work with – this is an unfortunate fact of life, but doing things like UI ontop of 3D frames and doing it cleanly and efficiently is something that WPF has done so much better that there is almost no competition between them.

I personally have a preference towards doing things as quickly and cleanly as possible, worrying about making it functional rather than working on structural work that has already been done before – however if someone has already produced a good embeddable engine for Java on 3D pages – I’d be very tempted to at least try and give it a shot.

Java3D does also have the downside that it still sits in a sandbox, and the sandbox is mandatory – there’s no way we can implement a local cache using it to the best of my knowledge, I personally feel that the installable option is a very good way to cross the barrier between ‘hosted’ and ‘installed’ once the user is familiar.

Updated Xenki Sources

As stated above – I’m going to push the new xenki sources out in the next 24-48 hours. Watch this space as I will post the URLs and also the SVN address where I will be keeping changes.

Authors note: I use terms such as ‘disadvantage’ when refering to Second Life’s building tools as a comparison to professional tools with professional artists, naturally user generated content tends to lean towards less efficient building techniques. This is not a slight on the content creators themselves, just that the tools make lots more work for people writing renderers and dealing with efficiency.

Second Note: Like my previous post, a large deal of this is speculation. I plan on confirming or denying a large number of my suspicious with the Xenki viewer’s design, but at this point should be just ramblings on the authors blog rather than any authorative statement.

As a sidenote from my previous post – I have some more ideas I’d like to try put into practice directly with rendering Second Life(tm)-style scenes faster for Xenki. The mainline SL client achieves it as far as I can tell through a combination of utter brute force (equivilent to sending an entire dam through a garden hose every minute – It’s pretty impressive.) and lots and lots and lots of caching.

This is not going to play well with WPF at all (I can see that much already), first we dont have access to low level hardware, and second I dont want to debug a thousand graphics glitches with every nuanced bit of hardware. Thanks, but no thanks, I’d rather let MS worry about that part.

So, if brute force is out of the question, what options exist for making things render faster.

First is the obvious one – let’s cache better.

One of the things that has been lamented previously has been the fact that Second Life has dynamic content, ergo we cannot cache the scene – I suspect this isnt the whole deal, while it is true that every object in the scene can potentially be moved (scripted or avatar building) at any moment, we can evaluate a lot of them on probabilities and discount swathes as likely to move.

Objects

Objects can be pretty easily split between “Likely to move” and “Unlikely to move.” Likely to move objects were either recently created, marked temporary or physical, or contain scripts. While it is true the others could still move, the probability is significantly lower, and therefor we can more readily cache them. If they get moved, then we’ll need to rebuild that cache (without the object that moved), but for now – it’s acceptable.

This cache could take the form of rendering the entire ’static’ portion of the scene to a single massive vertex buffer, and then rendering the dynamic elements individually (or in smaller caches). This is very similar to how modern games work – however in that case you have the advantage of being able to build a BSP tree in the editor. I am uncertain as to whether we are capable of doing BSP generation fast enough to make this dynamic cache feasible, but it is an interesting idea nontheless (Insert additional concerns about wide open spaces and BSP trees here).

A potential downside here is that we’ll need to change how LOD works for this to be effective – rather than having LOD calculated “on the fly” as your camera navigates, we will need to force the scene, then only update LOD periodically as the cache refreshes. In this case, LOD may become a function of the size of the object in absolute terms rather than relative to screen space.

Maintaining this cache on an idle processor

One of the great things about processors lately has been the abundance of cores added, this means chances are there is a piece of hardware sitting on this machine without much to do. We can leverage this by doing the cache building and maintainence on a seperate thread which runs on another processor, because the cache is not a prerequisite to rendering – we can optimise the cache in the background, then use it when it is availible.

Handling Textures Better

Second Life has the disadvantage of not using professionally created textures on every surface – this means that it’s possible for a microscopic object that you cannot really see having a massive 1024×1024 sized texture attached to it, increasing both bandwidth usage – and the amount of texture memory that is consumed in displaying your scene.

An idea for fixing this problem could be to measure the surface area each texture is applied to, then using this surface area to approximate what resolution we should render each texture as. (Converting that 1024×1024 texture down to a 32×32 texture if it is only used once, on that object).

By doing this, in combination with careful management of the amount of texture memory availible (downsampling to fit memory and applicability together) this may get around at least part of the “huge texture memory consumption problem”.

A quick update: Well, it looks like I’ve managed to almost solve the issue which I listed previously.

Here’s some screenshots of things sort of almost rendering correctly.

and one more showing the detail of one of Cubey’s planes (in this case his Ornithopter – and it renders correctly!)

Will post more screenshots later once I have completely fixed that issue.

So, I’m sitting here banging my head this morning over two issues. One, why the heck is terrain never deviating from zero height, and secondly why things werent looking quite right – as you could see in the previous posts it was clearly rendering prims but things were ‘missing’ or not quite right. Turns out the answer was both.

First, Terrain.

Yesterday I got the heightfield behaving properly, but couldnt understand why it was never being set when placed onto the live feed coming from the network stack. Answer turned out I was doing something stupid and had typo’d on a variable name for my indexer. Once that was fixed, we were rendering scenes like the one below.

It’s beggining to look a lot prettier, but as you may have noticed, the prims dont appear to line up in any recognisable pattern. While there is definetely a pattern there – something is very off.

Second, Objects – Part A.

Showing this one to Easy [Babcock] in the office, we quickly worked out that infact objects were never being rotated – every object had exactly the same identical zero rotation. After a few minutes debugging, this turns out to be related to the conversion from a Quaternion to a Euler rotation for WPF. Switching from Vector3 to Media.Quaternion internally solved the problem nicely.

Here’s a view of Abbotts aerodrome with the fix inplace.

Much better, although there’s still some clumps missing.

Second, Objects – Part B.

So, it’s looking like we’ve almost got this rendering correctly. At least object shape and rotation is being displayed correctly, although there’s still something lacking. It turns out that most of the bits missing corresponded nicely with camera view – so I’ve fixed this by telling libsl to ‘rove’ the camera position around the sim to download the entire thing. The above screenshot had this fix inplace.

This solved at least terrain loading completely and most objects.

Loading it up on OSGrid in Wright Plaza – everything seems to actually render properly now, at least so far as primitives go – we’re missing sculpties right now which form a large component of Wright Plaza’s design.

Second, Objects – Part C!? Huh?

Panning our Camera around a little, we notice something a little bit … odd. Namely that around 0,0,0 there’s a large congregation of primitives. I’ve been noticing this already and had discarded it as possibly being neighbouring sims in which case, I’ll just knock them out later.

But it turns out, there’s valuable prims being thrown there – it looks to me like maybe the child primitives in link sets having their “Position” being relative to the parent primitive rather than the sim (in OpenSim we have both .Position and .AbsolutePosition to seperate these).

So, I’m going to be working on that for the rest of this afternoon – after which I’m going to play with either texture rendering, or getting Meshmeriser to work so we can discard the dependency on the black-box GPL’d rendering library.

The basic rewrite of the viewer has been done, I’ve seperated out the renderer and the network code into seperate classes (which in turn will be abstracted into interfaces once I have a good idea on what functions are going to be required). There is a screenshot attached below.

Basic Scene Rendering using BoxRenderer.

Switching to a more advanced mesher

One of the things I have been tinkering with has been using more advanced meshing algorithms. libomv now has a IRendering interface which actually acts as the host for a variety of meshers (GPL, BSD, and others with varying levels of accuracy) – I’ve decided to use this interface internally when hooking in the mesher for the default renderer, so we can use any supported mesh generation algorithm, including both the GPL’d one derived from the viewer, the BSD box renderer, and potentially OpenSim’s BSD Meshmeriser. (The latter being my personal preference – since I’d like to be able to actually take a look under the hood rather than linking to magical black boxes. Insert obligatory licensing complaint here.)

In english – we can make things look prettier. The above screenshot was with the simple box renderer, the one below was linked against the more advanced one. The difference between these two functions is a single line of code and a difference reference when compiling the XBAP.

Now unfortunately, since I’d rather keep Xenki under a BSD/MIT-style license (usual reasons) – the distributed sources I’ll be releasing wont be linked against the more advanced renderer which I’m using for the screenshot below, but I do plan on spending some time tommorow attempting to get Meshmeriser working in it’s place if I can.

The timeline from here

My goals with this will be to try get textures (at least in a preliminary fashion) done tommorow, after that I will look at adding UV and Normals support to Meshmeriser and convert it to a IRendering interface so we can use it.

I also plan on cleaning up the UI a little more, below is a screenshot of the basic login form that now exists (so usernames and passwords are no longer hard coded, and this is almost venturing towards being sort of usable)

With a little luck, this will be sort of usable for basic navigation and viewing by the end of the week. I will post an updated source package sometime between now and then.

I’ve put the source for my previous XBAP experiment online on the OpenSim forge – we’ll move this over onto a dedicated system and setup SVN there. For now it’s just a convenient location for the code. In the process, we’ve christened it “Xenki” – it’s a mix of “X” (for XBAP) and “Enki”, an ancient god of crafts.

You can download the sources from the following URL:
http://forge.opensimulator.org/gf/project/xenki/frs/?action=FrsReleaseBrowse&frs_package_id=6

A warning in advance – the quality of this code is approaching rock bottom, the handling of triangles for instance is terrible (we create a seperate 3DModel instance for each triangle in the scene. You have been warned). It’s probably going to need a scrap and rewrite with a bit more forethought to the design – this was an initial “will this work?”

I’ve registered the domains, so we’ll have something up shortly – if your interested in contributing, leave a comment and I’ll get back to you once we’ve got some SCM up.

Enjoy.

Yes, It’s alive.

As previously posted – I’ve been playing around with doing a viewer in WPF as a XBAP Browser Application. The above is a screenshot from it – at the moment it’s pretty primitive. No textures or colours (everything is a black box), however scales and positions are set correctly.

This is about 2-3 hours work so far (most of which was learning the WPF SDK), but it’s showing promise already. One of the problems I have been having is that the security sandbox requires cryptographic signatures on each component (and there’s a valid reason behind this), but getting that signature on libomv  has been painful, so for the moment I have abandoned the sandboxed version and require a small installer on the first run.

I will post more as I finish bits and pieces. The code is very first-generation hacky and ugly, but in theory there’s some potential for promise here.

Update #1: Slightly better image

After discovering that everything was black due to a lack of light sources in the scene, I have added two and the results look a bit nicer.