The OSgrid Region Launcher is a somewhat newish utility I wrote back late August – it’s designed to be a dead-set-simple zero-configuration region launcher for OpenSim. It’ll automatically deploy your own region on OSgrid, hosted on your own computer; and while not perfect – it can be a nifty way of checking out OpenSim without too much hassle. This is a new version of the tool for those who were testing it several weeks ago – version 0.22 brings in a bunch of new features, including confirmed Linux compatibility (and presumably OSX too.)

This tool is a from start-to-finish utility for launching regions, it will automatically negotiate with your router to forward ports as required; and it will handle automatic updates of new versions using the official OSgrid.org Binary Releases. The basic release contains all the features of the default OSgrid binary release (including preconfigured Groups, Physics, Scripts, etc.). Shameless Plug: Obviously this has all the limitations of regions hosted from home – namely bandwidth & network considerations – if you want to host something permanently and reliably with support, come take a look at SimHost.

This new version of the launcher includes the following new features

• Confirmed & Tested Linux Support (and presumably OSX with Mono)
• New Zip Library (uses Ionic.Zip now) – works better on Linux.
• Completely rewritten UI – Now provides a great deal more feedback on what the tool is doing at any specific point in time.
• Ability to manually position a region optionally.
• Basic Input Validation (Bug: Glitchy on Linux – if you cant close the tool, input a few characters into the current field)
• ILMerge’d Release – now just one self-contained .exe

To use the tool, follow the following steps:

1. Drop the .exe in it’s own folder somewhere. Run it.
2. Enter a region name – must be between 1 and 64 alphanumeric characters. Some symbols & UTF8 characters are fine too. Known invalid symbols include: [ ] \r \n.
3. Select either automatic positioning (default), or enter a pair of free coordinate values. X first, followed by Y. Coordinates must be between 0 and 65,535.
4. Enter your Avatar Name on OSgrid.org – Firstname followed by Lastname.
5. Click Launch
6. At this point, the tool will download, unpack and install your region – you may be prompted to perform certain tasks manually (such as if Automatic Port Forwarding fails you may be asked to port forward manually.)
7. Once the OpenSim console appears – you should be good to go.

How to get this tool:

This is an Open Source utility released under a liberal BSD license. The latest source may be obtained via github

Source Code: http://github.com/AdamFrisby/OSGridLauncher

Release Download: http://www.adamfrisby.com/OSGL-r0.22.exe

Enjoy.

WARNING:

What is being described below is dealing in real currency – you owe it to yourself if you plan to use this, to understand how it works, and perform your own risk assessment. The module is completely unsupported and unwarrantied. Use it at your own risk.

Suppose you are a user of one of the Open Grids or Hypergrid system – and you want to purchase an item, pay into an object, or otherwise transact business where a real currency transfer occurs somewhere down the line. Up until now, in all environments you are reliant on trusting a third party to act as a middleman, providing some currency-equivilent (such as say V$, L$ or whatever.).

The problems with this scenario is that that currency is at best backed by a single corporate entity (and even then they may not choose to ‘back’ it at all) – leaving you exposed in the event something goes wrong. This is compounded by the general trading size of these operators – tending to be sole-traders or small-business; the best scenario is one where neither the user nor the merchant needs to rely on a third party beyond the credit card processor.

Which is where DTL-PayPal comes in, this is a free (3-Clause BSD), open source module we’ve developed to solve this explicit problem. It uses PayPal as the backend for the transaction, and prices inworld goods in US cents. You pay me, OS$100 – and you get a bill for US$1.00 from PayPal. Every transaction needs to be confirmed by you with PayPal thus adding security into the system; in addition you don’t need to carry existing balances of ‘currency’ in order to buy items – each item can be bought individually with a seperate transaction on your Credit Card for each purchase.

The transaction is a 2 step process for the user – which is illustrated in the diagram below. Step one, you ‘negotiate’ the payment size — this is basically filling out the payment or ‘buy’ dialog that the vendor or merchant has setup already. Step two is you will be asked to visit a special webpage (which links to one at PayPal) which sets up and pays the transaction. From a users perspective you need to do nothing more.

Steps 3 and 4 occur when PayPal has confirmed the transaction for you – once the payment is confirmed (usually within 10 seconds), PayPal notifies the module, which in turn completes the transaction, finally PayPal deposits the balance in the vendors account for immediate use.

Obviously the problems with inventory server issues, vendor malfunctions, etc still exist – but to a customer PayPal does allow you to dispute charges on non-delivery grounds (however beware doing this to scam the system – the merchant gets a chance at rebuttal and it can be a complicated process)

From a vendor perspective – the main drawback to this solution is cost, PayPal will charge you roughly $0.28 plus 2.2% for a standard account in order to process the transaction. On tiny transactions (such as one for $0.50, fee would be $0.31) this can add up to a significant portion of the transaction. For users using this exclusively, I highly recommend using a PayPal MicroTransactions account which has much lower fees (but certain additional terms & conditions).

So, where can I get the code for this? It’s on my personal GitHub account (along with a few of my other goodies) – http://github.com/AdamFrisby/DTL-PayPal – I will add some further notes, first this module is currently somewhat hard coded to present a warning to the user about it’s experimental nature, remove this at your own risk. Second – OpenSim is still alpha software, you may run into other issues, so be prepared to handle them if you want to accept payments from users in it. This software has only been tested on the PayPal sandbox so far (and I recommend you do the same), however should work with the live version fine.

Enjoy.

OK, so we didn’t quite get to 100 as originally planned – but this time it wasn’t OpenSim’s fault. Yes, by the end you could tell the sim was straining – and at about 65 avatars, the physics engine finally choked on trying to solve a 15 avatar capsule interpenetration (or at least, my interpretation of the bug – analysis pending); but it kept on accepting logins and people kept arriving – and very quickly we hit 70, … 75, … 80 then peaked at 85 before running out of people, slipping back to 79 and manually shutting the sim down to grab the all important debug dump.

It’s important to note here – these were real clients, using SL-derived viewers. By comparison libsl is a lot friendlier on the packet engine than the full viewer, so bots tend to be a less effective test. (Plus users introduce randomness that bots cant quite emulate). Wright Plaza with 85 avatars and their attachments weighs in at a healthy 15,400 prims – so there was no shortage of texture of prim data to be sent to each client – it’s actually probably one of the nastiest sims to do load tests in – which makes it great for this. Furthermore the hardware it is located on isn’t exactly top of the line, or even middle-of-the-line.

The short news is – we’ve made some really impressive progress in the the last week. Earlier we got up to 50 – which was tweaked, tailored and adjusted to get us to where 100 or even 150 isn’t really that out of the question anymore. There’s three big causes for this – first, abandoning OpenJpeg for decoding J2K textures made some very noticable improvements to stability (it’s in progress to abandon it for Encoding too); this means we’re not crashing on the way up – which means we can hit higher concurrencies more reliably. Second – John Hurliman from Intel rewrote our throttle routines and some low-level packeting code, which delivered a big boost to packet performance. Third – multiple efforts to reduce memory use in key places, has at least halved operating memory requirements – at 85 concurrent, memory was peaking at a mere 1.7gb (~20mb/user).

A result of these improvements has been memory IO is no longer such a major bottleneck – we’re actually beggining to hit the point where CPU usage is nearly becoming a more important bottleneck (we were hitting 90% CPU at peak — although the physics interpenetration mentioned above might be distorting this, since it could lead to run-away CPU use) – which is a refreshing change, since it is a lot easier to optimise around, and the tools for CPU use profiling are a lot better than those for memory IO profiling – and produce a lot more meaningful information.

We’d like to continue these load tests – the information the devs have gotten in the last week has been absolutely invaluable. Having a big pool of testers able to jump in on a moments notice has resulted in getting performance fixes tested and integrated a lot faster than usual – it’s also helped stability, each crash has been diagnosed and debugged in series as it is encountered. It’d be very easy to say that performance & stability wise, more has happened in the last week than the last 6 months – and we still need your help to keep going. We’re going to be continuing these load tests next week – there will probably be another major effort at getting 100+ avatars in a sim next Friday (same time, 1PM PST). If you want to know when the next test is planned, and help out – either hang around in #opensim on Freenode, or follow @osgrid or @adamfrisby where I’ll announce them they come.

Next stop, 150.

3Di – the Japanese OpenSim development company just opened the code up for their brand new in-a-browser viewer for OpenSim. It’s written in C#, has plugins for IE and Firefox; and is loosely based on the OpenSim-core team’s “Idealist Viewer”, which uses the Irrlicht 3D engine.

3Di’s innovations have been adding proper avatar support (albeit not the SL ones), improving the framerate fairly dramatically; and of course embedding it into a browser. It’s named “Rei” after the Japanese number for ‘Zero’ – and code is availible under a standard 3-Clause BSD license.

There’s already been a lot of news out there for the announcement of OpenViewer itself – so I probably wont go into too much detail; other than it lets you embed a 3D OpenSim space onto an arbitrary website – something which combined with say OpenID or anonymous login, could do wonders for increasing userbase concurrencies on OpenSim deployments.

OpenViewer has a couple of nifty extensions to the protocol like Mesh support – which uses the Irrlicht Model Format (realXtend uses the OGRE Model & Material Format, SL is planning to use the OBJ format – but dont expect anything for another year or two.); I’m not entirely sure how that plays into the building editor – specifically how you upload & use them, but I’m sure that will be clarified in forthcoming documentation. It will be interesting to see how well it performs on some big complicated sims (like say one of Shenlei’s behemoths), in theory it might behave a little bit better on the server, since libsl handles packeting a lot more sanely (and in lower volume) than the official viewer does.

There’s been some discussion in addition on the opensim-dev mailing list about getting realXtend & Rei both standardising on a common mesh format; COLLADA has been suggested – but the level of filesize bloat raises concerns for realtime streaming downloads of complicated regions.

Site: http://www.3di-rei.org (code & instructions)

Press Release (English): http://www.3di.jp/en/news/2009093001.html

Hopefully in the coming weeks, we can look at ways of embedding this into the OpenSim & OSgrid websites as a quick way to ‘try out’ the sims.

One racetrack, a little over 4 square kilometers of space (or 4,194,304sqm). This is my first attempt to build something with a bit of scale to it, now that we’ve got mega-regions working in OpenSim. (and vehicles.)

To start with, I need a terrain large enough to encompass the entire area (there’s no way in hell I’m manually terraforming this one) – courtesy of DeepThink, I have a license to World Machine Professional, a commercial terrain editing package that absolutely kicks ass. World Machine builds terrains in a semi-layered manner, then allows you to combine those through a railroad-style filter scheme.

Grabbing one of the World Machine samples as a base (since it had road layers already setup), I modified them and redrew the paths to build a 16×4 sim (4096×1024) terrain and rendered it. You can see the output in World Machine below, of course from this scale it doesn’t really do it justice.

Travelling at 55.5m/sec (200kph, 124mph) – you will cross into another region-sized space every 5 seconds; but the total racetrack length means that even at this speed it will take over a minute to complete, encompassing a minimum of 16 region-sized areas, which in turn is only a small subset of the entire space availible. Taking a picture of the terrain loaded into the mega region, you can see just how insignificant an avatar is at this scale.

This is compounded by the fact that the viewer refuses to put it’s “farclip” beyond about 4 sims distance, so at best you can see 1/4 of the track at once. The ringworld walls are each about 150m tall at this scale; and geographical features are quite recognisable as geographical features.

Of course, we can’t just leave it at that – to quote Mythbusters, “if it is worth doing, it is worth overdoing”, so I pulled up the terrain in world machine again, and decided to go a little bigger. I originally made it at 48×4 sims, however had to crop back to 32×4 due to my machine needing more memory to do the resulting 12,288 x 1024 terrain without swapping on disk constantly (resulting in a multi-hour render). For scale reference, the smaller of the two circles is about the dimensions of a single normal region.

Part 2 will cover texturing and detailing this race track, followed by some first impressions of how it behaves with Kitto’s new vehicle patches. Once I’m done with it, it will go up live on the Hypergrid (and probably OSgrid) for public perusal.

Kitto Flora deserves some major kudos for recent developments with OpenSim vehicles. Not only do they work, but the revised car script I toyed with earlier today – was actually quite usable; and dare I say it? good. All of this is thanks to Kitto’s work refining and improving the vehicle functions and how they interrelate with the internal ODE physics engine.

I could speculate here, and say if you were a decent vehicle coder – you could probably do some amazing things with OpenSim these days. So what happens if you combine these new vehicles, with Teravus’s recent mega regions code, and my sculpty terrain code (with mods from nebadon)? This. (Recommend jumping to about 4:40)

(Edit: Added a new link to the video – 22/09/09)

I probably do not need to go into too much detail here – the above is a 3_x3 megaregion, using 576 physical sculpts, Kitto’s latest ODE mods & vehicle script – running on a home users cable connection; with several people driving around at once. Not only does it work – it suprisingly works well. I expect a more skilled texture artist could pull some amazing things off with the sculpty terrain code – the terrain is just a quick output from L3DT done by Nebadon.

The vehicle code isnt in trunk quite yet – but if you ask around in #osgrid, I’m sure someone can point you to the github account where you can get the ODE patches. I’m sure much like mega-regions, it will be in trunk soon enough.

Credits: Kitto Flora (vehicles), Teravus (megaregions), Nebadon (video, hosting, testing, sculpty terrain), Bri Hasp & Hiro Protagonist (hosting & testing), Myself (sculpty terrain generator).

Connecting a region to OSGrid is a rewarding experience – many users however run into the difficulty of instructions being out of date; and while this set too will eventually go stale – they should at least be current for the next few months. Before getting started – you need to decide on a few things; information about your region (what to call it, where to put it on the map), where you will host it (at home, or on a dedicated server)

OSGrid is a free OpenSimulator network – connecting a region and using the central services is done free without charge (however donations to keep the infrastructure running are appreciated.)

First – this guide assumes Windows, and also assumes you are hosting from home. Users running in more professional setups will need to adapt accordingly. You may be able to skip a large portion of this setup procedure, by using the automated ‘OSGrid Region Launcher’ utility, you can download a copy of this experimental tool from the OSGrid Website; however this has not been confirmed to work with all users (nonetheless, I recommend it as a first step).

Preparing your region

Before we undergo the region setup process, you need to know your internet facing IP address, have setup appropriate port forwards. OpenSimulator by default will use Port 9000 on TCP and UDP; if you are behind a router you need to forward these ports to the machine that will be running the simulator instance. If you don’t know how to do this – see experimental tool listed above; or consult instructions for your router. You will also need to configure your router to enable ‘NAT Loopback’ – some support this by default, some don’t. If yours doesn’t, you may be out of luck and will need to host somewhere else (see my article on that topic).

You will need to gather two bits of information about your hosting environment, first is your internet-facing IP address – if you do not know it, visit this page. The second is some spare coordinates on the OSGrid World Map. If you don’t know them (or dont care to work some out), you can use this page here. Record both of these bits of information, because you will need them in a moment.

Download a copy of the OSGrid Package. It’s usually about 40mb, and can be currently found at this address: http://www.osgrid.org/elgg/pg/utilities/software

Unzip this package; and look for ‘OpenSim.32BitLaunch.exe’ – ignore the other files within the directory (regardless of whether you have a 64bit or 32bit system) – right click on it, and hit ‘Run As Administrator’; text will scroll by for a little while, then you will be asked a series of questions, in the format “Question Name [default]: _” – the following is how to answer these questions.

Questions

• New Region Name []:
  This is the name of the region that you want, it should be less than 64 characters long – and cannot conflict with any existing registered region.
• Region UUID[random]:
  You can ignore this, just hit enter.
• Region Location [1000,1000]:
  Enter here the region coordinates you wrote down above – exactly as printed in the ‘autocoord’ web page – that is, two numbers with a comma between them and no spaces. (The first number is the X coordinate, second the Y)
• Internal IP address [0.0.0.0]:
  Hit enter here – 0.0.0.0 means use ‘any availible IP’; this should be your choice unless you know better.
• Internal Port [9000]:
  This should be 9000, hit enter to use the default.
• Allow alternate ports [False]:
  If you are using port forwarding, leave this setting as ‘False’ and just hit enter.
• External Host Name [SYSTEMIP]:
  For this setting you need to put in your internet-facing IP address (or DNS address); you can use the exact output of the page listed above.
• Master Avatar UUID
  Skip this setting – just press enter.
• Master Avatar First Name
  Enter the first name of your avatar name here – my Avatar Name is “Adam Frisby”, so here I would enter “Adam” (no quotes)
• Master Avatar Last Name
  Enter the lastname of your avatar – eg; “Frisby” (again, without quotes)

With a bit of luck, your region should have registered with the grid and is now accessible. Login to the grid, and try teleport to your region via the world map. If it connected successfully; then that is all you need to do – your region is now online.

Chances are however, something went wrong.

Troubleshooting

The following are the common sources of problems when hosting a region from home

• “Remote Destination Timed Out”, or “Remote Destination is not responding”
  This error means that your port forwarding was not done correctly; and that the grid services were not able to connect to your region on the TCP port. Double check your port forwarding, and that your external hostname is correct.
• “Could not connect to region“, or “(done)“, or “Despite our best efforts, something has gone wrong“.
  This is a more insidious error, it means the grid was able to contact your region server on the TCP port, however the client was unable to contact your region server – double check your port forwarding and check that you have NAT Loopback enabled. If it still gives you problems; your router is incompatible with OpenSim – you may want to read this article on the opensim wiki for additional pointers.

Yesterday, OSGrid officially turned 2, at the opening presentation hosted by yours truly. OpenSim interrupted my presentation just once – at about 40 minutes into it with 53 avatars, Mono finally kicked the bucket. (It crashed again just after I had finished. Serendipity at work!)

We’ve been experimenting with Vivox, so the entire presentation was streamed via the new VivoxVoiceModule; running on the latest trunk code for OpenSimulator; both for the most part performed pretty admirably. A lot of the improvements from IBM and Intel lately into OpenSim’s packeting have been paying dividends. Mono’s previous concurrency “record” was about 40 users (for comparison .NET’s current record is 65).

So, what’s in store in the future and near-future for OSGrid? I had a couple of announcements at the opening presentation.

First – Hello General Store

The general store is the first “webshop” based on OpenSim that integrates directly into the grid itself. Listing of items, setting of permissions, etc are all done from within the viewer – directly integrated with your inventory itself.

This has one major advantage outside of the improved convenience – it means you do not need a region to mediate the transaction. Every transaction can be done directly on the inventory server (infact this is exactly what we do.); but what does this exactly mean?

When you create your general store – a special undeletable folder is created in your inventory called ‘My General Store’, items placed into this folder that are marked ‘Allow anyone to copy’ are listed on our website automatically, providing you are the creator of the item in question (and have copy/trans permissions on it).

Purchasing of items will have them sent to your inventory directly via the inventory server (although you will need to relog after a shopping trip for the items to appear.); saves a bit of time and effort doing the transfers – and you dont lose a transfer because you were in busy mode.

User Achievements

Many of you spotted our new achievements system located on the OSGrid website – it’s mostly a bit of fun right now (and probably will always be), but we are going to be opening achievements up to region operators across the grid. If you have something open to the pubic (some form of competition, game or other activity that is ‘achievement-worthy’) you can offer achievements as rewards.

The exact process is yet to be exactly finalised – but you will recieve a bit of LSL at the end you can include which allows you to award your achievements to specified users. There will be some caps on who is eligable to be able to award achievements (particularly so that people do not just set them up for friends)

“More information soon”

Acknowledgement of Direction

OSGrid has traditionally been a test grid for OpenSimulator – that has been good and bad – in our case however, we’ve definetely moved on from those roots – OSGrid today is now more of a social space than a testing one. As a group, the admin team has decided to acknowledge this as part of OSGrid’s long term direction – and include those aspects in our mission statement (while preserving our current roles too).

While the testing aspects will always be present, we’re going to be focusing on making the social aspects more useful; such as improving our events calendars and website integration features. We would definetely like to see more live events – and will help promote any regular events to our wider communities.

Some of these changes will be more profound, which is actually tied to…

Continents

We are going to be introducing two new continents into the OSGrid ecosystem; so there will be three major continents in total. For lack of a name so far, they will be ‘Terra Alphus’, ‘Terra Betus’ and ‘Terra Gammus’ (subject to change). The current continent centered around 10,000×10,000 is ‘Betus’; and will not be changing.

Alphus will be the most structured of the continents – and subdivided into three zones based on the server geography – US, Europe and Asia. Regions connected to Alphus need to be approved by a small committee, and subject themselves to a few construction limits. Alphus will:

• Consist only of regions on dedicated server hosts (or good quality VPSs)
• Have a reasonable uptime (that is they are significantly likely to be online)
• Follow a master terraforming pattern.

Regions in Alphus will be organised so that servers located in the US are grouped with other US servers; EU with EU and Asia with Asia — meaning border crossing between Alphus regions is a lot more likely to be successful. The goals with the Alphus area is to ensure a more consistent user experience [especially for new users] and can be thought of as a ‘grid within a grid’ of sorts.

Betus is the current continent – it’s not going to be touched (other than perhaps by a few regions moving into Alphus); the current strategy of nearly-anything-goes is not changing here. You can place your own regions in this section without any considerations at all.

Gammus is the final section, and also a new continent – this will be tied to our new highly experimental region launcher. Regions hosted from home using our launcher will end up here. Because these regions are less likely to be online at any given moment, we are conglomerating them together away from other regions – so that the other regions on the grid are not affected by constant neighbour checks.

In summation:

• Your current regions arent changing or moving. If you want to setup a new region on any random coordinates, that’s fine too.
• The new continents will be fairly close to each other in coordinate terms (within 500 coordinates; or within reasonable scroll distance)
• The goals of the introduction of the two other continents are to improve user experience (especially with border crossing, etc.)
• This is not a grading system; Alphus regions are not better than Gammus – only that Alphus voluntarily comply with a tighter set of user experience restrictions.

New Orientation Sims

We’re going to be working on some new orientation island sims with OSGrid; particularly to aquaint people with what OSGrid is and isnt. These are going to be seperate from our welcome area (LBSA), but we are going to look at ways we can integrate them together (such as providing a chat bridge with a helpers group, etc.)

Again, more information coming soon.

If you haven’t popped your head over onto OSGrid in a while, you might want to take another look. We’ve made some reasonably big improvements in recent weeks; above pictured is part of the new LBSA Plaza (welcome area); but the real improvements have been in our website.

I mentioned previously the development of Elgg integration on OSGrid – we put that into production just over a month ago, and have since spent quite some time refining the integration. Initially when we deployed, there were a couple of deficiencies in features over the old website – we’ve quickly gone past there, and the new website integrates a whole bunch of shiny new features.

Many of these features are or will be linked to popular places inworld – such as the website event calendar being visible on billboards inworld (using PHP+GD+osSetDynamicTextureURL). We’re working on doing some of the reverse too (such as each region getting it’s own “Page” automatically — similar to how each avatar has it’s own profile page.)

OSGrid itself has grown pretty dramatically in this period too – it grew on almost every metric by a full 25% in the last 4 weeks. Regions, users, active users – all shot up. Infact, using the stats on the OpenSimulator.org grid list (skipping grids with missing data) – OSGrid now represents between 48 and 56% of OpenSimulator grid users; it’s by far the largest and busiest OpenSim grid. OSGrid currently has about 2,100 active regions (by comparison, the Second Life mainland is approximately 5,000 regions)

Some of the other improvements we’ve added was a new achievements system (pictured above), which we will be expanding to encourage exploration and participation in OSGrid events. We’ll be rewarding people with ‘achievements’ for various things such as participating in official (and some unofficial) events, exploring, building and more.

Which brings me to OSGrid itself – it’s nearly turning 2. On the 22nd of this month, we’re going to be celebrating our second birthday. We’re still sorting out what exactly the celebrations will be – but you can find them on OSGrid’s Event Calendar closer to the 22nd.

So if you haven’t already – come register an avatar and take a peek.

[Apologies to readers for the lack of updates in the last 3 weeks - I've been buried in work, I'll be posting June's commits wrapup in the next few days -Adam]