Adam Frisby

Running OpenSim under a 64-bit Environment

without comments

Every now and then, this hits me. I fire up OpenSim (or RealXtend), and it crashes instantly with a “BadImageFormatException” - when you see this, you know you have a compatibility problem with an embedded C/C++ binary. In english, 90% of the time this means you are running under a 64-bit environment with a mix of 64 and 32bit code trying to run together.

OpenSim is 64-bit aware

The reason for this happening is that OpenSim by virtue of the .NET platform is quite 64-bit aware, this means natively it can address 16TB of memory just fine if run on a 64-bit operating system. (nb, under Mono you need to make sure Mono is 64bit to get the same benefits).

However unfortunately, some of the libraries we link against have to be compiled ‘one way’ or the ‘other way’ and not both.  When running a 64-bit application, things such as memory pointer lengths are difference, so there is no way to cleanly pass data between a 32-bit and 64bit application - everything must be one or the other.

So how do you fix this?

There’s two routes, the easy route, and the proper route. The easy route is good for 90% of our users - if you dont need to access more than 4GB of memory and dont mind about the slight slowdown when handling double precision numbers, then just run 32-bit.

How? Instead of launching OpenSim.exe launch OpenSim.32BitLaunch.exe instead - this forces OpenSim to run under a 32bit environment (under windows provided by WoW32).

The more detailed fix

The next option is to strip out the sections that are causing problems. OpenSim is modular - so you can swap bits and pieces with their alternatives. The following are components that are not 64-bit compatible by default (however it is possible to fix this, something I will get to in a moment.)

Incompatible Components List

Physics - most problems come from our Physics Engines, since these tend to be written in high performance C/C++ with ASM littered. The following list of physics engines by default are NOT 64-bit compatible.

  • OpenDynamicsEngine - The ODE DLL we provide is compiled for a 32bit operating system, however you can fetch the sources for our custom ODE DLL from (SVN) http://opensimulator.org/svn/opensim-libs/ and build it yourself on your native environment.
  • AGEIA PhysX - The AGEIA PhysX support (now Nvidia PhysX) relies on a natively compiled 32-bit DLL. At present there is no fix here.

Compatible Engines

  • BasicPhysics - This is purely managed but very simple in operation. Collisions are provided only between the avatar and the ground here. This runs without problems regardless of environment
  • POSPhysics - POS physics is a more complicated BasicPhysics engine with bounding box collisions between objects. For some tasks this is sufficient.
  • BulletX - Our own custom BulletX library is derived from the BulletXNA project, this is a properly developed fully managed physics engine that does proper collisions (however the friction coefficient can be a little low for some users).

Storage Engines - At least one of our storage engines relies on some hard coded components, it’s listed below

  • SQLite - Whaaa? SQLite is broken on 64-bit environments? Afraid it’s true. Unfortunately on Windows systems the only solution is to recompile the entire of Mono (including Mono.SQLite.dll) yourself under a 64-bit flag.

Compatible Engines

  • MySQL - The DotNetConnector provided by MySQL is 100% compatible and probably the recommended adapter for production installs.
  • MSSQL - Windows-only however MSSQL is also free and the adapter is built into .NET itself so is pretty much guarunteed compatible.

Other incompatible components

Perenially we have problems with the following components in addition to the above, this includes libsecondlife’s OpenJPEG version - this can be fixed by compiling libsecondlife and OpenJpeg (using make) on the target system. This may have been fixed recently however as complaints about this DLL have lessened sharply.

Running

Once you have swapped incompatible components out, or recompiled them appropriately, OpenSim should run under a 64-bit environment natively on Windows. As mentioned above, under Linux/Mono you will need to make sure that Mono has been compiled as a 64bit application for this to take effect.

The differences between the two are mostly marginal, however there are some definitve improvements in certain mathematical loops and stability when using large amounts of memory. In production environments, running 64-bit mode can be an attractive option, however will require constant maintainence to compile the above libraries as new releases occur.

Written by Adam Frisby

August 9th, 2008 at 7:30 pm

Posted in OpenSim

Tagged with , , ,

Leave a Reply