Somewhere in the deep dark recesses of my backup server, is a little directory called ‘AWSERVER’, last modified circa 1999. Inside that directory is my world database and the content path associated with my old ActiveWorlds regions. With a little (or maybe a lot) of work, I could probably re-load it today and log in.Yet, if I tried to revisit some of my full-sim creations in Second Life from 2004, I’d have a lot of trouble trying – I’d need to recover what I could from my inventory and do painstaking rebuilding work, if it was possible at all.
This is the problem with SL – the moment something is deleted, or a region is shut down – it’s almost always gone forever. If you wanted to revisit a earlier incarnation of Nexus Prime, or parts of SL’s ancient history – you are out of luck, as there’s a very solid chance those places are simply irrecoverable and lost to the sands of time. As digital data, there should be no good reason for this – disk space is cheap, and sims are small.
The classic case of this is the Bedazzle sims, among them were Gravity Space Station, Chinatown and UnrealSL – but none of them lasted more than between a few weeks or a few months. I’m digressing here slightly; but there are major projects I have been involved in SL that I would love to be able to still access that are gone forever (earlier versions of Aleph, the Atlas Underwater Complex, etc).
The problem isn’t so much that SL doesn’t store ancient rollbacks but that it is simply not possible to save a copy of one; even if you are the rights owner and want to back up your own work. Second Inventory can help here, but it too has flaws – it doesn’t have any kind of mass restore functionality; and it can only save inventory – there’s no chance to save the layout within a region, only the individual contents of it.
It is somewhat sad to see regions shut down by their owners for affordability reasons; knowing full well that the content cannot be ever easily restored later – I personally hate to see it when this happens, because something creative is lost forever.
OpenSim on the other hand, has some real advantages here – I have complete copies of a lot of my builds on OpenSim in varying stages of construction, courtesy of the Region Archive functionality. Every major construction project I have done on any of the grids is sitting somewhere on one of my hard disks as a .tar.gz file containing everything needed to reload it in later. In OpenSim, nothing is ever incapable of being saved – at all times you can dump a copy of the region to a disk, then reload it later somewhere else.
As a creator it’s fairly liberating – and convenient. I can work on a sim locally, export it, then import it into the production environment, and vice versa, take a production environment for local tweaks, edit it, then bring it back again.
Backing up Aleph
I have for a few months been testing an internal tool which allows you to export a OpenSim Archive from a Second Life Region – it was originally developed to export a clients region (their IP); but ended up being handy to preserve some of our workshops and builds from deletion when we closed the sims or rebuilt them. Today, I rewrote it – the previous version was based on the old libomv PrimWorkshop viewer, the new version is now based on the Simian Periscope (Periscope is a kind of multi-user version of GridProxy).
Before anyone asks, the modifications aren’t public – unfortunately for every legitimate user for a tool like this, there’s ten asshats prepared to use it as copybot deluxe, so the source is going to stay private (although I might release a binary version containing creator and permission checks similar to Second Inventory – we’ll see what my schedule looks like in the next few weeks).
This new version is overall a bit more reliable – a number of small bugs and niggles got fixed along the way – but the key factor is it’s now not a 2 hour effort to run, a region can be grabbed with 95%+ accuracy in minutes. You can see here, my personal workshop region ‘Aleph’ in Second Life – it’s a fairly old sim, but it’s gone through a ton of revisions in it’s history. The current revision is a sort of moonbase cross sandbox, complete with orbital lasers.
Aleph Null
Below you can see the same region and contents, but in my personal standalone OpenSim region. This one is located on my personal desktop – but with the same OAR file, I could just as easily reload it on any region running any version of OpenSim since OAR support was added. If I wanted to bring Aleph to OSGrid, it would take only as long as it took to copy and load the file on a region connected to OSGrid.
127.0.0.1:9000
There are limitations, the tool doesn’t copy any form of ‘deep inspection’ – so scripts, etc do not get saved. It’s theoretically possible however to back these up if you are the object owner, something I will be looking at in the future. Estate settings and a few other features aren’t in the v1 OAR format, so those also need to be recreated – but could be something we look at adding in future. It’s also worth noting that it will only backup content with a creator tag in a specified list; while that limit could be removed, it provides an easy way of assuring that you are backing up only content you have rights to.
The Wayback Grid
One of the projects I would like to see would be some kind of opt-in mechanism to subscribe to an automatic backup service similar to Archive.org for the web – the goal of which would be preservation of content in the long run. When someone drops their region, it would be nice to be able to restore it later on if they buy another region, or move to one of the open grids.
Ultimately, this could be taken to an extreme where you can dial back a grid in time, and see it as it was in a previous point in time – however the biggest limitations here are dealing with content prosciption. Most SL content is licensed with one or more restrictions – ethically ‘no copy’ is probably a blocker to performing backups, likewise ‘no transfer’ makes some implications about bringing something out of SL. To do this effectively, you need massive ‘opt-in’ by content creators to approve their content going outside.
The easiest starting point is instead probably to provide some kind of service for creators to voluntarily backup their sims (say, in the case of a creator shutting down their region – but wanting to preserve it). Perhaps there are options here to look at providing some kind of transfer service for people moving regions from say SL to OSGrid [providing they own the copyright]. If anyone is interested in that kind of service, let me know – this might be useful for folks contemplating migrating over to OpenSim/OSGrid and have all their own content.
Nexus Prime almost was immutable and immune, but someone had to go and erase half of it under the belief it’d ‘evolve’. Tsk. Something like your opt-in backup idea would have probably saved it from its half-empty status of right now.
Belaya
11 Jun 09 at 11:16 pm
Yeah, the sad thing is the average archive I have exported is about 30mb zipped. Storage costs are not in any way meaningful here I don’t think.
Adam Frisby
12 Jun 09 at 4:13 am
So much for opensource, has to hide the source of something ROFLMAO.
And it’s wonderful to get an admission on the record that “for every legitimate user for a tool like this, there’s ten asshats prepared to use it as copybot deluxe”
Your comrades instead go around lying instead that there is never, ever any copying on the open sims and DMCA notifications really do work.
Prokofy Neva
12 Jun 09 at 11:34 am
I tell the schools we work with they can have students create sims/worlds for each class and save them all like a virtual yearbook. We do this with oar files and even have the ability to “snapshot” the entire Operating system & grid/database using Hyper-V virtualization. I find the backup abilities alone very compelling and a major reason reason why we love OpenSim.
Kyle G-CEO ReactionGrid
See you in 3D!
Kyle G
12 Jun 09 at 1:33 pm
My deep and heartfelt congratulations, Adam. As you said, it’s sad to see how beautiful creations are lost when owners decide to leave their regions. Thanks to your work, some of these artists can enrich the openmetaverso moving regions from SL.
Albert Revolution
12 Jun 09 at 4:04 pm
Interesting to hear about the tool for extracting information to OAR from the client side, Adam. People seem to be tiptoeing again towards content transfer in this way (via client, not necessarily via OAR). I hear that the Meerkat viewer has content transfer facilities built in.
You’re right in that estate information saving is missing from the OAR format (which is technically 0.2 at the moment). It wouldn’t be too hard to put it in 0.3 – it’s just ordinary coding work rather than anything tremendously complicated. Anybody who wants to submit a patch for it is very welcome.
Justin Clark-Casey
12 Jun 09 at 8:56 pm
Nice work Adam. Whilst it can feel quite quaint and artistic to consider the builds we do in SL as transient places, ever evolving we are in fact losing a piece of history and of a fast moving culture with no real archiving in SL of any sort.
I realized this when I had to change Hursley and IQ sim into flat regions again in order to rent them out. We all took pictures and what we could, but a permanent historic record would have been so much better. After all this is something we cant do in RL
You might be interested in what Rodney Harrison says about archeology in SL in excavating the internet on this Open University podcast. http://deimos3.apple.com/WebObjects/Core.woa/Browse/itunes.open.ac.uk.2083662357
epredator
16 Jun 09 at 7:32 am
we are planning to release one of the tools we used to transfer ALL our sims over to OARs. Just finishing some stuff on it now, so it should be out in next week. Go take a look at the King Tut sim on heritage-key.com and you will see that it is identical content. In fact it is not split onto 6 opensim regions instead of clogging one SL sim. Cross-grid tps work fine as well (ok, we loose a few attachments every now and then).
rightasrain
16 Jun 09 at 6:38 pm
Very interesting article. I am working in Machinima and using both a region on the OSGrid and a Standalone for formulating my sets. I would like to be able to basically package up a region when I am done with a set and e able to store it away and then restore it when I need it.
With the current tools available, which method would you recommend for doing this task?
Looking forward to more articles
Mike Joyce
19 Jun 09 at 8:18 pm
Just be aware that if you incorporate any GPL code — any at all — into your tool, then you will have to release the source as well — not just a binary, as you stated above.
Linking to GPL libraries is okay, but if any of the code is inside your binary, and not dynamically linked, then that becomes a GPL violation.
Jeremy
30 Jun 09 at 3:35 am
Very cool! Interesting to speculate how different things might be if SL (say) had had this sort of capability (either server-side or client-side) from the beginning.
I have to ask, though: are they orbital *mind control* lasers?
Dale Innis
1 Jul 09 at 5:54 pm
I’m quite aware of the GPL – thankfully in this case, it’s all BSD, so the point is mooted.
Adam Frisby
5 Jul 09 at 9:41 am
Adam – I am ecstatic to hear about your new tool and how seriously you are taking the responsibility of its release. Can’t tell you how many times I’ve spoken to a coder involved in OpenSims say one of the following:
* Content protection is impossible
* Content Providers are a fool to think they can make a living out of making content.
* All content should be free and open source like the OpenSim software.
The reality is we need to have a system that allows the flexibility of a BuilderBot program with the security of say a Flash document. Sure, the very savvy coders and web dev folks of the world can hack at Flash assets, but on the whole its not profitable to do so and time consuming. And its simply not generally known how to by the casual user. BuilderBot and other such programs are so simplistic as to make it easy for a very younger child to execute.
What is needed is something that recognizes the person’s rights who made that content has a say in how it can be reused to a certain extent, yet allows their clients the freedom to use those same assets with reasonable mobility. Here’s why, seeing some folks just don’t understand the echo effect mass ripping of content can bring to bare.
* Corporations who purchase work should be able to move their content around, but not resell/give away to other persons or grids. They also DEMAND IP protection from random spoofing and competition copying their ideas and marketing materials. That’s why they spend millions every year on original content to artist like the content providers in SL.
*As much as we all hate the legal wrangling of business, its not going away and without reasonable security of these assets its a hard sell to their legal departments to give the green light. Without their approve a spending voucher just is not forthcoming.
* It should not be so easy for a 8 year old blooming coder to pick up the code for such a tool and go around copied assets from private and corporate entities at will and without any means to detect and prevent.
* The individual content creator should have a right to review any grid that their content is able to go into and give an approval for that grid and the specific region even that its going into – a sorta permission to rez on a certain grid or region or not.
This is very doable and if you’d like to hear how, just drop me a line at tkinneyjohnson@gmail.com.
There are three ways to accomplish this, that if done in unison would make for not only happier content providers, but a far more reputable OpenSim grid community. Without these reasonable restraints there is no incentive to produce high grade game quality content at low affordable pricing. Artist will have to up their prices to compensate for the loss of revenue that frankly they do deserve. Making great content is NOT a simple thing and takes real talent and expertise. In Summary, yes its impossible to get 100% protection. What we strive for is to replicate the success of real world product security for content providers. Retail outlets regularly sustain a 10-20% theft rate. How they manage that is by making it higher risk and more tedious work to complete the act of theft. That’s all most content providers are asking for
1.) Require real world info in exchange for the purchase of such a program. This can easily be done without being intrusive by requiring it to be purchase with a VERIFIED PayPal account.
2.) Make the price high enough that the average casual user will pass, but low enough that a serious developer or corp entity can afford. This is usually in the $150-$500. After all, why should such a powerful tool cost less than the hard drive it would be stored on? We see that hardware as a very affordable and necessary expense when backing up our content, yes? Is this really any different in value?
3.) Once this tool is able to grab scripts and animations and nested content, I think we’ll start seeing the coders of our industry get a bit more concerned over content protection, because up till now there work was immune from theft. In that vein maybe what needs to be added to these programs is a reporting tool that detects the creators and sends out notices of who is running the software and what they’ve grabbed. This would require a centralized DB of content providers to check for, but I’m sure most designers would have no problems signing up for that in exchange for more security on their assets. And when you couple this with the verification of real world ID through the purchase using PayPal I think anyone intent on ripping content, either for person or profit, would have second thoughts. It sets a precedence for accountability.
And that’s what this is really about – tugging a bit on our conscious. The ethical part of our humanity knows its wrong to steal stuff we have no rights to, but where there are no checks and balances to make us THINK before we ACT, then that’s where we get in trouble as individuals and as a society. Its really all about respect …. of other’s livelihood in many cases, AND OUR OWN SELF-RESPECT.
Tessa H
7 Aug 09 at 6:16 pm
@Tessa: Personally, I think the bar should be set at “Requires a specialised tool to infringe”. It’s impossible to reliably secure above that bar.
At that point, it’s where we are today with Second Life – there are countless ways to infringe content in Second Life. Copybot and glIntercept are just two (of many); the key that allows Second Life commerce to succeed is that they are specialised tools that the majority of customers are not interested in.
Going above the ‘needs specialised tool’ has seriously diminishing returns – it leads to erosion of consumer rights (see service-locked DRM on music for a good example); and it overall does not work. (see my post last year on the topic: http://www.adamfrisby.com/blog/2008/07/virtual-worlds-why-drm-cannot-protect-you-for-long/ )
The larger concerns in Second Life, et al – is the principle of commercial copyright infringement, while we can assume a fixed percentage of the population is fine with pirating content; the other 95% can be tricked into purchasing illegitimate content unknowingly. I personally feel there should be ways of pursuing those people for commercial copyright infringement to put greater dissuasion on the practice. (this forms into a larger topic which I want to write a post on later)
You can make a basic effort to reduce infringement by the average user – by say, not encouraging infringement in the basic tools and requiring more specialised utilities to proceed; but as I’ve mentioned repeatably before – trying to fight those specialised utilities is futile; the fortunes have been spent on things such as DeCSS, BD+, and every single one has been cracked quickly and without remorse.
Which leads me onto another topic – the importance that there is also truth in advertising – there’s a lot of OpenSim grids out there who claim to have ‘Copybot protections’ – but do not advertise how easily they are defeated. (and I will quickly edit in here – this isnt a personal attack on your operations, since I do know you do this; but rather to comment and say that it should be part of a wider broader solution.)
A number of grids do things such as block LLSD logins, or redirect clients to try block ‘copybot’ logins; without realising how easy it is to bypass those restrictions. There are other techniques being employed too; but while they may work in the short term while there is zero demand for clients adapted to that grid, if the grid ever got popular; it would be a simple case of adjusting the login methods to be indistinguishable from the mainline clients.
Adjustments which are made easier by the GPL license on the main client — making it impossible to distinguish between a legitimate and fraudulent client, because you can always request the source. [I'll add onto that - you can do it without the source, but it does require a more specialised programmer to dissasemble or reverse engineer your methods]
Not advertising the ease of bypassing accurately burns the operator/creator relationship in the long term – because they cannot trust what you say to be the fact. This is one of the reasons I prefer OpenSim developer communications to be leaning towards the ‘you can do anything’ angle; because it means that when something goes wrong – no-one is caught unaware of the possibility.
To pretend there are magic bullets, or even effective countermeasures is disingenous; and while it may soothe the souls of creators in the short term – it is equivilent to sitting on a powderkeg when someone releases the first adapted tool.
Adam Frisby
8 Aug 09 at 6:42 am
Hi,
Great! My compliments to you Adam, it is very interesting!
We’re a company and we’re planing to place our islands on opensim, when do you think this tool will be available?
Cinzia
20 Oct 09 at 8:28 am
Hi Cinzia,
I still need to do a lot of testing with the permissions side of the tool, so right now I am running it manually for people on a case-by-case basis – email me if that interests you. (adam@deepthink.com.au)
Adam Frisby
21 Oct 09 at 2:28 am
Hi,
thanks for your kind and fast reply, we’re very interested in this tool; can we help you with testing procedure?
We have a SIM in SL (FieraSL) that should be moved in opensim, the project has been designed by three builders of us (we have their accounts) and 99% of objects has been made by us.
Thanks in advance
Cinzia
21 Oct 09 at 7:58 am
Hi,
we’re very interested in your tool (read in your blog: http://www.adamfrisby.com/blog/2009/06/the-impersistence-of-memory/), we’re a company and we’re planing to move our sim (FieraSL) on opensim, can we help you with testing procedure?
The whole project has been designed by three builders of ours (we have their accounts) and 99% of objects has been made by us. Let us know please.
Thanks in advance
Cinzia
22 Oct 09 at 1:58 pm
Hi Adam – very interested in using this technology to back up and move the sLiterary sims. Could you release further information? Shakespeare sim was designed entirely by me, as well as Skin City and most of sLiterary. Primtings Museum island and architecture is mine, though much of the content rests in individual exhibits but most artists would likely be willing to port their exhibits elsewhere.
Ina
21 Nov 09 at 9:03 am