<?xml version="1.0" encoding="UTF-8"?><rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
		>
<channel>
	<title>Comments on: Inside the Asset Server</title>
	<atom:link href="http://www.adamfrisby.com/blog/2009/07/inside-the-asset-server/feed/" rel="self" type="application/rss+xml" />
	<link>http://www.adamfrisby.com/blog/2009/07/inside-the-asset-server/</link>
	<description>ZOMGWTFHAI</description>
	<lastBuildDate>Tue, 15 Jun 2010 00:36:58 +0000</lastBuildDate>
	<generator>http://wordpress.org/?v=2.8.4</generator>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
		<item>
		<title>By: Adam Frisby</title>
		<link>http://www.adamfrisby.com/blog/2009/07/inside-the-asset-server/comment-page-1/#comment-8403</link>
		<dc:creator>Adam Frisby</dc:creator>
		<pubDate>Tue, 08 Sep 2009 10:54:10 +0000</pubDate>
		<guid isPermaLink="false">http://www.adamfrisby.com/blog/?p=379#comment-8403</guid>
		<description>The reason LL dont &#039;update&#039; existing assets is it makes the assets uncacheable - which is a huge performance bottleneck.</description>
		<content:encoded><![CDATA[<p>The reason LL dont &#8216;update&#8217; existing assets is it makes the assets uncacheable &#8211; which is a huge performance bottleneck.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Khamon Fate</title>
		<link>http://www.adamfrisby.com/blog/2009/07/inside-the-asset-server/comment-page-1/#comment-8391</link>
		<dc:creator>Khamon Fate</dc:creator>
		<pubDate>Mon, 31 Aug 2009 15:11:59 +0000</pubDate>
		<guid isPermaLink="false">http://www.adamfrisby.com/blog/?p=379#comment-8391</guid>
		<description>I&#039;d be interested to read an article detailing the inventory process and explaining the reasoning behind not limiting inventory references on the Second Life grid, if you know those reasons.

The duplication prevention system is brilliant and brings to mind, though I&#039;m not sure why, the fact that SL notecards are &quot;recreated&quot; meaning, I believe, that they&#039;re assigned a new UUID every time they&#039;re updated.  That&#039;s what I&#039;ve been told is the complication in allowing scripts to write to them which is an interesting point you might add to your inventory article when you write it because I told you to so there.</description>
		<content:encoded><![CDATA[<p>I&#8217;d be interested to read an article detailing the inventory process and explaining the reasoning behind not limiting inventory references on the Second Life grid, if you know those reasons.</p>
<p>The duplication prevention system is brilliant and brings to mind, though I&#8217;m not sure why, the fact that SL notecards are &#8220;recreated&#8221; meaning, I believe, that they&#8217;re assigned a new UUID every time they&#8217;re updated.  That&#8217;s what I&#8217;ve been told is the complication in allowing scripts to write to them which is an interesting point you might add to your inventory article when you write it because I told you to so there.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Adam Frisby</title>
		<link>http://www.adamfrisby.com/blog/2009/07/inside-the-asset-server/comment-page-1/#comment-8292</link>
		<dc:creator>Adam Frisby</dc:creator>
		<pubDate>Wed, 29 Jul 2009 07:49:32 +0000</pubDate>
		<guid isPermaLink="false">http://www.adamfrisby.com/blog/?p=379#comment-8292</guid>
		<description>@Will: Asset ID&#039;s should be being issued by the asset server itself as part of the data upload; which avoids collision; however for compatibility purposes - you can upload an asset with a specific destination ID, however if that ID is already occupied, the upload is ignored.

UUIDs on the backend are actually fine when you arent using MySQL. MySQL optimises UUIDs very badly; treating them the same as any other text field (which arent good for indexes) - Postgres, MS-SQL and others either have a GUID class which treats it as a 128-bit integer, or handle text indexes better.

Inserting into MySQL; it&#039;s actually best to convert 128-bit into 4x32-bit integers, then make one &#039;unique&#039; key from the composite of all four fields. That hints MySQL enough to treat it as a 128-bit int and gets you the performance you want; albeit at the cost of not being able to read the UUIDs easily anymore.

@Flip: We dont use UUIDs for table joins anywhere - OpenSim is fairly naieve and doesnt need them; all the transactions we do are on a key/value store basis mostly. (which works well for scaling into distributed systems)

Table joins are only used for things like analytics -- and we have to be careful with some of those, since it&#039;s very easy to cause a nasty table lock which pauses the whole grid.

As for memcache - I can tell you OSGrid does this already; not memcache itself - but we do rely on a frequently accessed files cache to boost performance on the top requested 3GB of assets.</description>
		<content:encoded><![CDATA[<p>@Will: Asset ID&#8217;s should be being issued by the asset server itself as part of the data upload; which avoids collision; however for compatibility purposes &#8211; you can upload an asset with a specific destination ID, however if that ID is already occupied, the upload is ignored.</p>
<p>UUIDs on the backend are actually fine when you arent using MySQL. MySQL optimises UUIDs very badly; treating them the same as any other text field (which arent good for indexes) &#8211; Postgres, MS-SQL and others either have a GUID class which treats it as a 128-bit integer, or handle text indexes better.</p>
<p>Inserting into MySQL; it&#8217;s actually best to convert 128-bit into 4&#215;32-bit integers, then make one &#8216;unique&#8217; key from the composite of all four fields. That hints MySQL enough to treat it as a 128-bit int and gets you the performance you want; albeit at the cost of not being able to read the UUIDs easily anymore.</p>
<p>@Flip: We dont use UUIDs for table joins anywhere &#8211; OpenSim is fairly naieve and doesnt need them; all the transactions we do are on a key/value store basis mostly. (which works well for scaling into distributed systems)</p>
<p>Table joins are only used for things like analytics &#8212; and we have to be careful with some of those, since it&#8217;s very easy to cause a nasty table lock which pauses the whole grid.</p>
<p>As for memcache &#8211; I can tell you OSGrid does this already; not memcache itself &#8211; but we do rely on a frequently accessed files cache to boost performance on the top requested 3GB of assets.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: Will Dowling</title>
		<link>http://www.adamfrisby.com/blog/2009/07/inside-the-asset-server/comment-page-1/#comment-8291</link>
		<dc:creator>Will Dowling</dc:creator>
		<pubDate>Wed, 29 Jul 2009 01:33:16 +0000</pubDate>
		<guid isPermaLink="false">http://www.adamfrisby.com/blog/?p=379#comment-8291</guid>
		<description>Whilst I think GUIDs are still a bit shite for UIDs, I can see why you&#039;ve used them here. I&#039;m still not convinced they&#039;re ideal for the backend storage layer however (as FlipperPA mentions).

Perhaps it would be appropriate to have the backend database node keep an index of it&#039;s internal (highly optimised and indexed) ids (straight INT/BIGINT UNSIGNED) and mapped to the GUIDs that the requesting application would use.

Again, I&#039;m still interested in helping out with this side of things - but I think with my other commitments (for now) I&#039;d need to be given a specific problem and scope. You have my number Adam :)

Incidentally, What would happen if a new asset server came online with objects with the same GUID (via chance or even a malicious user)? It&#039;s vital that the platform can handle this gracefully, especially for a place such as OSGrid.</description>
		<content:encoded><![CDATA[<p>Whilst I think GUIDs are still a bit shite for UIDs, I can see why you&#8217;ve used them here. I&#8217;m still not convinced they&#8217;re ideal for the backend storage layer however (as FlipperPA mentions).</p>
<p>Perhaps it would be appropriate to have the backend database node keep an index of it&#8217;s internal (highly optimised and indexed) ids (straight INT/BIGINT UNSIGNED) and mapped to the GUIDs that the requesting application would use.</p>
<p>Again, I&#8217;m still interested in helping out with this side of things &#8211; but I think with my other commitments (for now) I&#8217;d need to be given a specific problem and scope. You have my number Adam <img src='http://www.adamfrisby.com/blog/wp-includes/images/smilies/icon_smile.gif' alt=':)' class='wp-smiley' /> </p>
<p>Incidentally, What would happen if a new asset server came online with objects with the same GUID (via chance or even a malicious user)? It&#8217;s vital that the platform can handle this gracefully, especially for a place such as OSGrid.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: FlipperPA Peregrine</title>
		<link>http://www.adamfrisby.com/blog/2009/07/inside-the-asset-server/comment-page-1/#comment-8281</link>
		<dc:creator>FlipperPA Peregrine</dc:creator>
		<pubDate>Sun, 26 Jul 2009 19:01:22 +0000</pubDate>
		<guid isPermaLink="false">http://www.adamfrisby.com/blog/?p=379#comment-8281</guid>
		<description>One other question I forgot to ask: any plans to support memcache (or something similar) in the future for frequently accessed assets? While it wouldn&#039;t show a huge performance increase with what OpenSim is being used for today, down the road, it could be quite beneficial.</description>
		<content:encoded><![CDATA[<p>One other question I forgot to ask: any plans to support memcache (or something similar) in the future for frequently accessed assets? While it wouldn&#8217;t show a huge performance increase with what OpenSim is being used for today, down the road, it could be quite beneficial.</p>
]]></content:encoded>
	</item>
	<item>
		<title>By: FlipperPA Peregrine</title>
		<link>http://www.adamfrisby.com/blog/2009/07/inside-the-asset-server/comment-page-1/#comment-8280</link>
		<dc:creator>FlipperPA Peregrine</dc:creator>
		<pubDate>Sun, 26 Jul 2009 18:58:26 +0000</pubDate>
		<guid isPermaLink="false">http://www.adamfrisby.com/blog/?p=379#comment-8280</guid>
		<description>I can verify that Linden Lab is using InnoDB, not MyISAM (or other). Good choice. :)

I haven&#039;t looked at the DB schema for OpenSim in a while - while I agree UUIDs are great for unique identifiers, they are several orders of magnitude slower than integers when doing things within a DB implementation, especially when joining tables, etc. Are there plans to keep UUIDs for unique lookup, while using integers for things like joins within a DB instance? Great work, and way to break it down in English, Adam.

Regards,

-Tim / Flip</description>
		<content:encoded><![CDATA[<p>I can verify that Linden Lab is using InnoDB, not MyISAM (or other). Good choice. <img src='http://www.adamfrisby.com/blog/wp-includes/images/smilies/icon_smile.gif' alt=':)' class='wp-smiley' /> </p>
<p>I haven&#8217;t looked at the DB schema for OpenSim in a while &#8211; while I agree UUIDs are great for unique identifiers, they are several orders of magnitude slower than integers when doing things within a DB implementation, especially when joining tables, etc. Are there plans to keep UUIDs for unique lookup, while using integers for things like joins within a DB instance? Great work, and way to break it down in English, Adam.</p>
<p>Regards,</p>
<p>-Tim / Flip</p>
]]></content:encoded>
	</item>
</channel>
</rss>
