TojiCode

...in which I give updates on everything

2012-02-09T11:16:00.000-07:00

Also known as: Why I've been so lazy lately.

So I know that theres more than a couple of you out there that have been very patiently awaiting the next post in my Building the Game series for the last couple of months. It's probably been a little frustrating to have the first 6 posts come out fairly quickly and then just abruptly cut off. For that I apologize.

The reality is that I've been buys with plenty of other, non-blog-related things lately and simply haven't been able to spend the time working on it. During the holidays I made a conscious decision to focus on family rather than code, and immediately after that I've been pretty swamped with a project from work (you know, the guys that actually pay me!) that has left me with little desire to do even more code when I get home. On top of that, I quickly realized that the where I wanted to take the bog series next would require some research into areas that I've never really poked into before (generating visible sets and such) and it's been difficult to find the time and resources to really dig into it.

In other words, I've been feeling a little burnt out. I think all programmers go through this from time to time, and I'm simply allowing myself to recover and gain back some of that all important enthusiasm.

Don't fret, this isn't my "well that was a failed experiment" post where I leave things forever unfinished. I'm going to continue the series soon, but I think I'll be back tracking a little bit and hitting some other aspects of it first before getting back into the whole level loading thing. Better animation management and player input feels like a good place to hit next, along with some retrospective tweaks to the model formats. I don't think I can give dates on when we'll see the next post, but rest assured it will be sooner than later.

Beyond the blog, however, there's been a couple of other developments worth mentioning lately. Since the contract is now signed I think it's safe to announce that I'm currently co-authoring a WebGL book! I was asked to help Diego Cantor complete a WebGL Beginners Guide for Packt Publishing, which has proven to be no small feat. I don't know anything really about potential release dates or anything like that, but rest assured that I'll be tweeting and blogging about it once I know more.

Also, I've been invited to attend Mozilla's HTML5 Games Work Week this February. Essentially, Mozilla is reaching out to people who have been active in game-related HTML5 tech (like WebGL) and is bringing them together for a week to ask "How can we make the web an awesome game platform?" This is a goal I think we can all support!

For my part, I want to try and address several concerns from the development community that I've run into either through this blog, via email, or that I've seen talked about on various other online resources. These include:

Synchronizing game logic and rendering
Considerations for reducing power consumption of games on mobile devices
Content protection
Content delivery
Robust input handling
High-performance audio

But I'd also love to hear what the rest of the HTML5 game dev community is concerned about! I can't promise that everything that I hear about will be addressed but if I'm hearing similar concerns from multiple people you can be sure I'll do my best to bring it up.

That's about it for now. There's a couple of conferences on the horizon that I may have a chance to speak at but nothing is confirmed right now. I'll let you know if things solidify. There's also some exciting new and upcoming bits of web tech that I'm itching to play with and do some little demos for, so expect to see some quickie proof-of-concept stuff in the coming months as well. In the meantime, I'm always available for questions by email, on twitter, or through this blog, and I've made a habit of trolling StackOverflow for WebGL questions.

WebGL JS1K entry

2012-02-09T01:55:00.001-07:00

As a sort of follow up to last years Itty Bitty WebGL post I decided to take a crack at actually entering the js1k competition this time around. Since the theme of the competition is "Love" I thought it would be fitting to do a minimalistic version of this awesome little "beating heart" shader.

You can see my under 1k version here.

And the not-quite-so-compressed-but-still-really-hard-to-read source code here:

c=document.getElementById('c');
c.height=300;
g=c.getContext('experimental-webgl');

b=g.createBuffer();
a=34962;
g.bindBuffer(a,b); 
g.bufferData(a,new Float32Array([0,0,2,0,0,2,0,2,2,0,2,2]),35044); 
g.vertexAttribPointer(0,2,5126,false,0,0);

p=g.createProgram();
function l(x,y){
    s=g.createShader(35633-x);
    g.shaderSource(s,y.replace(/#/g,'float ').replace(/\^/g,'vec'));
    g.compileShader(s);
    g.attachShader(p,s);
}

// See the uncompressed shader code at http://glsl.heroku.com/e#1599.2

l(0,'attribute ^2 p;void main(){gl_Position=^4(p.x-1.0,p.y-1.0,1,1);}');
l(1,'precision highp #; uniform #t;^2 r=^2(300,300);^3 h(#x,#y){#s=mod(t,1.0);s=1.0+(1.0-exp(-5.0*s)*sin(9.0*s));x*=s;y*=s;#h=abs(atan(x,y)/3.14);#d=(13.0*h-22.0*h*h+10.0*h*h*h)/(6.0-5.0*h);return mix(vec3(1.0,0,0),vec3(1.0),smoothstep(d-0.05,d,sqrt(x*x+y*y)));}void main(){^2 p=^2(-1.0+2.0*gl_FragCoord.xy/r);gl_FragColor=^4(h(p.x,p.y),1.0);}');

g.linkProgram(p);
g.useProgram(p);
g.enableVertexAttribArray(0);

u=0;
setInterval(function(){
    g.uniform1f(g.getUniformLocation(p,'t'),u+=0.02);
    g.drawArrays(4,0,6);
},16);

To be perfectly honest, I really don't care to explain the details of what's going on here. The basic setup is the same as my previous post on the subject, with the core difference being that we're now rendering a fullscreen quad. (That effectively makes this a tiny version of mr doobs GLSL Sandbox) The shader code is really just a minimal implementation of the original shader, and the only really interesting trick there is that I'm doing a string replace to simplify commonly repeated symbols like "float" and "vec".

I'm not certain that this even qualifies for the contest (since WebGL isn't enabled by default on some of the required browsers) but I had fun doing it and it was a good break from everything else going on in my life right now. (I'll be posting more on THAT shortly.)

OVERNIGHT UPDATE: So apparently I missed the line on the rules page that actually excludes WebGL from the competition (Second to last bullet under "Other things you should know"). That's a shame, but hopefully WebGL will be allowed for the next round! Doesn't matter much to me, I was just doing it for kicks anyway.

In other news, @p01 managed to cull 303 bytes out of my demo while I was sleeping which I think is deserving of a "I'm not worthy!" bow. There's some seriously clever little tricks in his version that I'm most certainly going to have to utilize the next time I try one of these! :)

Protecting WebGL content (and why you probably shouldn't)

2011-12-27T11:20:00.001-07:00

I received an email this morning from a developer that is in a predicament that I think many WebGL developers will find themselves in pretty soon:

Do you have any experience with protecting assets in WebGL? I wrote a quick solution to protect textures during transfer - base64 encoding + concat server side and splitting + creating imgs client side. Still, using WebGL Inspector I can get all textures transferred to the graphics card. I wonder if you have any ideas on how to prevent that. I don't mind open sourcing my apps, but my clients apparently do :)

Ah, that last line is really the crux of the matter, isn't it? Most developers I know don't mind sharing their creations with the world at large, as we tend to understand that there's rarely anything sacred and secret in the lines of code we crank out, but that's not always an easy thing to convince management of. So: What's an enterprising WebGL developer to do when the boss decrees from on high "Protect our content?"

There's a couple ways to approach such a request, in my opinion. First: Do whatever you can to convince said management that you don't need to protect said content. Second: Convince them harder. Third: If all else fails, obscure the content as much as possible. I'll be talking about both approaches in todays post.

Why your content doesn't need protecting, and why it benefits you to leave it open!

You probably don't need to protect your content in the first place. I know that many people's knee-jerk reaction to that statement will probably be: "But you don't understand our situation! Our content is special!" But please, hear me out!

Most of the time when we hear about content protection schemes (DRM), it's in relation to movies and music. There's a good reason for that, too! In those cases the movie or music is the product. But does that really apply to the assets of your WebGL app? What, exactly, is the product that you are trying to provide?

Since the original email was about textures, let's look at those as an example:

This is a wood wall/floor texture from Team Fortress 2. Opinions may differ depending on who you talk to, but in my opinion this is really a work of art. It has an inherent value even as a stand-alone image. Some talented artist put their blood, sweat, and tears into this image. Valve paid for this image to be created. In some way, big or small, the game would be a less polished, less interesting product if not for this image. It is a valuable asset.

So obviously it needs protecting, right?

Well, yes and no.

Problem is that nobody is going to print this image out and hang it on their wall. You're never going to be able to sell high-quality prints of this image. Even though it has a value, for a consumer that value approaches zero pretty quickly unless it is part of a larger product. We value the level that this resource helped create more than the resource itself, and since that level would not be the same without this resource it continues to have value to us as a consumer, but only as part of the larger scene.

So the scene as a whole should be protected, yes?

Maybe, but how much does the consumer really value a static level? People do enjoy being able to simply walk around a scene like that. And in certain industries that may be the product. Autodesk Cloud makes a good case for using resources at this level, and certainly you can imagine that static scenes like this would be valuable for architectural design or similar areas. But how about our TF2 level?

The fact is that, as beautiful as they are, almost nobody would pay to simply walk around a TF2 level. But there are lots and lots of people who will pay very good money to shoot other people while walking around said level. And this is where the actual "product" lies. It's not the assets that people care about, it's the overall experience. The assets are a big part of that experience, true, but only one part of a larger whole. That larger whole also happens to include the multiplayer servers, the achievement systems, the accessory store, the friends list, etc. All things that easily can and should be secured.

The players pay money and in return they download content to their machine, and as such it's easy to mistake the content for the product, but it's not. The ability to play a great game is the product. In other words, Valve is selling the service, not the content. And you should be too!

This is a common theme that you hear from the HTML 5 folks over and over again, and in many cases I fully agree with it. It doesn't always work, but given a bit of thought you'd be surprised at how many scenarios it can be applied to. It all comes down to what you can offer to your customers that can't be replicated on their machine.

The fact is, no matter how hard you try to prevent it the end user will alway be able to scrape out your lovingly built resources and serve them locally if they're really motivated to do so. We don't want to simply roll over and accept piracy as a fact of life, though, so we need to look at what we can provide to complete the overall experience that the users can't simply copy to their hard drive. Multiplayer servers are certainly the obvious choice, but even for single player games or non-gaming apps there's a lot that can be offered server side that lends to the "product". Community features, social network integration, achievements, realtime data feeds, the list goes on! And as web apps, we should be striving to incorporate these connected features from day one. Otherwise, why are you bothering with the web as your platform at all?

Maybe what you're worried about is the sanctity of your artistic vision, though? The author of the email above also mentioned this worry:

Any kid with firebug can download [the textures], draw a penis on my character's face and upload a modified version somewhere.

To which I say: what a wonderful problem to have! If you have a user base that is interested enough in your product to actually start modding it, you've hit a Happy Place™. Certainly nobody looks forward to seeing YouTube videos of their creations running around with crude genitals scribbled on their head, but how much damage is that really doing? One guy modifying his local texture cache won't propagate those changes to anybody else, and if anyone else goes and downloads his changes it's because they want to play as Sir CrotchFace. They get a laugh out of it, everyone else plays on oblivious, and (crucially) they're all using your product! Isn't that the goal in the first place?

It's no mistake that some of the most successful games of all time have been the ones that are easily moddable. Doom, Quake, The Elder Scrolls, Unreal, Half Life, all games that lived on well past their normal life spans because of a thriving mod community. Do you think Minecraft would be nearly as popular as it is now if Mojang had actively been discouraging texture packs and server tweaks? People love to create something that is uniquely theirs, and love to push the boundaries of what any given tech can do. It should be any developers dream to be the target of such affection.

So, long story short: Provide value through your services, not your content, and don't fear the user that wants to tinker. They're often your biggest fans!

Okay, that's great, but my boss doesn't agree. Now what?

So let's give you the benefit of the doubt and say that your content really is honestly and truly so special that it needs to be protected somehow. There are some things you can do to make it more difficult for the other party. Note that I say "more difficult" because the fact is plain and simple that if your content can eventually be displayed on a screen or played on a speaker it CAN be copied. Period. (Big Media is still struggling with this concept, apparently). They best you can do is make it inconvenient enough to discourage casual piracy.

The first way to protect your content is (and I hate to say this) not to use WebGL. Look, let's face it guys, open web standards are designed to distribute content, not hoard it. You're swimming against the current if you want to have open standards and DRM all rolled into a neat package. Content protection is almost inherently proprietary in nature, so your best bet is to go to a plugin framework like Flash, Silverlight, or Unity. These are systems that are designed from the start to distribute content securely, and thus will be much easier to work with if protection is your goal.

If "plugin" is a dirty word for you (or your boss), though, it's also worth looking at Google's Native Client. For those of you that don't know, Native Client (or NaCl, as Google likes to call it) allows you to distribute compiled binaries to users that are then executed in the browser in a sandboxed environment. Because your dealing with native, compiled code there are all sorts of nasty little tricks you can pull off to try and obscure your content from casual browsing. The downside here is that (for now, anyway) NaCl apps will only work in Google's browser. Of course, if you're that concerned about protecting your content requiring a specific browser probably isn't a huge issue.

But... what if you're the type that wants to have their cake and eat it too? You want to embrace the open standards of the web while simultaneously preventing anyone from extracting your precious assets. (You confuse me, I'll ignore that for now.)

First off, you'll want to make all of your code as hard to read as possible. Not the original source files, obviously, but the ones that you put on your public servers. The Closure compiler is great for this, and can actually make your code run faster in some cases! Bonus!

It probably also goes without saying that you'll need to invent your own formats for just about everything. Proprietary mesh and texture formats are always going to be harder to read than a standard COLLADA file and PNG.

Again I'll focus on textures for the moment: You can encrypt your texture all that you want, but at some point you have to unpack it into something that can be displayed on screen, right? The idea is to delay that step as long as possible to avoid having any intermediate tools like Firefox or WebGL Inspector be able to display it normally. To this end, you can get really creative with how you unpack things in your shaders. Ben Adams has a great post about his experiments with manually unpacking compressed textures in a shader that provides some insight into the type of steps that could be taken here. In any case, the idea is to have the fragment shader (which should be distributed obfuscated) do the appropriate math to decrypt/decompress your texture as it's getting written out to screen. Obviously you'll take a performance hit in doing so, but once again I'm guessing that isn't a problem if you are really concerned about protecting your assets.

(You can similarly decrypt vertex buffers in the vertex shader if you're worried about those being lifted, but they tend to be more obscure by nature anyway.)

Another thing to help prevent textures from being easily visible is that you don't have to use HTML images as your texture source. You can send TypedArrays to the GPU directly as your texture source as long as you provide the dimensions and format of your data. I use this to create textures of a flat color all the time:

function createSolidTexture(gl, r, g, b) {
var data = new Uint8Array([r, g, b]);
var texture = gl.createTexture();
gl.bindTexture(gl.TEXTURE_2D, texture);
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGB, 1, 1, 0, gl.RGB, gl.UNSIGNED_BYTE, data);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.NEAREST);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.NEAREST);
return texture;
}

This means you can request the texture as an opaque binary through an AJAX call rather than with the Image object, which prevents it from showing up in the Chrome developer tools. (It will still show up in WebGL Inspector, though, so encrypting the texture values is still needed for maximum obscurity.)

Beyond that... well... I think you're out of luck. Certainly there's nothing that I've suggested that prevents someone from tracing through your code, figuring out the algorithms, capturing the data streams, running them through the same decryption process, and reconstructing the original files. But as I said earlier, the idea is to make it inconvenient, not impossible.

But if you've got a user that's really so bound and determined to reverse engineer your entire app just to access the underlying content, I would ask you to think really hard about why you're trying to stop them in the first place.

Compressed Textures in WebGL

2011-12-09T23:22:00.001-07:00

I gave a presentation this last Friday at WebGL Camp 4, the slides of which are online now. I had a great time, met some awesome developers, and saw a lot of things that got me really excited about the future of WebGL. I highly encourage anyone that is interested in WebGL to try and make it to WebGL Camp 6!

During my talk I was able to show what I think may be the first public demo of compressed textures in WebGL! The demo isn't terribly impressive, it simply displays a DXT5 texture loaded from a DDS file, but it shows off the required code effectively enough.

http://media.tojicode.com/dds/

(Warning: That demo will only work on machines that support DXT5 compression. That should be most desktops, but the majority of mobile or tablet devices will be out of luck! You'll also need to be running a fairly new Chrome dev channel build)

Yay! I got a textured cube on screen! Surely I'm the first person ever to do this!

Okay, yeah... it's not all that impressive. The key here is the potential that it provides. Compressed textures have been an integral part of 3D games and many other 3D applications on the desktop, console, and mobile platforms that they've become something of an invisible, pervasive optimization that everyone tends to take for granted. Up until now, however, they've been something that's been left out of WebGL (not without reason, they're tricky to get right). The fact that we're gaining the ability to use them now is, in my view, something of a benchmark of the maturity of the standard.

So what exactly do we mean by compressed texture? If you're not already familiar with the concept from a prior life as a game developer it can be a bit confusing to really grok what we're referring to and why it matters. After all, JPEGs and PNGs are compressed images, right? What's different here?

(Note: If you already know the answers to the above questions just skip to the "Implementation" section)

Theory

A good place to start it to look at what WebGL does with a normal texture. Let's say we've got a texture that's 1024x1024 pixels. The textures from iOS RAGE are a good example:

This texture, as it's used in my demo, is saved as a JPEG image and is 187k. That's not too bad, all things considered! For download times, certainly, it's great! But what about when we create a texture out of it? What happens then?

Your graphics card doesn't know how to read the JPEG file format, and even if it did we wouldn't want to make it do so. Decompressing a format like JPEG is slow compared to reading, say, a BMP and doesn't account very well for random access of texel data. (A texel is just a pixel of texture data.) So when we call gl.texImage2d with an image, what it actually does in the background is completely decompress the image and send the decompressed version to the graphics card. This is more readily apparent in OpenGL as used in a native language like C because it actually forces you to do the decompression yourself before providing it any texture data. WebGL is very, very kind to us developers in that regard.

The decompressed image data is basically just an array of RGB or RGBA values (think BMP without the header). Each color channel takes 1 byte, so each texel of an RGB texture is 24 bits, and each texel of an RGBA texture is 32 bits. This means that it's really easy to figure out how much RAM a given texture will take up on your graphics card. Given the texture from above:

1024 (height) * 1024 (width) * 3 (bytes per pixel) == 3,145,728 bytes == 3MB!

So that nice, compact little 187k texture is more than 16 times bigger when it reaches your GPU! Ouch! This has two practical side effects.

Texture upload speeds can suffer because we're sending a lot of data (this was the topic of my presentation)
Video memory can fill up pretty fast. Especially on mobile devices that won't have a lot of wiggle room to begin with.

At this point, you'd normally have to start looking at your game resources and say "Well, sorry artists. We just don't have the room to put all of these texture in the scene. We've got to make them all a quarter of the size." And your artists will now hate you for downscaling their lovingly crafted pixels.

Unless...

Cue texture compression!

Texture compression is, at it's core, algorithms built into your graphics card that let them decompress specific texture formats on the fly. This means that the texture actually stays compressed in your video memory and is only decompressed when your shader does a texture lookup. That may sound slow, but it's not. You can only use very specific formats for the compressed textures, and those are formats that are designed for speed rather than super high compression rates. For example, let's look at one of the more popular formats: DXT5 (The one used in the demo from above.)

DXT5 offers a fixed 4:1 compression ratio. This means that for every 1 pixel of an uncompressed texture, DXT5 can store 4. Applying that to our above 1024x1024 texture, this would translate to 768k (1/4th of the uncompressed version). That also happens to be the same amount of space that a 512x512 uncompressed texture would take. Seems like a pretty clear win, right? Unfortunately it's not without it's complications.

The compression for most formats is lossy (though in different ways than JPEG), but typically the visual artifacts of the compression are offset by the fact that you can fit four times as much texture data into the same space! Ask yourself this: Would you rather have a 512x512 texture that's completely accurate or a 1024x1024 texture that's got some minor artifacts? I know which one I would choose! Of course, the nice thing is that the developer gets to choose which route they want to go: Do you need four times as much detail in your scene, or do you want your scene to fit in a fourth the memory?

Also on the list of downsides, obviously 768k is much larger than the 182k JPEG version, so (counterintuitively) compressed textures will probably not help you out much when it comes to download times, which is unfortunate. Just one more thing to consider in the course of your development.

A final hitch in the whole compressed texture thing is that not every device supports every compressed texture format. For example: Most desktop systems will support S3 compression (that's your DXT1-5 textures), but your iOS devices won't. They support PVR compression. The two methods are similar in usage, but have different properties in terms of compression, artifacts, and performance. Those aren't the only formats either. There's a lot of them out there! It's hard to argue that one is "better" than the other (though there's plenty of people who try, usually the hardware manufacturers), and on the developers end you simply have to use whatever your platform can understand.

That does make things difficult for WebGL, however. Since pretty much any device has a browser nowadays, and many of those will probably have WebGL in the future, it means that a WebGL dev that wants to use compressed textures will have to keep three or four variants of their textures around, query the device for which one's it supports, and then download the one that meets the platforms needs. Sounds fun, right?

That being said, in many cases the upsides will still outweigh the downs, and it's probably worth the pain on our end as developers to deliver a richer experience to the users.

So, now that we know all about what a compressed texture is, how do we use them?

Implementation

First, a big fat disclaimer: These APIs are all very new and very experimental. While I don't expect the overall concepts to change to much, the exact details may shift around a bit as the feature settles down. I will do my best to keep this post updated in the future with the correct API calls, but don't kill me if they're out of date from time to time. Also, I'm pretty much only looking at Webkit browsers here. I have no idea what the other browsers are going to try and do in this space, but I would imagine that it shouldn't be too terribly different than the methods described here.

Compressed textures are exposed to WebGL as an extension. To anyone that's familiar with extensions in desktop OpenGL, that may be a cringe inducing statement, but WebGL actually makes working with extensions fairly painless!

var ct = gl.getExtension("WEBKIT_WEBGL_compressed_textures");

If the extension is supported, ct will be an object that contains all the functions and enumerations for the extension. If the extension is not supported, you'll get back NULL. Not too bad, right? From this point on, anything that needs to be done for compressed textures will happen on the ct object rather than the gl object.

Unfortunately the contents of the ct object aren't documented anywhere that I know yet, with the exception of looking at the code used to implement it. [EDIT: Brendan pointed out in the comments that the spec for this extension is available] I got most of my information from this Webkit bug. Lucky for us, there's only a few functions that we need to look at.

First is the enumerations for the texture types. This list could conceivably grow, but as implemented right now the following symbols are available:

COMPRESSED_RGB_S3TC_DXT1_EXT
COMPRESSED_RGBA_S3TC_DXT1_EXT
COMPRESSED_RGBA_S3TC_DXT3_EXT
COMPRESSED_RGBA_S3TC_DXT5_EXT
ETC1_RGB8_OES
COMPRESSED_RGB_PVRTC_4BPPV1_IMG
COMPRESSED_RGB_PVRTC_2BPPV1_IMG
COMPRESSED_RGBA_PVRTC_4BPPV1_IMG
COMPRESSED_RGBA_PVRTC_2BPPV1_IMG

Each of these represents a compressed texture type that may be supported. It should be pretty apparent from the names which is which. Just because the symbol is defined, though, doesn't mean your system supports it. To figure that out we have to call:

var formats = gl.getParameter(gl.COMPRESSED_TEXTURE_FORMATS);

This returns a list of enumerated values for the formats the current device supports. To test and see if a particular format is supported, you'll have to loop over it doing something like this:

var i, dxt5Supported = false;
for(i in formats) {
if(formats[i] == ct.COMPRESSED_RGBA_S3TC_DXT5_EXT) {
dxt5Supported = true;
}
}

(Note: I previously had an "indexOf" there. That won't work, because the formats list is a Int32Array)

Once we know which formats are supported, we can create a texture using that format with the following calls:

ct.compressedTexImage2D(target, level, internalFormat, width, height, border, data);

ct.compressedTexSubImage2D(target, level, xOffset, yOffset, width, height, internalFormat, data);

This works very similarly to gl.texImage2D, with the following exceptions:

The internalFormat must be one of the compressed texture format enums
You must give it a width, height, and border. There's no variant of the function that will do it automatically.
You always give it a typedArray of the raw compressed data. No passing in image elements here.

Otherwise you treat compressed textures just as you would any other texture.

var texture = gl.createTexture();
gl.bindTexture(gl.TEXTURE_2D, texture);

ct.compressedTexImage2D(gl.TEXTURE_2D, 0, ct.COMPRESSED_RGBA_S3TC_DXT5_EXT, 512, 512, 0, textureData);

gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.LINEAR);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR_MIPMAP_NEAREST);
gl.generateMipmap(gl.TEXTURE_2D);

Okay, great. So far so good. But for those of you that are used to simply giving WebGL an image tag and letting it go, I'm sure you're wondering where that textureData comes from. And thus we hit on one of the primary sticky points of compressed textures.

You see, most of the time a DXT5 image will come in the form of a DDS file, one of Microsofts formats. But not always. Sometimes it may be wrapped in a custom container, like Valve's VTF format. In either case, the browser manufactures have to be very careful about what they implement, because the file formats and compression schemes they use may be patent encumbered. Chrome, Firefox, and Opera all work very hard to steer clear of patents so that they can deliver a great browser to you free of charge without shouldering a financial burden themselves. If they were to start handling some of these patented formats automatically they would open themselves up to all sorts of nasty licensing issues and possibly lawsuits. Nobody wants that, but at the same time they do want to provide a way for you to use these formats without being detrimental to themselves.

So a compromise is struck: WebGL doesn't attempt to decipher compressed textures at all. It simply asks you to give it an array of texture data and tell it how big it is. WebGL then hands that information off to the graphics driver for you and says "Here! You handle it!" without looking at the data that it was given. That way while the drivers and texture creation tools need to have the appropriate licenses (and they already do), WebGL happily acts as a dumb pipe that just shuffles the data around semi-blindly, and thus stays out of the crosshairs of our crazy legal system.

What this means for you as a programmer, however, is that you're shouldering some of the burden of parsing those files. This is actually how all textures work in desktop OpenGL, so it's not a big deal if that's already your background, but it feels like a big inconvenience here since WebGL has been pretty good about sweeping the ugly bits of image handling under the rug for us.

In my demo, I've implemented a simple class to read the header of a DDS file and figure out the width, height, and data buffer. If you wanted to use other file formats like PVR, you'd have to write a parser for them too. It's not terribly difficult, the format itself is well documented, but it's certainly something that begs for a good library to hide away the details. Hm....

So that's it for now! As I said earlier, I expect things in this space to change in the near future, and it's hard to recommend that anybody start using this commercially just yet, but I'm sure that a great many graphics devs (like me!) are eagerly awaiting the dust to settle here so we can all start using this great tech in our demos, games, and apps!

Building the Game: Part 5 - Static Level Geometry

2011-11-27T14:55:00.001-07:00

See the code for this post, or all posts in this series.
See the live demo.

(WARNING! The live demo for this post will probably take a while to load and will most likely not run at 60 FPS! Optimizations will come in later posts.)

Sorry about the long gap in posts, but this has proven to be one of the more challenging things I've attempted so far. That's primarily been because I wasn't too familiar with how Unity handled things behind the scenes until I started working on this particular post. I've found some surprising (and occasionally disappointing) things about how Unity handles it's levels, and it's forced me to rethink a couple of aspects of this project, but I think I've got a decent handle on it now and at this point the order of the day is progress in small increments.

To that end, we're going to start talking about exporting, displaying, and interacting with levels from Unity, but we're going to do so one step at a time. Today's step is simply going to be getting the level geometry and lighting information exported and rendering brute force. We're not going to be worrying about collision detection, visibility culling, or anything else right now than just getting those triangles out of Unity and into our browser!

Most of the level formats that I have worked with in the past have had contained most of the geometry needed to display them in a large internal mesh that's been broken up by visibility (the traditional "BSP Tree"), and then occasionally will have instances of common meshes (like crates or lights, etc) that need to be placed throughout the level. (The Source engine uses these and refers to them as "props") Typically the tools that you use to build these levels start by having you block out the large geometry with "brushes", convex polygons that typically make up the floor and walls of your level. Then you can go and place high detail meshes, usually imported from an external tool, to give the level some more detail and texture.

As such, I was surprised to learn that in Unity everything in the level is treated as, essentially, a "detail prop". What I mean by this is that even large scale geometry such as the floor or walls is defines by placing instances of a mesh throughout the level. Those instances may simply be of a texture cube that you created in-editor, or of a mesh that was imported from Blender or Maya, but Unity treats them all the same.

This approach has some upsides and some downsides, which I'm not going to dissect here, but it does give us one big advantage in that it's easier to get the basic rendering in place because everything is going to be rendered the same way! All we really need to do is put together a format that lets us define the positions, size, and rotations of mesh instances, and then we'll render them using the instancing technique described in the last BtG. We'll be sticking to JSON for this part of our format, and it's going to be dead simple for right now:

{

    "levelVersion": 1,

    "name": "Sample Level",

    "props": [ 

{

            "model": "root/model/Barrel_Large",

            "instances": [

{

                    "pos": [28, 3, -30],

                    "rot": [-1.2, -0.38, -0.05, 0.9],

                    "scale": 1

},

{

                    "pos": [7, 2.14, -62],

                    "rot": [0, 0, 0, 1],

                    "scale": 1.2

},

]

},

{

            "model": "root/model/Barrel_Small",

            "instances": [ ... ]

}

]

}

As you can see, all that our "level" really contains is paths to mesh files and then information about the transform of each mesh instance. Utilizing our previous post's instancing code, it's super easy to load the appropriate data into memory, as we show in level.js:

for (var i in this.props) {
var prop = this.props[i];
var url = prop.model;
prop.model = new model.Model();
prop.model.load(gl, url);

for(var j in prop.instances) {
instance = prop.instances[j];
instance.modelInstance = prop.model.createInstance();

// Set up the instance transform
instance.modelInstance.matrix = mat4.fromRotationTranslation(instance.rot, instance.pos);
mat4.scale(instance.modelInstance.matrix, [instance.scale, instance.scale, instance.scale]);
}
}

And drawing those instances is equally simple:

for (var i in this.props) {

    var prop = this.props[i];

    prop.model.drawInstances(gl, viewMat, projectionMat);

}

Assuming that we have some way of exporting to this new format, we've can now render an entire level! Check it out!

This, of course, is just a small segment of a level, (in this case, the "AngryBots" sample level included with Unity). And it's really cool to look at that and say "Hey! That actually looks like something!" We've made a jump from random floating crates to a full blown environment in just a few lines of code! (Mostly, anyway. We're ignoring the export for the moment.)

A key element is missing from our scene, however, and that's lighting. There are several different options for lighting. We could, for example, go with nothing but dynamic lights, a technique famously employed in Doom 3 (which is open source now! Sweet!) But the most common technique, and what we'll be using here, is static lighting via lightmaps. Lightmapping has been in use since the original Quake, and is alive and kicking in modern marvels like Modern Warfare 3, and it should serve use well in our game too. And, fortunately for us, Unity will build the lightmaps we want!

Unity's lightmaps work a bit differently than most others that I've seen, but I kinda like the way they're set up so for the moment I'm going to be sticking to how they do it for my own code too. Typically anything that gets lighmapped in a level will have a unique set of lightmap coordinates, even if that mesh is instanced several times throughout the level. This can mean you end up storing the same mesh with different lighting UVs over and over again. In Unity, and subsequently in our renderer, each mesh gets a single set of lightmaps UVs, as if it were just another normal texture. Each instance of the mesh within the level contains a lightmap index, a UV offset, and a UV scale. The meshes lightmap UVs are transformed in a shader by that offset and scale to line up with the rect on the lightmap that has been reserved for this mesh instance. The lighting information for the mesh is laid out the same way for each instance, just shifted around the texture.

The lightmaps color information is also a bit different, and it took me a moment to figure out how to display it correctly. The lightmap itself looks like this, but it will be hard too see much because there's a lot of alpha:

If you try to simply multiply your texture colors by the lightmap RGB values like you might expect to be able to you'll just get an over-bright mess. What you actually need to do is modulate the RGB values by the alpha value multiplied by a brightness factor (I've found that 9 seems to match the Unity rendering reasonably closely.) I'm not 100% sure why they do this, aside from maybe some higher precision on brighter lights. It does give you some nice effects where the lighting can be really strong and starts to wash out the diffuse colors, though. Kind of a "psuedo-HDR". The shader code is pretty simple:

void main(void) {
vec4 color = texture2D(diffuse, vTexCoord);
vec4 lightValue = texture2D(lightmap, vLightCoord);
float brightness = 9.0;
gl_FragColor = vec4(color.rgb * lightValue.rgb * (lightValue.a * brightness), 1.0);
}

We also have to add the approriate lighting info to our level format. This means an array of lightmap paths at the beginning of the file

"lightmaps": ["root/texture/level1/light0.png","root/texture/level1/light1.png"]

And the lightmap index, offset, and scale for each instance

{

    "pos": [0, 0, 0],

    "rot": [0, 0, 0, 1],

    "scale": 1,

    "lightmap": {

         "id": 1,

         "scale": [0.0126953, 0.0126953],

         "offset": [0.566791, 0.587325]

}

}

We'll also tweak our instance rendering to give us a way to render lightmapped instances (see drawLightmappedInstances in model.js). The result? Not too shabby looking!

What a difference a few textures can make, huh? Now we have a nicely lit scene that we can fly around in and build the rest of our level information (collision, item positions, etc) on top of. The entire scene weighs in at about 32 Mb, which isn't spectacular but is far better than the 200Mb I was seeing for the Source Engine demo.

There are a couple of other implementation details that may be of interest here. I've implemented a simple Texture Manager (in texture.js) to do simple checks to see if a requested texture has already been loaded and if so return a reference to the existing one. This helps improve performance quite a bit, since this particular level tends to share textures between a lot of different meshes.

I've also gone and fixed some issues with the flying camera (camera.js), since this is the first demo to really use it. I had made a few really stupid mistakes in earlier versions that caused the camera to lock up if you were looking straight up or down. (doh!) That's fixed here, though you'll want to avoid the flying camera from earlier posts code.

Of course, I've avoided talking about the export process for this the whole time and, um... I'm mostly going to keep avoiding it. Fact is, it's kinda ugly and I'm still working out wether or not I want to keep doing things the way I'm doing them now. If you really want to see all the gory details feel free to dive into WebGLExport.cs and look at the logic in ExportLevel. There are a few quirks that deserve special mention at this point, though:

I'm basically just looping through every mesh in the scene and exporting any that are visible and static. Anything that has a dynamic component to it is simply being skipped right now. This does leave a couple of gaps if you export a complicated scene like the one above, but handling dynamic scene components is a subject for another day.

Also, I'm kinda cheating on mesh scale at the moment. Unity stores instance scales as a 3 dimensional vector, and I'm boiling it down to a scalar value. Most of the time this will work fine, since scaling unevenly leads to squashed looking meshes, but if anyone is doing that intentionally in a Unity scene we'll loose it during the export. I'm doing it this was to keep my transform matrices orthogonal, which has various nice properties in terms of inverting and so on, but I may change my mind on this restriction later if a compelling reason presents itself.

Another thing worth mentioning about the export is a leak that I've run into and am really not sure how to fix. If you try to export a large scene with a lot of textures (like AngryBots) the export will die part way through and spit "Too many files open" out onto the console, at which point Unity basically needs to be restarted. Preventing texture exports will allow the entire level to be output without issue. Obviously I've got a file handle leak, but I can't see where, and nobody on Unity's Stack Overflow imposter seems to know (or care) either. Interestingly, it seems to perform better on OSX Lion but I can still get it to crash after a few exports. I'd appreciate anyone that cares to give any suggestions in this area.

Oh, and finally I should mention that Unity apparently hates the way that the rest of the world orients their X axis, so they've flipped it. Yeah, I was happy about that too. I'm still deciding wether or not I want to fix this, but the immediate consequence is that everything we export is mirrored for now. The easiest thing to do if this gets to be a problem would be to invert X on our projection matrix, but I'm gonna just leave it be for now.

So, where does this leave us? Well, we can render all the static, lightmapped geometry in a level just fine, but we are rendering ALL of it. There's no visibility culling yet. I was able to get away with that in Quake 3, but it probably won't fly here. (Note that on my MacBook I can render the whole AngryBots level at 60fps, but without much wiggle room. We need better performance if we want to start adding game logic.)

Beyond simple visibility culling, however, I'd like to try to optimize how some of the geometry is handled. There are some meshes that appear in the levels that don't really need to be "instanced", since they only appear once and won't be applicable to other levels. (Many of the floors and walls will meet this criteria.) It would be more efficient to pack this geometry into a level specific buffer and do some additional state sorting on it. That will be a nice future optimization at some point.

We also don't have any collision information in our export, and that will obviously be critical moving forward, as will be adding things like triggers for doors and items like health packs or weapons. And we're still sidestepping the whole material issue by rendering everything with a single shader. So... yeah. Long ways to go yet. This is a decent step in the right direction, though!

I'm hoping that the next post won't take as long as this one did, but sadly with Christmas just around the bend, a new project starting up at my work, and a WebGL Camp to prepare for I think that it's best to brace for a bit of a delay. Sorry in advance!

Thoughts on the iPhone after switching from Android

2011-11-13T12:59:00.001-07:00

In the last few months, for various different reasons, I've had been looking to upgrade my beloved Droid X. There was a lot back and forth on my part about which phone to get, as I considered Android heavyweights like the Bionic and Galaxy Nexus, but at the end of the day Android was having a hard time really impressing me with it's latest and greatest. I've been pretty vocal in the past about my love of Android phones, and at one point even wrote a blog post about why the thought of me getting an iPhone was pretty absurd. And yet... today I've got an iPhone in my pocket, and I couldn't be happier with the decision!

I don't want to spend a lot of time talking about why I made the switch (I've already ranted on Google+ about that.) But it did want to mention a few pros and cons about the switch, to help out anyone else who's wondering which way they want to go.

iPhone Pros:

UI Fluidity: I've got the latest and greatest iPhone now (4S), but even going back and playing with a 3GS there's simply no questions that Apple's UI is snappier than Android in just about every way. Android has gotten better, but even the much anticipated Ice Cream Sandwich doesn't quite match the iPhone's effortless interactions and transitions. (And yes, I've played with an ICS device.)
Better Browser: On a professional level, as a web developer, this was a very compelling reason to switch. Thing is, I use Chrome exclusively on the desktop so I have a lot of faith that Google can make a great browser. They just haven't on the phone. Once again, ICS makes strides towards this, but it's still not at the "Chrome on your Phone" point that we really want. Apple, on the other hand, has delivered "Safari on your phone" quite nicely. It's a very fast, robust browser that's pretty close to desktop level in terms of capabilities. If you're developing for the mobile web, Safari is what you target before putting hacks in place to support everyone else.
More Stable: It was rare that I could go for more than a few weeks without having to battery pull my Droid X. Even when I didn't have to reboot though I had to manually kill off rouge apps more that I wanted to. On iOS I've never had to kill a misbehaving app (although my Wife often has issues with Facebook, but I'm happy to blame that one on a shoddy app.) and I've rebooted exactly once, to install a new OS update. Oh, and speaking of which...
OS Updates, when they happen: On Android, even with the best of manufacturers, you were looking at many months between the time Google said "Here's Android 2.x! Have fun!" and the time that you could actually get it on your phone (in skinned, unstable, bloatware infested form). With iOS, I was able to upgrade to the latests version the day it came out. That's pretty spectacular, from a developers point of view.
Keep your grubby hands off my phone! I have exactly 0 Verizon apps on my phone, which is the same number that came pre-installed on it. I don't have a Verizon logo anywhere on my phone. I don't have some special, unremovable skin, and I don't have random social networking bloatware that can't be turned off. Obviously this is possible, but apparently Apple is the only manufacturer that's willing to make an argument for the consumer when it comes to carrier installed crud. I appreciate that more than I can possibly express.
Accessories: Everyone and their dog makes iPhone accessories. Motorola makes Droid accessories. Guess which one yields the better selection? (Oh, and I love being able to buy headphones that actually include a working mic and inline volume!)
The Screen: This one is a bit love and hate, but first with the love. With every Android phone on the planet (even the almighty Galaxy Nexus!) shrugging their shoulders and says "PenTile is good enough" Apple's retina display really shines. People, this is what a phone screen should look like! No dithered colors, no jagged edges, no funky refresh rates. Just beautiful, high DPI goodness.
The Camera: I don't even want to try comparing my Droid X's camera to this one. It's cruel.
The Apps: This, right here, is the big daddy of switching reasons. Look, I love Android. I really do. But let's face it guys: the Apps kinda suck. For one, Apple's store has a larger selection of apps, and far more platform exclusives (especially if you care about games!) But even for those apps that do have an Android counterpart, the Android version is usually the neglected stepchild compared to the iOS app (Pandora, this still means you!) or you may not be able to run the app at all, due to hardware/software incompatibilities (Netflix, anyone?) And then there's the fact that even assuming that you get a great app that works well it's probably already been out on iOS for months before they bothered to port it, and will likely always be behind in getting updates. The fact is: Apple's App selection and quality makes Android's Market feel kinda laughable. Sorry, but it's true.

iPhone Cons:

The information is in there... somewhere: To be honest, I was never big into slathering Droid X with widgets. I did make use of most any widget, however, that notified me when something within the app was newly available. Google Reader and TweetDeck were my favorites in this regard. This had the very nice, practical effect that I could pull out my phone, glance at it, and stuff it back in my pocket knowing there was nothing worth looking at. With iOS, EVERYTHING requires you to go digging to see what's new. The most absurd instance of this is Reeder (which is all other respects is a great Google Reader replacement). It gives you the option to but an unread badge on the app icon (yay!) but then only checks for new items when you open the app (whatisthisidonteven...) This is also true of the notification bar, which I loved on Android but almost never use on iOS because, well, there's no way of knowing that it has anything in it till you pull it down. And, of course, there's no notification light. I always thought the Windows Phone 7 commercials were clever. Now I understand exactly what they're taunting.
4G: I knew that I would be giving up 4G going with the latest iPhone and it wasn't a big deal to me. It's still a bit of a disappointment, though, especially since I don't feel the tradeoff bought me much in terms of battery life (I get about the same battery life on my iPhone as I did on my Droid X)
The Screen: I could gush all day about the quality of the retina display, but then I hold my iPhone next to my Droid X and sigh wistfully. It's not a huge problem, and the high DPI certainly helps, but 3.5" is really just a bit too cramped. I don't expect (or even want, really) a massive 4.6" beast or whatever is all the rage these days, but is 4" too much to ask?
Mail: I was quite put out to realize that in it's infinite wisdom Apple has crippled it's Gmail interface. No push notifications. No contacts. You have to go through the clunky Exchange service just to get it working halfway decent, and then there's still weirdness such as "Delete" actually "Archiving". On top of that, I really hate how Apple handle's conversation threads. Google has just recently come out with an actual Gmail app for iOS, which gives me some hope, but it's still a pretty young app and is lacking some critical features, such as multiple profile support, which means I can't use it. sigh.
Maps: There are many things that the iPhone does better than Android. Maps is not one of them. Android's built in maps offering completely blows away anything the iPhone offers, either built in or on the apps store. Frankly, it's embarrassing to see Apple touting things like "alternate route selection" in their new OS when Google's had it forever. And that route selection doesn't help me one bit because I have to stare at the screen while I'm driving just to use this sad excuse of "navigation." Voice navigation was one of the first things I missed when making the switch, because Google does it so very well. iOS does have a free MapQuest app that will do voice navigation passably, but otherwise it's a pretty horrid map application, and it makes me sad to have to have two separate map apps on my phone.
Stupid app restrictions: I really don't care to use iBooks when I have a reasonably large pre-existing Kindle library, and it pisses me off that I have to go digging around in the browser to buy new books for it. On the same token, I loved Amazon's CloudPlayer on Android and it makes me sad to know that I'll never be able to use it on this device. These are things I was fully aware of going in, and not enough to sway my decision, but that doesn't stop it from being annoying.
Syncing: I made the rather "stupid" mistake of plugging my phone into my work laptop to pull a couple of songs off right after I got it. Little did I know that this would form a bond between machines that was eternal and unyielding, to be broken only by the death and rebirth of my phone, like a Phoenix, through a process that shall henceforth be known as "nuking it from orbit."
All jokes aside, it's patently absurd that you can only sync your phone with one machine. As long as I log in to iTunes with my Apple ID, can you give me one good reason why I shouldn't be able to use that instance of iTunes to manage my phone content? No. You cannot. It's funny, I've heard a lot of people complaining that Android devices show up as just a dumb USB drive when plugged into your PC. In my mind that's vastly preferable to this insanity.

Now that's a decent list of complaints, but honestly when you look at the whole package iOS comes out on top by a pretty wide margin. You have to be Richard Stallman-style fanatical about avoiding Apple's walled garden to actively let it deter you from the otherwise incredibly solid phone that they've created. I'm still going to be keeping a close eye on Android, and I think that eventually it CAN win out over Apple, but in the meantime I've been liking my iPhone very much, and have yet to regret the switch.

Building the Game: Part 4 - Static Model Instancing

2011-11-02T23:43:00.002-06:00

See the code for this post, or all posts in this series.
See the live demo.

Today's post is going to be far less involved than the last one, but it's an important subject that we need to nail down the basics of before we move on too much further.

Thus far we've been doing a decent job of showing one thing on the screen at a time, which is great if the game you're building is "Crate in empty space" or "Look at this thing!", but that's not the game we want to build! We want to build games like "Holy crap! That's a lot of stuff on screen!" and "Look at all these things!"

...or something like that.

Point being, many times in a game we're going to need to display multiple instances of the same model. Good examples would be: Health packs scattered around the level, a forest where we simply duplicate and repeat 4 or 5 different tree models, a long row of computer terminals, or even just players sharing the same skin.

There are a few naive ways of going about this. The worst would be to load up a new instance of the model for each instance we want on screen, like so:

var crate1 = new Model("root/model/crate");
var crate2 = new Model("root/model/crate");
var crate3 = new Model("root/model/crate");

crate1.draw();
crate2.draw();
crate3.draw();

The horror!!! This is going to eat your game alive, because you're pumping your GPU memory full of duplicate data and you'll end up page swapping in no time flat. (Page Swapping == Bad Things for your Framerate)

What we need to think about with something like this is what is actually going to be different about each instance of the model and store only that. Now, in our case, we'll start by saying that the only difference between models is going to be the position, rotation, and scale. In other words, we need to store a transformation matrix per-instance. Further down the road we may want to add things like alternate skins or lightmaps to that, but for now let's just stick to the transform.

So, by tweaking our model to allow it to take in a transformation matrix, we can now do something like so:

var crate = new Model("root/model/crate");
var instances = [matrix1, matrix2, matrix3];

crate.draw(instances[0]);
crate.draw(instances[1]);
crate.draw(instances[2]);

And this will perform so much better than the first code snippet that it's not even funny! And, really, you could probably build a game like this if you wanted to. But.... well, we're still repeating ourselves a little too much. Let's look at the (pseudocode) internals of draw:

Model.prototype.draw = function(matrix) {
bindShader();
bindBuffers();
bindUniforms();
bindTransform(matrix);
bindVertexAttributes();

for(mesh in this.meshes) {
for(submesh in mesh.submeshes) {
drawTriangles(submesh.start, submesh.count);
}
}
};

The important part to realize here is that WebGL is a state machine, and so all of those bind... commands will continue to take effect until they're overridden. This means that in our previous draw code, although it looks really nice and clean what's really happening is this:

bindStuff();
draw();

bindStuff(); // Unnecessary!
draw();

bindStuff(); // Unnecessary!
draw();

It's certainly not helping us to do all of that binding over and over again if we only need to change one little part of the state (the transform matrix.)

(As a quick aside: there's a good possibility that the graphics driver will recognize that you are binding the same stuff again and simply ignore your call rather than spend the time to redo it. That's awesome, but unreliable. Maybe they do that on the desktop, but what about mobile? Or what if the manufacture decides that that optimization is causing a problem and removes it one day? Point being, we don't want to rely on the driver being smart for our performance when we can improve our own code instead.)

What we'd really rather do is something like this:

bindStuff();

bindTranform(matrix);
draw();

bindTransform(matrix);
draw();

bindTransform(matrix);
draw();

By only changing the stuff that actually changes from draw to draw we make our rendering as efficient as possible. Now we just need to implement it in a way that makes it easy to use. My version looks like this:

var crate = new Model("root/model/crate");

var instance1 = crate.createInstance();
var instance2 = crate.createInstance();
var instance3 = crate.createInstance();

crate.drawInstances();

Since the only way to get an instance is through the original model, that lets us keep a list of the instances internally. That makes drawing all the instances easy. You can see the instancing management and rendering code in model.js.

There's a few other features that I've slipped into the instancing code to make life easier. For one, in anything but the most trivial of circumstances you probably won't want to draw every instance every frame. To this end each instance has an updateVisibility function that takes in an integer. That may sound a little odd, but it's utilizing an old trick that I picked up from the original Quake code. Here's how it works: We keep track of an number that increments with every frame (or every change of view, if you want to be more conservative). Then we can do whatever algorithm we have in place (BSP, Oct-Tree, etc) to determine which instances are visible and flag them with that same frame number. Finally, when we go to render we only draw those instances that share the same "visibility number" as the current frame. What this gains us is a system where we don't have to hit every mesh every frame to explicitly flag it as "not visible". Instead, we just update the meshes that are visible and the rest become inherently invisible. It's a small thing, but one that's simple enough to implement and can cut down on the amount of looping we do per frame.

The other thing implemented on the instances is an individual draw method, which essentially just draws the Model with that instances matrix. This draw does not attempt to optimize the draw at all, it just binds the state and draws the mesh once, but it can be used for debugging or specific effects.

To demonstrate the instancing system, I've put together a simple demo that takes four different meshes and creates 250 instances of each with random translations and rotations for a total of 1000 models being rendered per frame. (In my tests I could get it to render more than 4000 before I started to see slowdown, but I went lower to account for slower systems.) The results looks like an explosion in a barrel factory.

This code for todays post is relatively simple, and will need to be expanded as we work out exactly how we're going to be using instances in the game proper. Also, it doesn't account for instancing skinned models just yet as I have some more complex plans for that. Finally, although we're referring to this technique as "Instancing" it's important to note that this is NOT hardware instancing. Unfortunately WebGL doesn't support hardware instancing, so this is the best we can do for now. If we ever do gain that ability, however, it would be relatively simple to update this code to support it without changing the high level code usage.

Still not completely certain what the next post is going to cover. I've got a couple of different things I want to talk about and it really depends on which piece of code starts working first. Hope to see you there no matter what the subject matter ends up being!

Building the Game: Part 3 - Skinning & Animation

2011-10-30T10:19:00.000-06:00

See the code for this post, or all posts in this series.
See the live demo.

In the previous BtG we got the basics of a model format in place, but it only accounts for static meshes. Now, static geometry is very important and will make up the majority of any scene in our eventual game. But we all know that the most interesting things you see on screen are the ones that are moving, wether it be the player sprinting across the screen or the rocket careening towards your head.

There are many different techniques for creating motion in a game. It can be sliding a static mesh back and forth for a door, running a rigid body through a physics simulation, generating particle effects for smoke and sparks, transforming texture coordinates to fake flowing water, or deforming a mesh to look like waving cloth. We'll end up talking about some of those methods as we work our way through this series, but today I want to talk about the big daddy of animation: Skeletal Animation and Mesh Skinning!

I'm not going to cover the basics of those topics here, because this post is going to end up being long enough as-is. But if you're unfamiliar with them and would like to know more I'd recommend checking out the Wikipedia Article on the subject for a high level overview. Beyond that, Googling for "Mesh Skinning" and "Skeletal Animation" will turn up lots of good resources. For my part, I'm more interested in talking about how these systems will be implemented in my codebase.

So, first off, we know that we're going to need a list of bones with some information about their bind pose in here somewhere. This is information that could, conceivably, be stored in the binary file but in this case I've found it to be more convenient to store it in the JSON, if for no other reason than easy debugging. The data sits along side the "meshes" block, and looks like so:

"bones": [
{
"name": "root",
"parent": -1,
"pos": [0, 0, 0],
"rot": [0, 0, 0, 1],
"skinned": false
},

{
"name": "lower-back",
"parent": 0,
"pos": [0, 1.723, -0.56],
"rot": [-0.454, 0.509, 0.551, 0.479],
"skinned": true,
"bindPoseMat": [1, 0, 0, 0, 0, 1, 0, 0, ...]
},
...
]

As you can see, each bone has a name, a parent bone ID (-1 means no parent), a position (Vector) and rotation (Quaternion), a "skinned" boolean, and if skinned is true a bind pose matrix. The bind pose matrix will be stored as an array of 16 floats, and actually represents the inverse of the bind pose skeletal transform. We need that information to allow the vertices to transform properly.

Now that "skinned" value might be a little confusing, so here's the logic behind it: For each bone in our skeleton we are going to need to recursively update the rotations and positions based on the parent bones, calculate a transformation matrix based off of that, then multiply that matrix by the bind pose matrix to get the ACTUAL matrix that the vertices will be transformed by. In normal circumstances that's a lot of work, and in Javascript that's crazy expensive! But, not every bone in your skeleton will have vertices attached to them! It's not unusual to have the character's root bone or parts of the spine, etc, not have any vertex weighted against them, but we still need them there to guide the overall animation. If skinned is false, we know that we can skip the matrix calculation on those bones, which will save us a little bit of time. Now, to be honest I'm not sure if that will make any real difference in the long run, but it has proven useful on the meshes I've tested thus far.

Another thing that's worth pointing out: The bone structure is inherently hierarchal, and as such it's extremely tempting to make it into a nested structure here. That's probably going to cause more headaches than it's worth, however, so instead we lay them all out in a flat array. What we do enforce here, however, is that every bone in the list MUST appear after it's parent bone. This allows us, when updating the bones during animation, to always do a single pass, non-recursive calculation on each bone because we always know that it's parent will have already been processed and have the correct values accumulated. Plus, iterating over a flat array is faster than navigating a tree anyway and it makes the bone indexing more straightforward, so it's a win-win all around.

On the vertex side, skinning information is simply added to the vertex buffer. We had installed a vertex format flag in the binary file in the last post, and now we can use that to indicate that the vertices have skinning data as well. They're packed in the interleaved data just like the positions and normals and whatnot in the pattern: "Weight 0, Weight 1, Weight 2, Bone 0, Bone 1, Bone 2". I originally wanted to store the bone indicies and weights as bytes, since we can only reference a small number of bones per mesh anyway, but after some experimentation I found that:

WebGL doesn't allow you to use bytes as attributes. (Aww....)
You can convince it to use shorts, but the drivers hate you for it. You get nasty speed hits.

So I bit the bullet in this case and simply allowed the bone indices and weights to be floats. (I cast the indices to ints in the shader). The waste of bytes makes me grind my teeth a little, but it's not a huge deal for the better performance. That said, I'd be very interested to know if mobile devices behaved differently in this regard.

I also made a personal call to say that I would allow three bone weights per vertex instead of four as a way of allowing reasonably complex skinning but still cut down on the amount of data we're slinging around. To be perfectly honest, I don't have enough experience with these things yet to know if that will help or hurt me in the end (maybe I could have gotten away with only two?) but it feels like a good middle ground. Nevertheless, if you find that you need more or less than this for whatever reason it won't be too difficult to change.

Implementation wise, there was a new ModelVertexFormat flag added in model.js as mentioned earlier, but the bulk of the skinning specific loading code is in a new file and new class: skinned-model.js. The SkinnedModel class inherits the Model prototype, and adds a few new bits of loading code as needed. It's worth noting, however, that rather than try and build on top of the Model's draw routine, SkinnedMesh get's it's own completely independent implementation. This may seem somewhat imprudent, as about 80% of the code is the same and the opportunity for abstraction is high, but I'm resisting the temptation for a very simple reason:

Abstractions slow you down.

For the loading, I don't care so much. Yes, we want it to be fast but a few milliseconds here and there while loading meshes aren't going to be missed. We're going to be executing our rendering code hundreds or thousands of times every frame, though. As such, do we really want to be forcing the system to jump through multiple layers of function calls and redirections with every draw if we don't have to? Not to mention we'd probably have to introduce some tests to determine if we were rendering skinned or unskinned so that we could set our state correctly, and we'd have to pass more data around to ensure the right shader uniforms got filled in and... You know what, it's not that much code! It's totally worth it to me to repeat the ~40 lines of code and tweak them a bit to make sure that each Model variant renders as quickly and directly as possible.

So that takes care of the mesh, but what about the animations?

There are a bunch of different ways that animations can be stored, but I'm making mine relatively simplistic: We'll store a list of the bones that are affected (so we can have animations that only modify the legs, for example) and a list of frames. Each frame will be a snapshot of the rotations and positions (I'm ignoring scaling for now) of the bones at that frame, wether or not they've changed. That keeps the calculations really easy. We'll have a few more informational items added to the header, and for the sake of simplicity I'm doing this one as JSON. (At least initially. We'll see if the space savings justifies moving it to binary later on.)

{
"animVersion": 1,
"name": "run_forward",
"frameRate": 30,
"duration": 633,
"frameCount": 18,

"bones": [ "player_root", "Bip001", "Bip001 Pelvis", ... ],

"keyframes": [
[
{ "pos": [ -6.148, -0.052, 0 ],
"rot": [ 0, 0, 0, 1 ] },
{ "pos": [ 2.850, 1.012, 0.062 ],
"rot": [ -0.431, 0.514, 0.567, 0.476 ] },
{ "pos": [ 0, 0, 0 ],
"rot": [ -0.499, 0.500, 0.499, 0.5 ] },
...
],
[
  { "pos": [ -6.148, -0.052, 0 ],
"rot": [ 0, 0, 0, 1 ] },
{ "pos": [ -0.008, 1.003, 0.062 ],
"rot": [ -0.44, 0.506, 0.573, 0.470 ] },
{ "pos": [ 0, 0, 0 ],
"rot": [ -0.499, 0.5, 0.499, 0.5 ] },
...
],
...
]
}

A few other points that are worth talking about with this format: The bone references are done by name, not by index. I went back and forth on this a lot, but in this end I feel like this method allows a little bit more flexibility. Otherwise you would essentially have to export the model and all it's animations at the same time to ensure that the bone indexes were correct (I'm reordering the bones in the exporter to meet my "children after parents" requirement). Also, the order that the bones appear in the "bones" list is also the order that their transformations will appear in for each frame. Keeps things simple that way.

Loading of the animation and calculation of the matrices for a given frame takes place in the animation.js file.

So now the model has it's bone data and the animation has the motion data, let's tie it all together and get this mesh moving!

When it comes to skinning meshes, there's basically two approaches: Software Skinning and GPU Skinning. Software skinning basically means that after calculating the bone matrices we multiply all the vertex positions in our application code (javascript, in this case) and push it out to the GPU each frame. You can actually see an example of this with my old Doom 3 model demo. This is a perfectly valid method... when you're not running in freaking javascript! As is, we want our javascript code to be as lightweight as possible, so we turn to GPU skinning. With GPU skinning we'll still calculate the bone matrices in javascript, so we can do complex animation blending later on, but the only thing that we have to push the the GPU for each frame is the matrix array. The mesh vertices are allowed to stay static, and they're transformed into the correct position in the shader. This is invaluable in an environment such as ours! You can see the skinning shader code at the top of skinned-model.js

There is one complexity that GPU skinning introduces however. Shaders have a limited number of uniform variables that they can use at once, and since our matrices are being passed as uniforms, we can eat that limit up quickly. Not to mention, we need to have some uniforms available for other things like lighting information, textures, etc. What this means in the end is that we can only process so many bones in a single draw call. How many? Well, that's kind of a complex thing to figure out: The answer is that it depends on how many uniforms your hardware can support and how many uniforms you need for other purposes like lights. Quite honestly, I don't have a good answer for that yet, as I think it's something that we'll have to experiment with as the game moves forward. So for the time being I've just picked a number, 50, and started working with it.

So, being limited to 50 bones per draw call now, if we have a model that uses more than that we have to break the mesh up into several sub-meshses, each of which can reference at most 50 bones. Of course, we're already one step ahead of the game here, as you may recall that sub-meshes are part of our original model format! Yay! All we need to do in this case is add a couple of new bits to the submesh to make it skinnable:

"submeshes": [
{
"indexOffset": 0,
"indexCount": 11760,
"boneOffset": 0,
"boneCount": 35
}
]

The meanings of these new elements should be easy to guess. boneOffset is the first bone in our list that this submesh uses, and boneCount is how many bones after that need to be passed to the shader. This, of course, makes the assumption that the bones each submesh needs are grouped together. That's great, but we've ALSO dictated that bone order must place parents before children. Getting those two limitations to play together nicely may prove to be a formidable problem...

... and it's one that I haven't solved yet. Actually, the exporter code that comes with this post's git branch doesn't account for submesh splitting at all, which works for now because the demo mesh has less bones than our bone limit. I'm putting the issue on hold for now in favor of getting other things done, and I'll tackle this problem when it actually becomes a problem. It's going to be a hairy one when I get there though, and I may have to reverse a couple of the decisions I'm making now to make it work. We'll see how it goes.

Anyway, theoretical potholes aside, now that we've figured out how all of the formats are supposed to work, we need to actually get some models out that implement them and to that end we've extended our Unity exporter. The previous "Export Selected Meshes" menu item has been extended to handle skinned meshes as well, and we've added a "Export Selected Animations" that will, predictably export any animations that you highlight in the Project view. In the AngryBots project the prime testing target for these is the main player model (main_player_lorez) and his various running animations (run_forward, etc.)

So, with the files exported, we need to actually get them showing up in our renderer. Loading a skinned model is easy, we simply swap out our Model class for the SkinnedModel class. (game-render.js, line 50)

this.model = new model.SkinnedModel();
this.model.load("root/model/main_player_lorez");

(It's worth mentioning that you CAN load a skinned model with the static model class, it will just ignore the bone information and render it in the bind pose)

Loading an animation is similarly simple, and for now we're going to use a very basic and manual method for actually playing the animation. (game-renderer.js, line 54)

this.anim = new animation.Animation();
this.anim.load("/root/model/run_forward", function(anim) {
// Simple hack to get the animation to play
var frameId = 0;
var frameTime = 1000 / anim.frameRate;
setInterval(function() {
if(self.model.complete) {
anim.evaluate(frameId % anim.frameCount, self.model);
frameId++;
}
}, frameTime);
});

And suddenly, we run!

Of course, at this point we still have a lot of missing pieces to put in place before this will be a game-ready animation system. For one, our method of playing an animation (setTimeout with a frame counter) is pretty lame, and won't do the job for anything but a simple demo. Secondly, we're applying the animation directly to the model's skeleton, which means that we would have to duplicate the model if we needed another instance that was playing a different animation. As mentioned earlier, we're not really handling the limits on bone count in our export, so we'll run into trouble as we try working with larger models. Also, we don't have any concept of mixing animations yet, nor are we attempting to interpolate animation frames at all. And that doesn't even take into consideration performance. This animation performs pretty well, but how fast will it run when we're trying to animate 20 different objects at the same time? These are just a few of the things that we'll have to fix as the game moves forward, and many of them will justify their own blog post.

But for now, the immediate goal has been achieved: We can export skinned models and the animations to play with them. Our skinning happens on the GPU, which will speed us up quite a bit, and we've got all the pieces in place to start fleshing it out as we go along. Not too shabby!

For the next post, prepare to see double (and then some) as we talk about mesh instancing.

Building the Game: Part 2 - Model Format

2011-10-26T08:57:00.000-06:00

See the code for this post, or all posts in this series.
See the live demo.

So, after the previous BtG post we had a bare-bones pile of boilerplate code and nothing terribly interesting to show with it. Obviously it's impractical to hand-code all of our meshes into our game (although that would make it fast!), so the next order of business should be creating a way to get meshes out of the various different modeling tools and into our renderer in a format that's efficient and flexible.

In layman's terms, we need a model format!

This is actually going to involve two parts - Designing the format and creating an exporter that writes to our new format from an existing tool (because I have absolutely no desire to write 3D modeling software!) And since we can't write an exporter for a format that doesn't exist yet, let's look at the proposed format first.

Most of the existing code out there will take one of two routes when it comes to getting meshes into their renders:

Use an existing format like OBJ, COLLADA, or an existing game format. This is traditionally the route that I've taken.
Create their own format that typically is represented as a JSON object or Javascript code. This is the route that J3D and three.js have gone.

Both routes have good points and bad, but it's hard to claim that either is completely optimal. The problem with existing formats is that they are typically not designed with the web in mind which can make them difficult and slow to parse, and can mean that you're potentially downloading a lot information that will never be applicable to your game. Wasting bandwidth is a big no-no when it comes to browser-based games, so this doesn't really suit our needs.

The JSON route is typically a far more sensible one. After all, browsers have very good JSON parses built right in, and the end result is a ready-to-use javascript object that can often be used to start rendering with only minimal modifications to the data. ASCII data might not be the most compact in the world, but we're going to be G-zipping the files as they go out to the user anyway and the parsing code will be far cleaner and faster than if we tried to do the whole thing binary. Win-win all around right?

Close, but there is one thing that we can probably improve on:

"vertices": [-0.4375,0.164062,0.765625,0.4375,0.164062,...

That's pretty ugly. Not only does having a massive array of floats in the middle of your JSON make it difficult to read, it also takes up more space than the binary equivalent and will be slower to read. (No matter how fast your JSON parser is, turning the string "0.164062" into a float is gonna be ugly) Not to mention that many existing formats store positions, normals, texcoords, and other attributes in their own separate arrays. This means that we're either taking a hit at load time to combine them into an interleaved array or taking a hit at render time to bind four or five buffers when we could have just done one. Neither of those are going to kill us in the performance department, but at the same time why do something sub-optimal if we can figure out a better way?

Well, as it turns out there's a ridiculously efficient way to handle these large arrays. Any web browser that supports WebGL now also supports the ability to return AJAX requests as an TypedArray, which is great for parsing binary data!

var vertXhr = new XMLHttpRequest();
vertXhr.open("GET", url, true);
vertXhr.responseType = "arraybuffer";
vertXhr.onload = function() {
parseBinaryArray(this.response);
};
vertXhr.send(null);

This is especially true if our data consists of large arrays of floats or integers, which happens to be exactly what we need it for! If we are provided the offsets and lengths for the vertex and index buffers in the binary file, we can pretty much just pass them straight to the GPU, like so:

var vertexArray = new Uint8Array(xhrBuffer, vertexByteOffset, vertexByteLength);
var indexArray = new Uint16Array(xhrBuffer, indexByteOffset, indexByteLength / 2);
// Indicies are shorts, so 2 bytes per index

var vertexBuffer = gl.createBuffer();
gl.bindBuffer(gl.ARRAY_BUFFER, vertexBuffer);
gl.bufferData(gl.ARRAY_BUFFER, vertexArray, gl.STATIC_DRAW);

var indexBuffer = gl.createBuffer();
gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER, indexBuffer);
gl.bufferData(gl.ELEMENT_ARRAY_BUFFER, indexArray, gl.STATIC_DRAW);

Slick, huh?

Of course, the binary data should probably have a little bit more than that in it. We'll need some simple header information and a few concessions to expansion in the future. In my case, I'm taking a page from some of the old Quake BSP files and using a simple "lump" scheme. That is, the beginning of the file contains some arbitrary amount of offset and length pairs that describe where each chunk of information is found. Each "lump" can then have it's own read logic, and it's easy to tack on new lumps to the header if you find out that you need more data. My format looks like this (forgive the pseudo-C-ish descriptions):

struct Header { // Read at byte offset 0
char[4] magic; // File identifier. In this case: "wglv"
uint32 version; // File version. 1 for now
uint32 lumpCount; // How many lumps this file contains
Lump[lumpCount] lumps;
};

struct Lump {
char[4] lumpId; // Four char id that tells us what this lump contains
uint32 lumpOffset; // Byte offset of the lump data from start of file
uint32 lumpLength; // Length of the lump in bytes
};

For now we only have two lump types:

struct VertexLump { // lumpId = "vert"
uint32 vertexFormat; // Bit flags to describe the vertex elements
uint32 vertexStride; // Stride of a single vertex in bytes.
byte[lumpLength - 8] vertexBuffer; // The raw vertex data
};

struct IndexLump { // lumpId = "indx"
uint16[lumpLength / 2] indexBuffer; // The raw index data
};

Pretty simple, but with plenty of wiggle room for expansion later.

Now that's all well and good, but models are more than triangle strips alone! We need material information for a start. If the model is animated we'll need bone data. We'll probably want things like collision hulls and physical properties too. And frankly, a lot of that would be a pain to work with in binary form. So we'll leave those bits to a JSON file! Like so:

{
"modelVersion": 1,
"name": "the_crate",
"meshes": [
{
"material": "root/materials/crate",
"defaultTexture": "root/texture/crate.png",
"submeshes": [
{
"indexOffset": 0,
"indexCount": 36
}
]
}
]
}

This is obviously far easier to read, so I won't spend as much time on it, but let's touch on a couple of things really quickly: The indexOffset and indexCount will be offsets into the index buffer of the binary file (in indices, not bytes). The binary only has one index buffer and one vertex buffer which all of the meshes and submeshes share. This means we only have to bind buffers once for the model (yay!), but also means that we are limited to 65536 verts in a single model. That's not as bad as it sounds though, after all we ARE designing a game for the web. How detailed were you planning on making your models, exactly?

Then we've got arrays of meshes and submeshes. A mesh is simply all of the triangles that share a single material. The submeshes within a mesh are a little more obscure, because they're primarily there in anticipation of a future feature: Skinning. We're going to want to use GPU skinning down the line and the GPU can only accept so many bones at a time. Submeshes will describe the triangles that share a group of bones, so we can render them in the biggest batches possible. (We could also theoretically use the submeshes for tri-stripping, but I'll leave that as an exercise for the reader.)

This structure allows us to be very efficient about rendering, changing state only when we must. The basic pattern for drawing a model becomes:

Bind the vertex and index buffers for the model

Loop through the meshes

Bind the material for the mesh

Loop through the submeshes

If we're skinned, bind the bones for the submesh
Draw the submesh

Next, the material and defaultTexture. We're actually going to completely ignore materials for the time being, because that's a big complex topic that's best reserved for another blog post down the line. We know we'll need them at some point though, so stubbing in a simple path will do nicely. We do want to be able to show something sensible on screen, though, so we'll provide a default texture to use for the time being. This will usually just be a path the the diffuse texture for our mesh, or in the case of some more complicated effects might be a simple colored texture with the word "Water/Fire/Whatever" printed on it. It can be used in a level editor when we don't want to be rendering everything full detail or it can serve as a fallback if the real material is missing or corrupt. The point is it gives us something to display.

Having the default texture there also enforces a principles that I like to encourage: It allows people to elegantly grow their support for the format. If you're a third part that wants to use my format in your own project, you don't HAVE to support my material system from day one. In fact, you don't have to support it at all. But you can implement the defaultTexture and still have something reasonable show up. The same goes for the lump system in the binary file: if you see a lump you don't understand you can usually just skip it! As long as you support the vertex and index lumps the rest can be built up over time. This is a theme that you will see crop up again and again as this series goes on: Don't force a monolithic implementation, and always give developers "checkpoints" that they can hit as they're building up support.

In any case, we now have the two pieces of our model format laid out and ready to go! We'll give them extensions of .wglvert for the binary and .wglmodel for the JSON, and we'll make it a part of the format definition that these two must always share the same name with the exception of the extension. That way we can start the load for both of them simultaneously and let the browser get them to us that much faster.

You can see my code for parsing the format I just described in the model.js file.

Alright, so now we have a nice format all ready to go and exactly no content that uses said format. That simply won't do! We need a way to take models from popular modeling packages and write them out in our own custom format. We need... an exporter!

The first question to be asked is: what tools do we want to support? And the best answer would be "All of them!" After all, all of the good modelers have well documented systems for scripts and plugins. If our new format catches on we should theoretically be able to write an exporter for any format that you can think of. But to do so at this stage would be a daunting task and for our needs right now we only really need one good one to work.

In this regard Blender is a very tempting choice, in that it's free, fairly popular in the open source community, and has a very nice Python scripting system. But after a bit of fiddling around with a few different options, and after seeing Bartek Drozdz give an excellent presentation on J3D at onGameStart, I was inspired to give Unity a try as my initial exporter target.

Unity has a lot going for it. The basic version is free, but it's got the support of a commercial product behind it. The community is reasonably large and fairly active, and the engine has proven itself everywhere from the desktop to the iPhone. The scripting systems are well documented and robust. It's also got a very nice level editing system, something that will be very beneficial to us later on. Plus, it has methods for getting pretty much every major model format into it, so it's an effective bridge for us. Oh, and it comes with some very nice sample resources, which is great for us as we start working on our exporter!

This doesn't mean that we'll support every feature that Unity does, of course, nor does it mean that we're going to do anything that ties us to Unity specifically. It's just a good tool to use as a starting point.

This post is already long enough, so I'll avoid going over the exporter script in too much detail. You can see the full code in WebGLExporter.cs and WebGLModel.st. I do want to hit on a couple of points, though.

First off, to give credit where it's due: The C# script is based on the OBJ Exporter example included with Unity. The technique for using ANTLR templates to build the JSON files was borrowed from Bartek's own J3D Unity exporter. (He's really done a great job with J3D, by the way! You should go check it out!)

The other think to note is that the exporter does (or will do) quite a bit of caching, mapping, condensing, and transforming as it preps the data to be written out to the file. This is because the data layout Unity exposes is not what we want to use in our renderer, so we have a choice of doing the transformation once on export or every single time we load the file. The choice there should be pretty obvious! No matter how ugly our exporter gets, if it creates files that can be loaded efficiently it's totally worth it!

To use the exporter with, for example, the sample AngryBots Unity project you'll need to do the following:

Open up the "Unity/Assest/Editor" folder in the code base
Copy all of the files, including the dll's
Paste them into the "AngryBots/Assets/Editor" folder (on my Mac this is at "MacHD/Users/Shared/Unity")
Unity should automatically load the exporter the next time you switch to it, which will give you a menu item of "WebGL"
Select an mesh or meshes from the scene, then go to the "WebGL" menu and click "Export selected Meshes"
A message should appear in the status bar reading "Exported n meshes"
Your exported .wglvert and .wglmodel files, as well as the exported textures, should now be under the "AngryBots/WebGLExport" folder (We'll make that customizable later)

You should be able to copy the folder structure that the export produces directly into the "public/root" folder of our WebGL project.

Once you've got the exported resources in your root folder, adding them to our renderer is a fairly simple matter. In game-renderer.js, in the constructor we add:

this.model = new model.Model("root/model/model_name"); // No extension!

and in the drawFrame function we add:

this.model.draw(gl, viewMat, projectionMat);

And suddenly... Spinning Vat! Soooo much better than Spinning Crate! :)

The draw will automatically handle things like waiting till the mesh is loaded to draw, so we don't need to worry about synchronization much just yet. Now, this method isn't the best for drawing multiple instances of the mesh, or for drawing scenes where multiple meshes share the same materials, but we'll cross those bridges when we come to them. For now the important bit is that we can verify the export is working correctly.

So that's it for this post! Coming up next, we'll look at mesh skinning as we work our way up to an animated character.

I should mention that while these first three posts have come out in pretty quick succession, that's only because I wanted to start getting to the useful bits fairly quickly. After the next post I would expect things to slow down a bit. Posts will come as I hit various feature milestones.

Building the Game: Part 1 - The Setup

2011-10-25T09:04:00.000-06:00

If you haven't already, read Building the Game: Part 0 - The Foreword to find out what this is all about.

See the code for this post, or all posts in this series
See the live demo.

So, you want to make a game, huh?

Most of the time when you hit that point the best thing you can do is pick up an existing set of tools or code base and run with it. While you'll probably never find a pre-built toolkit that will fit your vision exactly (and if you DO, why are you building your game? It's already been made!) there is a lot to be gained by picking up Unity or UDK or even something a little less recent like the Quake 3 Source and hitting the ground running. You'll inherit a tested, robust tools pipeline to handle things like level building and model importing. You'll benefit from the experience of veteran developers in the code base. You'll have a large community of people playing with the same tools that you can turn to to ask questions. In short, you'll skip over all the boring, tedious parts of painstakingly tweaking the technology to the point that it's game-ready and focus instead on doing something that makes your game unique.

Essentially: You're going to be very hard pressed to come up with a good reason for NOT building on someone else's work. Save yourself a lot of heartache and try an existing solution first, because you'd have to be insane to want to start your own game from scratch...

In other words, you'd have to be me!

Yes, I'm one of the crazy ones, but I want to make it clear that I have absolutely no delusions about the path I choose being an easy one. I'm going this route purely because the technology that I want to focus on (WebGL) doesn't have that wealth of existing code, tools, and community. There are some good frameworks out there (three.js, J3D) but nothing that's really purpose-built for a game. I really want to make something that's not just a generic "framework" or "engine" but actually a real, playable game that can be studied, built on, played with, and improved over time. So let's see just how far down that particular rabbit-hole we can go, shall we?

Starting with a blank page in a text editor is always the hardest part of any project, because there's a lot of boilerplate code that needs to be put in place before we can start putting things on screen. So that's what today's post is aimed at: giving us that starting point.

(Coincidentally, the code from today's post also serves as something of a replacement to my previous "WebGL Sandbox")

First things first, we need a simple index page with a canvas that we can latch a WebGL context onto. It's also certainly helpful while developing to have a FPS counter, so I'll add one of those. Also, to take advantage of a newer bit of HTML5 goodness, we'll throw on a "fullscreen" button, because we all like to play our games fullscreen, right?

Of course, we need some code to wire this all together too. First off, since this is going to be a fairly large project when we're done, we'll want to use an "include" system that lets us link our javascript together in a way that's nicer than the standard "source tag on the index page" method we all know and ~~love~~ despise. To this end, I'll be using RequireJS. I won't say that the syntax is my favorite, but it does a good job at what it does so we'll run with it.

Next, we need a place to handle our rendering logic. To this end we'll build out a simple GameRenderer class that gives us a place to manage our render resources and state. Moving forward this will be distinct from the core game logic like player input, physics, collision detection, etc. We only want this class to handle the actual drawing of a frame. To demonstrate that things work and get something on the screen it's got the logic for a simple cube rendering in it.

(Side note: I gave the cube a crate texture, which means that my game has crates in it months before actual gameplay! I think I may have just received the worst StC score ever!)

We're also going to be spending quite a lot of time in the first few posts of this series working out various different formats for our 3D models and environments. As such, we'll want a simple way to control our view of said environment which will come in the form of our Camera Classes. There are two of these:

ModelCamera - Orbits around a given center point at a fixed distance. Useful for viewing individual models (hence the name). This is the camera type we'll use the most initially.
FlyingCamera - Using a floating WASD + MouseLook movement scheme to allows us to navigate larger scenes. Basically the same as what you see in my Quake 2 demo.

Both camera types include all the required event hooks internally for their movement, and only require that camera.update() be called at each frame to make sure the motion is properly timed. Note that these classes will almost certainly outlive their usefulness once we get into the meat of the game, but you've gotta start somewhere, right?

Finally, we need some sort of entry point into our code (a "main" function, if you will) that will tie the varying pieces together for us. This comes in the form of our index.js. It is run once the page is loaded and contains the logic to initialize the WebGL context (and show an error to the user if that fails), handle the aforementioned fullscreen button and the associated resizing, and starts up the game loop, which currently just calls renderer.drawFrame() and updates the FPS counter. (Using RequestAnimationFrame, of course!)

Of course, we have a few utility functions hiding away in the background to help out with some of the more common and boring bits, and as always we're using glMatrix for our mathematical needs, but that's pretty much it! The result:

A spinning crate! Impressive, no?

Okay, it's not much, but it gives us a place to start. And now that we've got some of the boilerplate out of the way, we can start working on the interesting bits.

Up next: Creating a model format! (I swear the next part will be far more interesting!)

Building the Game: Part 0 - The Foreword

2011-10-24T10:53:00.000-06:00

See the code for all posts in this series

While at onGameStart this September I had a chance to talk to a lot of great developers, many of which were either building or marketing their own web games. A lot of them knew about the demos that I was doing, and I got a bunch of compliments, but I also got one question over and over again that I had honestly never really thought much about:

"Where are you going with all this?"

I was truly surprised at how many times the question came up, and I realized very quickly how silly it was that I couldn't really point to any sort of end goal to it all beyond "Make a cool proof of concept."

The truth is, I've been doing the WebGL demos primarily because, well, they're a hell of a lot of fun! Also, I've enjoyed the opportunity to contribute to the dev community in some small way through the code I've made available. And those are great reasons, to be sure, but given the demos that I've already done and the time that I've sunk into it I could be half way to a working, playable WebGL game by now, not just some static scenes. So... I figured I would give it a go!

I've got some reasonably modest goals for the game itself, but ones that will push the capabilities of a Web-based game. Obviously it will be WebGL based (I mean, it would be a bit of a letdown if I decided to do a 2D game in flash, right?) and right now I'm thinking it will be a very simple, multiplayer only, 3rd-person shooter. This automatically dictates a couple of the development focuses going forward:

As a multiplayer game, I don't have to worry about AI at all. (At least not at first)
Being multiplayer also means that I need to develop a decent server and set up the basics of realtime communication. (Hello Websockets!)
Third person games tend to play well with both mouse and joystick based controls, so while I am dependent on the browser manufacturers to get one or the other implemented it's probably a safe bet that one of them will be available by the time I reach that point in development.
We'll need to have systems and formats in place for Models, Maps, Materials, Animations, etc.
We'll need to establish a tools pipeline for getting content into the appropriate formats.
And since this content in being delivered through your standard web connection, it would also be great to have some asset management on the client side.

None of these are what you would label a simple problem, but as long as we keep the scope nice and tight and don't try for anything crazy it's all within reach. Being just one programmer, "not trying anything crazy" means making use of a lot of freely available tools, libraries, and art, with a healthy dose of the ever dreaded programmer art thrown in for good measure. What I'm getting at is that this will probably not be a stunning looking game, but I'll be working to ensure that the engine is at least capable of supporting nicer art should an actual artist want to use the code that I create.

Which brings me to a very important point: I'm still very interested in giving back to the community, so everything I do for this project will be open source. The code will be done with an eye towards encouraging others to take it an run with it, either as a whole or in parts. I'm not interested in making a "framework", the code will designed specifically for the game I'm building. But that doesn't mean that it can't be built to be "mod-friendly". I would love for other devs to be able to build on the base game that comes out of this.

I'll also be writing blog posts and putting up demos as I hit various milestones. The posts won't be tutorials, nor will they be simple progress updates but probably something in-between. I'm more interested in talking about the high level decisions that go into the process and how my experience with the previous demo's that I've done influences the choices I make for my own game.

Now, I'm not a professional game developer. I can't promise that everything I do will be a "best practice" or completely optimal. I'll probably re-invent a few wheels along the way, and it's very likely that from time to time I'll have to admit that one choice or another didn't work out so well and code will be scrapped and redone. It's basically a grand experiment on my part, but that's what makes it fun.

The first couple of posts worth of code are already done, and I'll be putting them up over the next few weeks. Wish me luck, and let's hope that this doesn't fall flat on it's face.

Next post in the series: The Setup.

glMatrix 1.0 Released

2011-10-07T00:45:00.000-06:00

glMatrix has just received a relatively few minor updates, but in order to make my life easier I've decided to relocate the repository to GitHub so that I could manage all my projects in one place. I've decided to mark the occasion by making glMatrix officially Version 1.0!

https://github.com/toji/gl-matrix

There is certainly still room for improvement here, but hopefully GitHub will make it easier to work with the community on features and bugs than Google Code did.

One thing to note, since it's a fairly significant behavioral change: I was notified that quat4.toMat3/4 was producing transposed matricies in older version of glMatrix. I have fixed that in this version but if you were relying on that behavior in older versions you may have to update your code to compensate.

Source Engine Levels in WebGL: Video + Source

2011-10-05T23:08:00.000-06:00

Took the time to get the source and video demo for the Source Engine demo online tonight. Have a look!

GitHub Source

And of course don't forget to check out the Tech Talk post if you want to know more of the nitty gritty details!

I stressed for a little while about fixing some of the remaining issues before posting anything (like getting water rendering) but in the end I decided that the demo serves it's purpose in that it shows that we can push a lot of geometry for a complex scene and still run at the speeds that realtime games require. Beyond that, however, I won't be able to make use of the format for any other projects and at this point I want to start working on something that is actually playable, not just looks pretty. As such, spending any further time on the format isn't practical.

I have started on my next WebGL project, and hopefully I'll be at a point that I can start posting about it soon. See you then!

The state of the Javascript Fullscreen API

2011-09-29T00:07:00.003-06:00

I gave the javascript Fullscreen API a shot tonight, something I've been meaning to do for a while. To test, I went and added a simple fullscreen button to my Quake 3 demo. I've posted the current results online (Webkit only for the moment, sorry!) so that you can play with it if you want, but don't go expecting too much just yet.

The good: It does indeed switch your browser to fullscreen mode and isolate the given element.

The bad: Pretty much nothing else works yet.

[UPDATE! nornagon has addressed pretty much all of these issues in the comments! I've also updated the Q3 demo to reflect his advice. See below.]

So, first off I noticed immediately that once in fullscreen your keyboard basically stops responding. You can click and drag your viewport around with the mouse like normal, and interestingly enough you can jump (space), but WASD stop responding, so you can't move. According to the spec, there should be two different fullscreen calls: "requestFullScreen" and "requestFullScreenWithKeys". One would imagine that the "withKeys" variant would give us our movement keys back, but in this case Chrome complains that the function doesn't exist. Hrm... This won't be much use to me if I can't control my games!

Secondly, once I've gone fullscreen the canvas size stays the same as when it was windowed. I would ideally like to have my WebGL canvas take up the entire screen when in fullscreen. Okay, that shouldn't be an issue! An "onFullScreenChanged" event fires when the screen goes to fullscreen and back, so I can utilize that to set my canvas size to that of the document when in fullscreen mode and put it back when in windowed mode. But this leaves us with a couple of problems:

The event fires both when entering and leaving fullscreen, but doesn't tell us which. Furthermore, I'm not entirely sure how to detect which mode I'm in at all! There are CSS Pseudo classes that should be associated with those states, but how would I query it from code?
Even if I could detect which was which, apparently this event fires before the document is actually resized, so querying document.width/height when going fullscreen with actually give you the size of the window just before the transition, which is less than useful. Certainly I could add another listener to the document when I get the FullScreenChanged event to see when it's size changes and respond accordingly (if I can tell that I'm fullscreen, see point 1), but that feels clunky. Why would I ever want to know what size the document was on an event like this?

So, anyway, I'd love to do something like this:

canvas.onwebkitfullscreenchange = function(event) {
    if(canHazFullscreenz) { // How do I know which?
        canvas.width = document.width; // When does this update?
        canvas.height = document.height;
    } else {
        canvas.width = 854;
        canvas.height = 480;
    }
    gl.viewport(0, 0, canvas.width, canvas.height);
    mat4.perspective(45.0, canvas.width/canvas.height, 1.0, 4096.0, projectionMat);
};

But I don't think there's enough info exposed to actually do it yet.

Ah well, I'm sure this feature will work itself out eventually. I know that there's some discussion about improving it already, so hopefully it sees some love soon. It's certainly not game-ready right now, though.

(Testing done with Chrome 16.0.894.0 canary on OSX Lion. Your mileage may vary.)

Follow Up:

Okay, so within hours of my posting this nornagon (aka: Jeremy Apthorp from Google) corrected pretty much everything from my post in the comments. But I figure that if I had a tough time figuring it out then others will too, so here's the working code.

To go fullscreen with keyboard support in Webkit, you use:

canvas.webkitRequestFullScreen(Element.ALLOW_KEYBOARD_INPUT);

This goes fullscreen, but doesn't actually resize your canvas (as I mentioned before). At this point, there are a couple of things that you can do about that. The simplest is to simply add a stylesheet:

canvas:-webkit-full-screen {
    width: 100%;
    height: 100%;
}

Which will make the canvas fill the screen, but will not resize it. This is roughly equivalent to choosing to play a game at 800x600 on a 1024x768 monitor, in that it won't look as good but you still get the fullscreen experience and it will perform better as well. This may or may not be what you are looking for, and it probably depends on the scenario. If you want the canvas to match the screen resolution, though. there's a tiny bit more work to be done.

// Handle fullscreen transition
canvas.onwebkitfullscreenchange = function() {
    if(document.webkitIsFullScreen) {
        canvas.width = screen.width;
        canvas.height = screen.height;
    } else {
        canvas.width = 854; // These are your original windowed size
        canvas.height = 480;
    }

    // Need to update the WebGL viewport.
    gl.viewport(0, 0, canvas.width, canvas.height);
    mat4.perspective(45.0, canvas.width/canvas.height, 1.0, 4096.0, projectionMat);
};

This is the code I posted earlier, but functional this time around. Note that what I said about the document size reporting wrong in this function is still true, so you can't rely on document.width/height in this case! Here I'm using screen.width and screen.height, which report the full dimensions of your monitor. That may not be the best numbers to use, but it seems to be working for me right now.

And with that we now have fullscreen that functions like you would expect! Yay! Still don't have Firefox support in the demo, because I can't find a version on the Mac that actually supports it (am I missing something?) Otherwise, things are much better than I originally thought. They just need better documentation. :)

Source Engine Levels in WebGL: Tech Talk

2011-09-27T13:38:00.000-06:00

So I've just gotten back from onGameStart and have been very pleased with how well my "Surprise Project" was received! For anyone not at the conference or following me on Twitter, at the end of my presentation I demonstrated a Source Engine level (2Fort, from Team Fortress 2) running in WebGL at an absolutely stable 60fps! Now, to be perfectly fair there are a lot of bits of the rendering that don't work properly yet. Off the top of my head, it's still missing: Normal mapping on brush surfaces, displacement surfaces, water rendering, 3D skybox, any shaders that use cubemaps, accurate lighting on props.. you get the idea. Over the next few weeks I'm going to try and fix some of the more egregious omissions after which I'll put the code up on github for any enterprising developers, and I'll also post a Youtube walkthrough of the level, but don't expect a live version any time soon.

[Update: Video and Source Code are online now!]

There's a couple of reasons why I'm not planning on posting a live version of the demo. First and foremost, I obviously don't own the rights to the content. That's actually true of any of the demos that I've posted thus far, but whereas Quake 3 is 11 years old at this point(!) Team Fortress 2 is a very modern game that people are still actively playing. I feel a little different about distributing resources for a game that is still making it's publisher good money, and I certainly don't want to step on any toes at Valve. Secondly, however, is the practical matter that the resources for this single level take up nearly 200MB! (In comparison the Quake 3 demo requires about 12MB of binary and textures.) My web host (Dreamhost) is decent enough about giving me "unlimited" bandwidth, but I'm not entirely certain how well they would hold up to hundreds of people pulling that much data all at once.

Anyway, I've already had a couple of people request that I do one of my "Tech Talk" posts about the demo, which was my intention from the start. As always, this is more or less a brain dump of anything interesting I can think of saying about the project, so append all the standard disclaimers about technobabble here and lets get started!

So the first thing that struck me about the Source Engine BSP format is how extremely similar it is to the Quake 2 BSP. I had heard this before, but I had assumed that people meant it was similar in the same way that, say, Quake 3's format was similar to Quake 2: some core shared concepts but improved suitability for the platform. To my surprise, Source's BSP is really more accurately termed as Quake 2++. It's so similar that you could probably accurately read half the format with the same loader! I'd love to say that this was a good thing, but the reality of the matter is that Quake 2's format is very unsuitable for modern, hardware accelerated games. In fact, some readers may recall that I did a post about that exact issue a while back. Sadly, Source inherits just about every single limitation I listed there, and then adds a few more besides!

Now, I'd like to take a moment and clarify something: If it sounds like I don't like the format, well, that because it's true. But I'm not going to go so far as to call Valve out on it and suggest they do something better. The fact is, the format as it is now is almost certainly a side effect of the fact that Source was built off the original Half Life engine (or GoldSrc, if you will). GoldSrc, in turn, was built off of code licensed from id. All along the way, it's doubtful that anybody wanted to take a set of content creation tools that were proven and working and scrap them just because the file format was less-than-optimal. Joel Splosky has a really excellent post about scrapping your code and starting again being the worst possible approach to any program, and Source is a very good example of that. After all, a bad file format hasn't stopped the Valve games from being insanely popular and very profitable, has it? So, no, this isn't the format that you would build if you were starting from scratch, and I certainly wouldn't recommend it's use outside the Source engine, but there's no reason for Valve to dump it just yet.

Credit where credit is due: Most of what I did was figured out from the documentation posted at the Valve Developer Community. There were also significant bits here and there that were gleaned from the Source SDK.

As far as the BSP format itself goes, there's not a whole lot new to say about it. It's very similar to previous BSPs that I've worked with in that it contains brushes (convex hulls, defined by a list of planes) which are used for collision detection and also get broken down into the triangles that make up the bulk of the world geometry. Both brushes and triangles get attached to leaves of the Binary Space Partitioning (BSP) tree, which you can use to quickly determine where on the map any given point is, and thus narrows the elements that need to be tested for visibility and collision. The biggest difference in terms of map makeup is that the source maps rely a lot more on "props" for their overall layout. Props are models (typically static) that are placed around the level as detail geometry and in some cases can make up the bulk of the geometry in a level. As a general rule, anything that you see in a source level that isn't a flat surface like a wall or floor is probably a prop.

A side note here: Have you ever noticed how insanely detailed some of these maps are? Seriously, start up a server with nobody but yourself in it some time and just go wander around 2fort. It's absolutely stunning how many little nooks and crannies there are in the levels that you never really notice because you're far too concerned about not dying. Especially in a game like Team Fortress 2, that everyone associates with big, colorful shapes and clean outlines, it's surprising just how beautifully cluttered the world is.

There are a few new types of geometry in the source maps, neither of which I'm handling particularly well at this point. We have displacement surfaces, which can best be described as distorted planes like landscapes or rock walls, etc. These are built with a fairly complex structure of nested vertices, presumably so that we can easily reduce the level of detail on lower end machines. Then there are water surfaces, which you can probably guess at the usage for.

Worth mentioning is the fact that Source maps take Quake 2's approach to lightmapping, in that the lightmaps are actually stored per-face. This means that for performance reasons I once again broke out the code I did for Quake 2 to pack multiple lightmaps into a single texture and reused it here. It's not a huge pain, but it does introduce one more step that must be taken before we can start compiling the vertices, since we don't know where to set the lightmap UV coords until after the lightmaps have been fully parsed. Quake 3 had a much better system for this, where all the lightmaps were pre-packed and the texture coords were already correct.

Also, a special shout out needs to be given to the insanity that is the texture coordinates. Rather than store UV's directly in an easily readable format, they instead store, essentially, a texture matrix that you apply against the vertex position to derive a pixel X/Y offset on the texture that you can then divide by the texture width and height to get the appropriate UV's. At the very least, they have the decency to give you the texture dimensions in the BSP file rather than making you load the external resource like Quake 2. This is one of those things that makes a lot of sense for an editor format, and next to none in the "compiled" map. It drives me batty!

Materials are stored in a format that strongly resembles the material format from Quake 3 (Looks a little bit like JSON), but in this case there are only a few basic shaders that are supported. These are things like Lighmapped geometry, Vertex lit, water, and so on. Basically each material selects it's shader and then the rest of the material represents the variables that are fed into the shader, like texture maps, transforms, shinyness, etc. The range of variables that can be applied can be quite large in some cases, making the more common shaders (like the vertex lighting shader) very robust and flexible. (Ubershaders, so to speak.) It does end up feeling slightly more limited than the Quake 3 approach, but I'm quite certain that it's much more efficient and probably one that is worth emulating in your own engines.

There are some materials that get embedded into the BSP itself, and these usually make use of the embedded cube maps that are generated at different points throughtout the level and also stored in the BSP. I wasn't able to make use of these, however, since they're stored in a zipped format within the BSP and, well, frankly I don't want to write a gzip decompressor in Javascript. As a result any surfaces that are supposed to be really shiny and reflective in the level typically show up black or with the default checker pattern texture. Sorry!

The model format, used for player meshes and props, is somewhat better in it's suitability for modern hardware than the BSP file but is still not the greatest for web use simply because the format is actually spread across three file types.

.MDL, which stores basically everything that isn't geometry-related. This includes bone hierarchies, materials, attachment points, animations, etc.
.VVD, which stores the raw vertex data. It also contains some level of detail information which is required to lay out the vertices in the correct order.
.VTX, which stores triangles indices, sorted by body part, model, mesh (which is associated with a material), level of detail, strip groups, and finally triangle strips.

Some data elements are repeated between files, or share extremely similar structures too, which makes then all the larger to download. As I said, not the most web-friendly of formats.

The information in the VTX file is somewhat difficult to parse, as it has a lot of layers of information in it, and several indirections that you have to go through to get the raw index data. For example: the indicies don't point directly at a vertex offset, but instead give an index into a "vertex table", which itself contains the actual index. Even that number, however, is not a true index since you also have to manually calculate another offset into the vertex array based on the number of vertices in all prior meshes. (That's not documented anywhere, by the way, and was painful to figure out.) So in order to build that all important index buffer you have to do a lot of pre-processing.

The data structure also feels a little heavy for it's intended purpose. The body parts seem to have a lot to do with game logic but I'm not certain what the difference between models and meshes are. LOD is pretty clear, and Strip groups are obviously intended to allow Direct X renderers to cut down on the number of locks that you would need to do by providing vertex/index buffer offsets and lengths. (GL renderers aren't so lucky, as vertex offsets require a re-binding of the shader attribs). Strip groups are broken up based on the number of bones that they are associated with, which is smart and allows for easy GPU skinning.

Still, since I don't have any skinning implemented yet and was more concerned about performance of static props, on load I condense the entire format down into a series of meshes, one per materials, and a set of triangle patches that make up that mesh. A triangle patch is nothing more than a start index and index count. If we were concerned about a model having more than 65536 verts a vertex offset would be required as well, but the Source Engine apparently limits that, so no worried here. When a model is a prop on the map, we actually do store the vertex offset, since we combine the vertices and indices for all of the props on the map into one big buffer, which allows us to bind it once for all props on the map.

I mentioned as part of my Quake 3 tech talk a while back that I wasn't doing any geometry culling. That was perhaps applicable at the time, but certainly is not a principle that applies to this format! You simply cannot brute-force render your way through the entire map! That being said, I still did keep it as simple as possible. Each leaf in the BSP tree contains a list of props and brush geometry that is built during the map load. We also parse out the Potentially Visible Set (PVS) for each leaf, which is stored as a run-length encoded series of bitflags inside the BSP. There's one for every leaf, and the number of bits corresponds to the number of leaves. To see if a leaf of the map is potentially visible from another one, you simply check the bit that corresponds with the leaf's index. 1 means it can be seen, 0 means it can't.

To help speed up the visibility flagging, I use a couple of tricks. First, I only bother doing the visibility checks when you change leaves as you move around the level. I also don't bother (yet) with any sort of frustum culling, which means that until you move from one leaf to another the visible set is exactly the same. With props I flag the entire prop as visible or not, but with brush surfaces I do the flagging by material. If a material is visible for any of the potentially visible leaves, we draw all geometry associated with that material, no matter where it is on the map. It's still not worthwhile to break it into multiple draw calls for the sake of saving some triangles. Additionally, I don't use a boolean "visible" flag on the geometry, but instead have a "visibleFrame". Each time I change leaves, I increment a "currentFrame" variable, and set all visible geometry's "visibleFrame" equal to it. With props I also have them apply the flag to their prop type. Then during render loop I check to see if the visibleFrame is less than the currentFrame I don't render it. It's an old trick from Quake that prevents you from manually clearing the vis flags on everything. Still a very valid technique!

The core render loop for the map ends up looking like this:

Draw Skybox
Bind Brush Geometry buffers
Loop through Opaque Brush Materials

If the material is visible:

Bind material
Draw all geometry for it (one drawElements call)

Bind Prop Geometry buffers
Loop through all Prop Types with Opaque Materials

If prop type is visible:

Loop through materials

If material is opaque:

Bind Material
Loop through all instances of this prop type

If instance is visible:

Bind the appropriate light and model matrices
Draw all geometry for that material

Bind Brush Geometry buffers
Repeat Brush Geometry loop with transparent materials
Bind Prop Geometry buffers
Repeat Prop Geometry loop with transparent materials

As you can see, the whole thing is hard wired to switch state as little as possible. Also, we have to draw transparent geometry after everything else or it wont render correctly. Normally you would want to sort the transparent geometry from back to front, but I'm not doing that here. It's a known flaw and you can see it manifest in a couple of places on the demo map where, for instance, a tree will disappear when you try to look at it through a fence.

Something that's worth pointing out about the way that way I'm rendering props: It's basically begging for instanced rendering! I'm not sure if the Source engine actually uses instancing for these meshes, but it feels like a very natural fit. I was told once that one of the big reasons that WebGL doesn't have instanced rendering is because it's hard to make use of it in real world scenarios, so very few games do. I'd like to present my counter example, and ask nicely for the people making decisions to reconsider!

Alright, so I think that's enough rambling for one post. I usually end up tweaking and amending these things after I've put them up, so if I've omitted anything it won't stay omitted for long. Also, there's typically some good questions that pop up in the comments, so be sure to browse through them!

Oh, and for anyone still wondering about what the last teaser post was, it was a top-down point-cloud of the Heavy:

jsStruct: C-style struct reading in Javascript

2011-08-23T12:28:00.010-06:00

Protip: I ramble a lot before getting to the link in question. You probably just want to jump straight to the project.

So in case you haven't picked up on it yet I tend to work with a lot of binary files in Javascript. This is, to put it kindly, an absolute mess. (I would put it unkindly, but this is a family friendly blog!)

Now, to the credit of the browser makers, binary parsing has certainly gotten a lot better in a very short period of time. When I was doing my Quake 2 and Quake 3 demos, the only way to parse binary was to request you file as a raw string from the server and use String.charCodeAt() to grab the bytes one by one and reconstruct them into the appropriate data types. This meant that parsing a float looked like this:

BinaryFile.prototype.readLong = function() {

	var off = this.offset;

	var buf = this.buffer;

	var b0 = buf.charCodeAt(off) & 0xff;

	var b1 = buf.charCodeAt(off+1) & 0xff;

	var b2 = buf.charCodeAt(off+2) & 0xff;

	var b3 = buf.charCodeAt(off+3) & 0xff;

	this.offset += 4;

	var result = (b0 + (b1 << 8) + (b2 << 16) + (b3 << 24));

	return result;

};



// Code "borrowed" from Google's Numbers.java file in the GWT Quake2 port 

BinaryFile.prototype.readFloat = function() {

	var i = this.readLong(); // TODO: inline

	var exponent = (i >>> 23) & 255;

	var significand = i & 0x007fffff;

	var result;

	if (exponent == 0) {

		result = (Math.exp((-126 - 23) * bin_log2) * significand);

	} else if (exponent == 255) {

		result = significand == 0 ? +Infinity : NaN;

	} else {

		result = (Math.exp((exponent - 127 - 23) * bin_log2) * (0x00800000 | significand));

	}

	return (i & 0x80000000) == 0 ? result : -result;

};

Yikes! Not only does it look terrifying, it's also slow as dirt.

Fortunately some bright fellow out there (I wish I knew who, I'd love to shake his hand) looked at the ArrayBuffer code that was being built for WebGL's vertex arrays and said "Hey! With just a tiny bit of tweaking we can use this for arbitrary binary manipulation, not just WebGL!" And suddenly the above blob of bit shifting turned into this:

var floatValue = dataView.getFloat32(offset, true);

And there was much rejoicing!

So now we have the very nice Typed Arrays specification, and have even gone so far as to allow XHR calls to return ArrayBuffers directly, which is awesome! Binary is a first class citizen in Javascript for the first time!

Despite the massive improvements, however, there is at least one thing left to be desired: c-style struct reading. For those of you not familiar with the concept, in C/C++ you had the ability to "map" a random chunk of binary data onto a struct with nothing more than some pointer fiddling/casting. This also made reading binary structures from a file incredibly easy and insanely fast. For example, take this structure from the Quake (original) source code:

typedef struct

{

        float           mins[3], maxs[3];

        float           origin[3];

        int                     headnode[MAX_MAP_HULLS];

        int                     visleafs;

        int                     firstface, numfaces;

} dmodel_t;

If we have a large array of those in a binary lump somewhere (like Quake does), we can interpret them in a single, superfast call like so:

dmodel_t* models = (dmodel_t*)(binaryBufferPtr + modelByteOffset);

models is now a pointer to an array of dmodel_t's! Hooray! (C experts: please forgive the gross oversimplification!)

Now, let's say that you want to read this same structure into your javascript code from binary. With the latest and greatest Typed Array-powered code, that would look something like this:

var view = new DataView(arrayBuffer, modelByteOffset); 

var model = {

    mins: [

        view.getFloat32(0, true),

        view.getFloat32(4, true),

        view.getFloat32(8, true),

    ],

    maxs: [

        view.getFloat32(12, true),

        view.getFloat32(16, true),

        view.getFloat32(20, true),

    ],

    origin: [

        view.getFloat32(24, true),

        view.getFloat32(28, true),

        view.getFloat32(32, true),

    ],

    headnode: [ // I'm assuming MAX_MAP_HULLS == 4, that's probably wrong

        view.getInt32(36, true),

        view.getInt32(40, true),

        view.getInt32(44, true),

        view.getInt32(44, true),

    ],

    visleafs: view.getInt32(48, true),

    firstface: view.getInt32(52, true),

    numfaces: view.getInt32(56, true),

};

And, of course, that only reads in a single struct. You need to do that in a for loop if you want to accurately match the original code. (And make sure that the offsets are updated for each loop!)

Now, realistically that isn't too bad. It's certainly legible enough, and as long as you don't inadvertently goof up a byte offset somewhere it's not too hard to write out either. But it's a far cry from our one-line "parse" in good ol' C.

For me, after writing the 21st variation on the above code in my current experimental project, I got sick of counting bytes and decided that there must be a better way. After a bit of research online I didn't come up with anything too promising, so I decided to do what any good programmer would do and write my own! The result is jsStruct.

jsStruct allows you to declare Javascript objects in a way that mimics C declarations. For example, if we wanted to rebuild our previous example struct, it would look like this:

[EDIT: After valid ordering concerns raised by some-truth-some-guy the syntax has been tweaked]

var dmodel_t = Struct.create(

    Struct.array("mins", Struct.float32(), 3),

    Struct.array("maxs", Struct.float32(), 3),

    Struct.array("origin", Struct.float32(), 3),

    Struct.array("headnode", Struct.int32(), MAX_MAP_HULLS),

    Struct.int32("visleafs"),

    Struct.int32("firstface"),

    Struct.int32("numfaces")

);

Nice and compact! Of course, the cool part is reading, which now looks like this:

var models = dmodel_t.readStructs(arrayBuffer, modelByteOffset, modelCount);

Yay! Back to one line! models will now contain an array of modelCount dmodel_t objects, which will in turn contain all the appropriate data from your binary buffer. Easy as that!

Now for all the appropriate disclaimers: I have only tested this in Chrome/Chromium, so it may need some tweaking on other browsers. I know that Firefox doesn't yet support Typed Arrays fully, so it may be a bit before this works there. Also, there is absolutely no consideration given to older browsers here, you either support Typed Arrays or you don't use this utility. Same goes for ECMAScript 5. I also haven't added struct writing yet, so this only helps you at the moment if you want to read binary files, not create them.

It should also be pointed out that while this gets us closer to the convenience of C-style struct manipulation, it's still going to be far slower. I've tried to make the struct reading pretty efficient: A new "readStructs" function is custom when you call Struct.create(), so there will be an on-load performance hit as we create the required code dynamically but thereafter it should be about as speedy as javascript can be for this type of operation. At the end of the day, though, we still have to read the values one by one, so we'll never have a prayer of being as fast as a simple pointer assignment.

I honestly have no idea if there's anyone out there other than myself that will find this useful (not too many people are crazy enough to want to muck with binary in Javascript in the first place) but hopefully this will make life a little easier for the next guy that's as crazy as me! :)

Another Teaser Time!

2011-08-20T12:23:00.003-06:00

So, I figured I'd post another teaser image. I'm not 100% sure that this is going to work out, but it's certainly turning into a fun project! Hopefully I can have it ready for onGameStart in September, as I think it would make quite an impression!

[UPDATE: See the exciting conclusion!]

Hey, Chrome! Fix your texture uploads!

2011-08-10T10:23:00.004-06:00

I've been poking and prodding at my RAGE demo as onGameStart draws nearer, trying to clean up the code and squeeze a bit more performance out of it. During this testing I've made a depressing observation:

Chrome's performance, in reference to speed of texture upload, sucks. And by "sucks", I mean as it atrociously, painfully, unforgivingly slow.

The most technically demanding aspect of the RAGE demo is the constant texture swapping that happens every few steps. We pre-allocate an array of 30-40 some 1024x1024 textures (exact number depends on the map) and as we progress along the path we identify upcoming textures that will be needed, download them, and push them into an unused texture, hopefully well before that texture is actually needed. The Webkit and Firefox nightlies can handle this fine (though we do drop a frame or two here or there) but Chrome 14 on my Mac basically breaks down and cries when asked to do this. You can see for yourself in my simple jsperf benchmark.

So, to put this in perspective: Chrome is able to squeak out a measly 22 texture uploads per second at 1024x1024. That's ~50ms per upload that your browser is blocked on that call. If you are shooting for a 60hz game (~16ms per frame) this means that uploading one texture to graphics memory just caused you to drop 3-4 frames. One texture, 3-4 frames lost. Ouch! For a medium that will be highly dependent on streaming, that hurts!

By comparison, Safari 5 gives me 62 uploads per second (ie: you may drop a frame here and there, but performance will stay pretty solid.) and Firefox 7 blasts out a whopping 188 uploads per second! That's ~8ms per upload, leaving lots of breathing room for rendering!

It's a real shame too, because in most other ways Chrome seems to be the most solid performer with WebGL. If I chop all the textures in my RAGE demo down to 512x512 I can run at a rock solid 60hz with no tearing or stuttering. (Though the texture upload is still painfully slow compared to the other browsers.)

Maybe the benchmarks look a lot better on Windows, but I don't have a machine to test that with right now. Regardless, this is something that the Chrome team really needs to smooth out. Pretty please?

(All timing is taken from my iMac)

[UPDATE: I wrote up a bug report on the issue, lets see if it goes anywhere. Similar reports have been added in the past, as seen in the comments on this post]

WebGL Frameworks are awesome, here's why I don't use them.

2011-08-08T23:25:00.007-06:00

Tonight I was posed a very interesting question by @HunterLoftis on twitter:

What's your opinion on three.js, glge, etc? I haven't seen anything in that camp half as performant as your quake 3 fullscreen demo

I answered him with my < 140 char assessment of the situation:

I think they're great frameworks, but I question if a high-performance WebGL app can afford the overhead.

I feel like this is a subject that could use a bit more explanation, though, because it's not at all black and white.

First off, it's worth looking at why these frameworks exist at all. After all, somebody thought they were worth building, right? Overwhelmingly, the big driving force behind any of the frameworks I have seen is simplifying your setup. I mean, there's a reason that some of these people have cool demos coming out their ears and I have, what, six? It takes a lot of work to set up that damn little WebGL canvas! Initializing the canvas is pretty easy, but in order to get anything to show you need to create your vertex buffer, fill it, bind it properly, write your shader, compile it, link it, bind it, and then dispatch the draw call. And that's only if you don't care about perspective and nothing moves! If you want anything beyond a simple triangle or quad you also need to create some matricies (something that there's no native library for, BTW!) run through the required maths, bind them to the appropriate shader slots, and then render. Oh, and if you want it to be interactive at all now you're doing input handling too. And by the way, where are your vertices coming from anyway? Anything more complex than a cube is impractical to code by hand, so you need a model format that you can export to and parse...

Tired yet?

That's a lot of pain and suffering to go through if all you really want to do is test your cool new shader. So a few industrious fellows such as the wonderful Mr. Doobhave put a good deal of time and effort into creating some nice libraries that hide away the worst of that crud behind a simple interface and let you get to the bits that you're really interested in working on, wether that be a shader, an animation technique, a physics simulation, etc. And for most purposes they will work beautifully! There's a lot to be said for skipping the "boring" bits and just making your app work. In fact, 99% of the time I would fully recommend using them. If you are questioning wether or not you should be using a framework, the answer is probably "Yes!"

There is really only one exception: If performance matters.

Now, hold up one second. You're a programmer, so your first thought is immediately: "Well, yeah! Performance ALWAYS matters!" And you're right, but there's a difference between wanting good performance and really needing top notch performance. Simple example: Google uses a framework for Google Body. Does your need really outstrip theirs? Probably not.

[UPDATE: So, apparently I chose a bad example for this one. @won3d mentioned in the comments that Google Body only uses TDL for some math and texturing convenience functions. So, how about a better example: ROME uses three.js, and they aren't really a slouch in the graphics department.]

A good counter example, though, would probably be realtime games. (Which always tend to be edge cases in any environment). If Google Body hitches for a second or so, nobody really cares, but if your Diablo clone stutters a few times you may have just killed your player, and they're not going to be happy at a loss that was clearly not their fault. So yeah, now performance matters quite a bit.

In a scenario such as that, suddenly a generalized framework makes a lot less sense. In fact, I would go so far as to say that even the WebGL-centric "Game Engines" are ill suited as well. The fact is, Javascript is a slow environment by most any measure. If you're really looking for raw speed, you don't want to be doing ANYTHING that doesn't directly serve your purpose. The only person who's going to know what you need, though, is you, and as such any framework you use is going to be doing things that you don't need or want, or at the very least doing them in a less optimal way. So there is a very tangible benefit to coding the ugly bits by hand.

Do you actually want to do that, though? It's worth considering that you really need to know what you're doing to get any benefit from coding at that "low level" (a phrase which strikes me as very amusing in browser land). And, yes, it will be painful. But if you really need that performance it will be worth it! If you're on the fence, though, crack open your framework of choice and give it a try there first. I'm guessing that whatever your project is it will probably do fine there. If you can prove to yourself that you need more control over the environment, though, then by all means go crazy!

So, the final question: Why don't I use any of the frameworks available out there? I have nothing against them, I promise, nor do I feel that I'm "above" the need for them. For me it comes down to two simple things: On the one hand Some of my demos (Quake/Rage) are built specifically to eek out every last drop of performance I can get, to really show what WebGL is capable of. Performance is the whole point of said demos, so it's worth it to be bare bones. Secondly, I personally enjoy the challenge! I've been coding OpenGL for a while, and it's fun for me to see how much I can optimize this scenario or that. But if I were to go back and do, say, my Spore or Hellknight demos again? Yeah, I'd be jumping on three.js in a heartbeat.

I'll leave you with one more thing to consider, however: Web browsers are evolving crazy fast. My advice about performance will probably hold true for the first generation of WebGL games, but check again a couple years down the line and it's entirely possible that the performance situation will have improved enough to render this whole discussion moot.

Getting cozy with GitHub

2011-08-06T12:29:00.002-06:00

When I posted my code for the WebGL Rage demo, I was quite surprised at the number of comments that I got requesting that I out the code up at GitHub instead of Google Code. At the time I mentioned that I would do so, but then promptly got caught up in the chaos that accompanies relocating to a new job, and never got around to moving the code. (Sorry about that!)

A nice side effect of the delay, however, is that I've gotten quite comfortable with git in the meantime, since that's what I use at my new job! I'm still probably more of a fan Mercurial due to simplicity of the interface, but I've found myself using git even for hobby projects lately just for consistency. So, yeah, git doesn't seem nearly as scary to me now as it did when you all were first pestering me about it. :)

In any case, I've finally got the Rage code up on GitHub. And as an added bonus I've put the Quake 3 code up there too for easy access! I'll most likely put any future projects I do up on GitHub as well, just for the benefit of having them all in one place.

https://github.com/toji/webgl-ios-rage
https://github.com/toji/webgl-quake3

The somewhat depressing state of Object.create performance

2011-08-01T22:59:00.007-06:00

I have recently been introduced to the niceties of the new EMCA 5 Object model, which revolves around Object.create. The syntax is a bit wonky to those of us that have been javascripting for some time now, but once you get used to it there are some really great features at work here, not the least of which are actual properties, a much better inheritance model, tighter access control, and more! I'm not crazy about losing the ability to "new" my objects, and the funny little hacks that you need to put in place to simulate a constructor are turn-offs for me, but past that there's an awful lot to love here...

... except the performance.

I was curious about how the new model compared with the tried and true methods in terms of speed, so I whipped up a simple jsperf benchmark to gauge how different aspects of the two object methodologies performed. (And I'm not the first either) The results, frankly, were rather depressing.

On my iMac with Chrome 14 (dev channel) new Obj() is currently outperforming Object.create() by a factor of 10! Seriously! 10 times slower, and we've lost constructor functions along the way! Fortunately member access and function calls are virtually indistinguishable performance-wise once the objects are created, which is good (if expected). Sadly, however, utilizing Properties (one of the big bonuses of the new model) is painfully slow. My tests showed a Property to be 200 time slower than a good old setFoo/getFoo pair. The numbers are about the same on Safari, though Firefox showed some interesting variations. There wasn't a single platform where the new model could be called a clear performance winner though.

Of course, the feature is fairly new and hasn't undergone the rigorous optimization that some of the older methods have, so I would fully expect to see these numbers improve moving forward, but for now if you're performance conscious you'd do well to steer clear of Object.create.

(Oh, and despite drastically redesigning the Javascript object model we STILL couldn't be bothered to add operator overloading? Really?!?)

Dirty Full-Frame WebGL Performance Hack

2011-07-27T22:15:00.010-06:00

So since WebGL first started appearing in browsers it's been people's natural instinct to create a 3D canvas that fills the entire browser window. Obviously this is because we like our 3D games to run full-screen (or as close to it as we can get). But you'll notice that I usually have my demos run in a window (usually 854x480). The reason for this has traditionally been because when WebGL was still gaining steam there was a severe performance penalty that was directly related to the size of your canvas. (See this early thread for a good idea of what I'm talking about)

Of course, things have improved on the browser side, and computers are always getting faster so this problem isn't as noticeable any more, but that doesn't mean it has disappeared. Netbooks/Chromebooks/etc are still popular, and don't have a lot of muscle. WebGL-capable mobile devices and tablets probably aren't too far off either. (N900 anyone?) For these environment, it would be great to preserve that fullscreen feel (especially on mobiles!) but still maintain a reasonable framerate (hopefully ~30fps or more.)

Since the dawn of 3D games we've had the ability to render at a lower res than your monitor is capable of and still have it fill the screen. Gamers are often willing to deal with some jagged edges to get smoother gameplay (but not many want to play in a window the size of a postage stamp.) So, is this an effect that we can emulate on the web? As it turns out, yes! I was playing with just such a situation a couple of days ago and stumbled on a great little hack.

The idea is simple: Create the WebGL canvas at a lower res (say, half width and height), and use CSS3 transforms to scale it to the full browser size. The code snippet is pretty simple:

// Create a WebGL canvas at half the document size
var canvas = document.getElementById("glCanvas");
canvas.width = document.width/2;
canvas.height = document.height/2;

And apply the following CSS style to the canvas element:

#glCanvas {
/* Anchor to the upper left */
position: absolute;
top: 0;
left: 0;

/* Scale out 2X from the corner */
-webkit-transform: scale3d(2.0, 2.0, 1.0);
-webkit-transform-origin: 0 0 0;
}

And done! Everything else works just like your standard WebGL app! In my experience, there is a small performance hit for the upscale (and yes, it interpolates), but it's nowhere near the performance hit of rendering everything at twice the resolution. On one slower machine I tried the fullscreen render was running at 6 fps, the half-sized render was going at 20 fps, and the half-size upscaled was getting about 16 fps. Not bad numbers overall!

As a proof of concept, I retrofitted the technique onto my Quake3 demo, which has a new variant here:

Full Screen Quake 3 (Touch enabled)

I've also taken the time to add some basic touch controls to the demo, since this technique will probably benefit mobile devices most as they gain WebGL capabilities.

One finger drag: Look around

Two Finger drag: Move/strafe

Three Finger tap: Jump

A small caveat for this demo is that the canvas will not scale to fill the window dynamically as you resize, but that wouldn't be too hard to add. Still, it's really cool to put your browser in fullscreen mode and see corner-to-corner WebGL running at a decent speed on most any device!

So now the fun part: What's the coolest device you can get this sucker to run on?

WebGL RAGE source is up

2011-05-21T22:12:00.006-06:00

https://github.com/toji/webgl-ios-rage

(Edit: Code now resides on github, by popular demand. I won't be maintaining the Google Code version in the future.)

Okay, this is a bit of a different release for me, because I'm really not happy with the state of the code as is, but I'm putting it up anyway. Basically what it boils down to is that I promised to have the code up this week, and I don't want to break that promise. Beyond that, however, I have a new job at Motorola that I'm starting on Monday. (I haven't mentioned that on the blog yet, have I? I forget that not everyone on this site reads my tweets.) I leave for California tomorrow, and since there's no way in hell that I'm checking my iMac as luggage I'll be without a good personal development machine for something on the order of three weeks. So when it came down to a choice between releasing some ugly code now or releasing some prettier code a month from now, I figured you guys would forgive me if I just pushed what I had!

A few notes about the release:

You'll need to compile the PVRtoJPG.cpp into an executable that can be called by the tex_parse python script. (I've included an OSX build) The python script breaks an .iosTex file down into individual PVR files, and then translates those into the JPGs that you need for the walkthrough.

For the YouTube demo I was running off of a server on my local machine. This skirts around the problem of how long it takes to download the texture files. If you run this from a remote server currently you'll likely end up with the wrong texture most of the time. This could be solved by buffering image downloads much like a streaming video, where we start downloading before the initial render, time how long it takes for X number of images, then figure out how far ahead we need to have downloaded before we start moving.

The number one slowdown in the demo is unquestionably the process of actually pushing textures into video memory (gl.texImage2D calls). There's very little that can be done to mitigate that (use smaller textures, maybe?) but things could be helped if I restricted it to at most one gl.texImage2D call per frame. Right now they have the potential to come in big batches.

As I said in my tech talk notes, it took me a while to figure out exactly what the texture structures were. As a result, the code sometimes refers to them as "offests" and sometimes as "textures". Sorry for the confusion, I'll clean that up next time I get a chance.

If anyone has questions about the code I'll be happy to answer! And expect me to push out some improvements to the project once my personal life settles down a bit.

RAGE WebGL Tech Talk

2011-05-16T22:08:00.013-06:00

Wow!

I'm really blown away by the response to the video of my RAGE WebGL demo! It's been getting a lot of word of mouth on Twitter and it's been really fun to follow what people have been saying about it on sites like reddit. And most of what I've been hearing is positive too! Which is awesome... mostly...

..except I think people don't really understand what they're seeing. I'm getting a lot of credit for doing an awesome WebGL demo, and certainly I'm proud of it, but the fact is that the real work done on my part was reverse engineering the format. Once that was done the rendering was pretty trivial. And if the video looks awesome then ALL of the credit for that goes to id Software's incredible art and tools teams! The fact is, outside of some careful management of the textures this project pales in comparison to the complexity of, say, my Quake 3 demo. Of course, to a certain degree that was the point of this whole exercise...

I'll talk about the rendering in a moment, but first I want to back up and talk about why I chose to do this project and how I figured out the format in the first place, since that's the bit that I found the most interesting. If you don't care much about that aspect, skip down to the "Rendering details" section. I'll warn you now, this will end up being quite a bit longer than the Quake 3 tech post since a lot of it will be stepping through my thought process. Reader beware...

Choosing Rage

So, obviously I have an affinity for the id formats. This, combined with seeing a working virtualized texturing implementation in WebGL got me thinking about how feasible it might be to render a level from RAGE when it came out on the PC. Obviously the game not being out yet put a damper on that, and there's a lot of technical issues to be considered, but the desire has always tickled at the back of my mind.

When I was asked to speak at OnGameStart about my Quake 3 demo, I knew immediately that I wanted to build a new demo that I could show off alongside it, and I wanted it to be a format that was better suited for the web. Quake 3 makes for an impressive "look what I can do" demo, but it relies pretty heavily on the assumption that all of your resources reside on a local disk, and that the player doesn't mind waiting for the whole level to load before doing anything. It also assumes that it's faster to cull and render many small batches of geometry instead of pushing large batches to the card at once. These are not very web-friendly assumptions. And to top it off it's a first person shooter, which means that the controls are basically impossible to do nicely in Javascript.

A good web-oriented format is one that can have small chunks streamed down to the player at a time while they navigate the environment so you aren't stuck looking at the "loading screen of doom" for several minutes. Said format would also be built around the idea of changing state as little as possible, rendering geometry in large batches, and making geometry culling as trivial as possible. Ideally it would also be designed with a control scheme in mind that didn't grind against the nature of a web page (ie: require total mouse control).

It took a couple of days, but at some point it suddenly dawned on me that, based on everything I had read about it from various interviews, RAGE on the iPhone seemed to meet a lot of that criteria! Plus, it was built for a mobile device so I knew it wouldn't be too outrageous in terms of performance requirements. And, yeah, it wasn't "the" RAGE that I had been thinking about, but it was pretty close. I knew I had to at least give it a try! So I asked a friend that owned the SD version of the game if I could rip some of the resources from his copy to play around with. (I later bought a copy of the HD version) The files in question were pretty easy to find (SD_RageLevel1.iosMap/Tex) and so within a few minutes I had all of the required files to pursue my new pet project...

...and no freaking idea what the files contained.

Figuring out the Format

Well, yeah, I knew they contained a map and textures respectively, but how those were formatted was anyone's guess. Unlike many of id's past games there's no community documentation of these formats (well, NOW there is...) so I was left with nothing but some assumptions based on what I had read and seen. Amazingly, many of my assumptions proved to be correct or at least pretty close. (I got lucky on that one.)

Since performance was obviously a concern, I didn't think the files would be encrypted or intentionally obfuscated (beyond, you know, being binary)

It seemed reasonable that some elements of the Quake BSP files (like the basic lump structure) would carry over to this format

I knew that the textures were 2BPP PVR, based on interviews that Carmack had given about the game, and I assumed they would be in a easy-to-find and easy-to-parse form to assist the speed of rendering

I assumed there would be a player path stored in there somewhere, and that it would have visibility and texture information associated with it

And finally, I assumed that SOMEWHERE in that big ol' lump of 1s and 0s there would be a list XYZ coordinates

That last assumption seemed like a pretty reasonable place to start, so I got busy trying to figure out where those coordinates were.

To that end, I grabbed my WebGL sandbox and hacked up a quick and dirty form that let me specify an offset into the file, a byte stride, and a element count and started brute-force stepping through the file to see if anything looked sensible. I got an awful lot of this for a while:

The problem ended up being that I assumed those XYZ values would be 32 bit floats. After a day or so of not getting any results at all, it suddenly dawned on me "Hey! This is built for OpenGL ES, maybe they're using something more compact." My first instinct was half-precision floats, but that didn't give me much better results. Eventually I tried parsing the values as shorts, and after a little bit more browsing...

Hallelujah! Now THAT looks like a level!

So now I knew that the vertex array consisted of 5 shorts - An X, Y, Z position and what I could only assume was a texture coordinate for the last 2 values. That was great info, but I didn't know where the vertex list started or how long it was. For this, I started looking at the file in a hex editor. On a quick scroll through it was easy to see that there were several distinct sections of "patterns" in the hex and ascii data, so it should be pretty easy to pick out where one section ended and another began. I started by jumping to the offset that I had hit in my visualization tool and tracing backwards looking for a change in the general "look" of the data.

And I certainly found it, sticking out like a sore thumb. As I said earlier, I assumed that this format would be similar to the other id formats in how it indexed it's lumps, so I guessed I would find something near the top of the file that pointed me at this spot. I took the byte offset of that "border" and did a search through the file for anything that matched that value (427432), and found a match 144 bytes in. That sounds like a header to me!

Next I traced my way down through the vertex values till I found another "border" (this one was harder to spot, but still visible), and calculated the size in bytes of that lump. (4224760) I tried finding that value near the top of the file, but failed. I did, however, find 422476, which happened to be the byte size of the lump divided by 10, which I knew to be the size of each vertex. It seemed very likely that this represented the element count for that lump.

I played around with some more of the numbers near the top of the file and pretty quickly was able to match up a list of offsets, element counts, and element sizes for 5 different lumps. There was a mystery 6th chunk of the file that none of these header values seemed to reference, but I could figure that out later. My next task was to figure out what each of my newfound lumps represented.

The first I already knew to be vertices. The second I found I could guess at pretty easily. Looking at the values a very distinct trend began to emerge. Almost all of the values looked like this:

1,2,3 4,3,2

That may seem like gibberish, but most graphics developers would probably recognize that as a familiar pattern. When indexing the faces of a square, you build two triangles. These triangles need to keep the same winding (counter clockwise or clockwise) for face culling to work, so you almost always store your indicies for a square face in the above pattern. And this lump was FULL of that pattern! It didn't take too much to determine that this lump was an array of unsigned shorts that represented indices. (And that fit very well with OpenGL ES, which can't accept anything larger than an unsigned short for indices.)

So now equipped with my list of verts and my array of indices, it seemed like I should be able to start rendering triangles, right? Well... sort of.

If I only rendered the first few hundred indices as a triangle list I got what looked like very sensible geometry. After around 300 or so, however, the triangles started to criss-cross the map in weird and very obviously wrong ways. And really, this made sense. Since the indices were unsigned shorts, the maximum value they could hold was 65535, but we had 422476 vertices! We simply couldn't index them all unless we were using an offset of some sort.

So now it was time to get creative. I knew in order to reach all of the vertices my offsets would have to be stored as longs, so I started parsing values from all the remaining lumps as longs and then watching the min and max values as I went. What I was looking for was values that did not dip below 0 and did not exceed what I knew the vertex and index counts to be. Any values that fell outside of that range couldn't possibly be my offsets. As fortune would have it, I found those values in the first and third longs of each element in the third lump. The first value matched very nicely with the vertices, and the second match very nicely with the indices.

(Side note: I later discovered that the second and third values of that lump represented the index and vertex counts, so I could have actually started rendering things right here, but for some reason that never occured to me.)

On the assumption that these were my offsets, I then went looking for something else that would reference them. Using my min/max trick again, I determined that the first long in the fourth lump matched the offset element count exactly, and the following two longs were typically nice, low numbers. I took a wild guess that one of these would be a vertex count and started playing around with it. It took a few tries, but eventually I hit on the sweet spot and discovered that the second value was actually an index offset to be applied on top of the previously discovered offset, and the third value was the vertex count. And now... we had solid walls! I had found my meshes.

Now, all of that happened over the course of three afternoons worth of tinkering, and things slowed down a bit after that so I'm not going to bore you with all the details and just hit the high points. I figured out that the last lump started with 7 floats just by observing that they're values were reasonably sane in the hex editor (by sane I mean nothing that got big enough to trigger scientific notation, no NaNs, etc.) I also observed that the mins and maxes of those elements tended to be within the range of a short or between -1 and 1. It struck me a day later that these values probably represented my path and orientation through the level, and some quick renderings of the values proved my hunch correct.

I figured out that the path elements also contained offsets into the file and element counts simply be looking for values that referenced the aforementioned "missing lump", and once again using my min/max tracking discovered that the list of values they pointed to corresponded with the mesh lump elements. It was pretty obvious from there that these were meant to be the meshes visible from each point along the path.

The part where I really got hung up was the textures.

Getting the textures out of the massive .iosTEx file proved easier than I initially thought. As I mentioned earlier, I knew that the textures were 2BPP PVR based on an interview Carmack had given and blog post he had written. Using the PVRTextTool I was able to open the texture file as a "RAW" file with the appropriate compression format and visually determine that the textures were stored as 1024x1024 tiles. I saved one of these tiles off to disk to see what the file size would be. (327732 bytes every time. PVR is a fixed rate compression scheme.) I then divided the total texture file size by that number (minus the PVR header size) to get the number of textures in the single file. I then wrote a quick python script to extract out each chunk of 332KB and stick a predetermined PVR header on them and in no time flat I had a folder full of individual PVR files.

Of course, PVR files don't do you any good in a browser, so I had to convert them into JPEGs. That, surprisingly, proved to be quite the challenge since apparently nobody ever bothered to write a command line converter to get images OUT of the PVR format, only in. I eventually wrote up a quick and dirty one myself using the PVR and JPEG C libs. Long story short, I got all the images in a usable format.

I had no idea how they matched up with the meshes, though. I think the thing that threw me was that I identified pretty quickly that an array of 16 shorts in the path elements lined up nicely with the texture count. As such, I thought that those 16 textures were the only ones used at that point of the path, and I kept searching for a value that tied each mesh back to one of those 16 textures. I got stuck on that one problem for nearly a week before throwing my hands up and (as I said in the video) going to Carmack for help.

Based on his reply, I took another look at the values I had already parsed and realized, to my shock, that the number of "offset" elements in the map ALSO exactly matched the number of textures! In fact, beyond being simply a list of offsets these elements were implicitly the texture index for every mesh that used them! That discovery made, paired with the realization that the 16 textures on each point represented upcoming textures (for streaming), not ones currently in use, tied up all the loose ends for me. The final trick was (after some experimenting) determining that the texture coordinates were read in as shorts that needed to be "unpacked" into a 0.0 to 1.0 range float to line up with the textures correctly. Within a day I had my first glimpse of a fully textured map.

At that point all that remained was to do some rendering optimizations and tweak the controls a little and then it was off to YouTube for all of your viewing pleasure. Obviously that's the reader's digest version of it all, but hopefully it gives you some insight into the reverse-engineering process that went into this demo.

Rendering Details

Compared to figuring out the format the rendering process for these maps practically handles itself, which is a massive credit to the individuals that designed the format and the tools that power it. Everything is packed in a manner that lets you change state as little as possible, and effectively forces you to render only the geometry that is visible from where the player is standing.

The central mechanism to the rendering is actually the path that the player follows. Each point along the path contains a position and quaternion (for orientation), and movement along the path is done by interpolating between the positions and orientations at the desired speed.

The path points contain a list of up to 16 texture IDs that will be visible from an upcoming point (but not this one), and that should start being streamed into memory. These "upcoming textures" tend to be repeated by several points in a row, each time moving closer to the front of the list, so you are given a bit of breathing room to get everything loaded and ready. The header file has a value that indicates the maximum number of textures that may be visible at once, so you can pre-allocate that many textures (plus at least 16 for the upcoming ones) and then swap texture data in and out of that static set for the remainder of the level. I can say from experience that having these statically allocated gives a pretty significant performance boost in WebGL and reduced the amount of texture "popping" that I saw on a walkthrough.

Each point also contains a list of geometry that is visible while standing on that point (or standing between that point and the next one). This list has been pre-sorted by texture and vertex buffer offset (which are the same data structure, if you recall from the parsing notes), so binding of state is kept to a minimum.

Every mesh in the level consists of nothing more than position and texcoord data and the texture to apply. All lighting is baked directly into the textures (as opposed to the lightmaps used by previous id games), which makes a lot of sense when you consider how everything in the world is uniquely textured.

We're still getting RAGE-style virtualized textures, in that objects only use the texture space needed based on their size on screen. The difference is that rather than determine the texture levels needed dynamically like the full PC version of RAGE will do, all of the texture levels here are pre-computed based on the path you walk. The textures themselves are actually more accurately called "texture atlases", since they almost always contain textures for multiple objects at varied distances. Here's a good example, to get an idea of what they look like:

Most meshes in the level are stored multiple times, with the same geometry and different texture coordinates for each instance. This eats up more space, certainly, but saves you from needing to do any calculations at run time to determine how to line the mesh and texture align. So to render any given mesh you bind the texture, bind the vertex buffer at the appropriate offset, and then render starting at the given index. Done. No fancy shader tricks, no special effects, no weird state mangling. Just texture and draw.

This is why I give so much credit to the artists at id. I've seen a lot of comments about the video I posted claiming it was "The most impressive WebGL demo they had seen so far", which actually makes me laugh a bit because almost every other demo I've seen is graphically more complex in some way. Even my Quake 3 demo is far more interesting on a technical level than this one. What it highlights, though, is that your graphics don't NEED to be Crysis level, shader heavy monstrosities to look good. Art direction is SO much more important than the technical workings of the engine.

So that it! I feel like after all the attention I've been receiving some people are going to feel like I just pulled away the curtain and revealed, not a wizard, but a simple old man. Sorry if this explanation makes anything seem less magical somehow, but I still think it's an incredible feat on id's part.

I highly encourage anyone that has enjoyed this demo to buy and play RAGE on their iDevice, it's a fun little game and I always find it more interesting when I know what's going on below the hood. I'm happy to answer any questions you guys might have about the format (that I can, anyway) and look for my javascript source to be posted later this week!

iOS RAGE rendered with WebGL

2011-05-15T16:43:00.003-06:00

As promised, here's the new WebGL demo that I've been working on! I won't be posting a live version this time (watch the video to find out why), but I will be demoing it on stage at OnGameStart this September as part of my presentation.

I have several follow up posts to this one that I'll be making soon to post the source code for this demo, describe the file format (at least the bits that I was able to figure out) and talk about some of the rendering techniques used here and why I feel this is a great format for WebGL.

UPDATE: I've now got the file format notes that I kept while building this demo avaliable as a Google Doc.

UPDATE AGAIN!: Tech talk for this demo is up now.

Interleaved array basics

2011-05-06T22:41:00.034-06:00

I got a question from Jon in the comments on the WebGL sandbox project yesterday:

Hi Brandon, Using your demo as a starting point, I try to display a pyramid, but this far I've only been able to see one of its four faces. If I use gl.LINES instead of gl.TRIANGLES, then I see only one triangle (i.e. on face). I'm also a bit confused by the way you mix texture coordinates into the vertArray. Can you explain how these coordinates get sorted out in the shader?

Honestly I don't know if I'm the best guy in the world to be explaining this, but I'll give it a try, since there seems to be remarkably little WebGL-specific information about this. Most tutorials prefer to keep the arrays separate for simplicity, but that's not optimal for performance. The concepts all work pretty much the same as OpenGL, but it's nice to see them put to use in the environment you are using. It requires some basic understanding of subjects that don't normally apply to Javascript, like counting bytes, but it's not too hard once you get the hang of it.

The first thing to wrap your head around is that the WebGL API doesn't know anything about vertices or texture coordinates or normals or anything else like that. What it does know about is lists of numbers. It knows how to save lists of numbers in your graphics memory, and it knows how to tell the shaders to start rendering those numbers, but really that's about it. YOU are the one that tells the system what the numbers mean, via the shader code and gl.vertexAttribPointer().

var values = [
    -1.0, -1.0, 0.0, // Vertex 1
    0.0, 1.0, 0.0, // Vertex 2
    1.0, -1.0, 0.0 // Vertex 3
];

You've probably all seen that before, right? The thing to notice this time is that aside from the comments and the way that we've structured our line breaks there's nothing that separates the vertices. That line can just as accurately be written like this:

var values = [ -1.0, -1.0, 0.0, 0.0, 1.0, 0.0, 1.0, -1.0, 0.0 ];

Of course, it's a lot harder to read for us human beings now, but to the computer it's exactly the same. In either case we push the data into a vertex buffer like so:

var vertBuffer = gl.createBuffer();
gl.bindBuffer(gl.ARRAY_BUFFER, vertBuffer);
gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(values), gl.STATIC_DRAW);

Take note of the Float32Array bit, we'll be talking about it later. End result of this, though, is that vertBuffer now points to an array of numbers somewhere on the graphics card. So how does the computer know how to interpret these seemingly random values? gl.vertexAttribPointer!

Before we render, we have to call code like this:

gl.bindBuffer(gl.ARRAY_BUFFER, vertBuffer);
gl.vertexAttribPointer(shader_attrib_position, 3, gl.FLOAT, false, 12, 0);

This tells the system everything it needs to know about how to read your vertices. (Well, more accurately how to tell your shader about the vertices...) There are 4 points that we want to pay attention to, here: The "3" tells WebGL how many values make up each point. Since a position is defined by x, y, and z coordinates we tell it "3". If you were doing 2D rendering in your WebGL code you could easily pass a 2 in here, and you're allowed to give it anything from 1 to 4 depending on the situation. Next, we tell it what kind of numbers we're passing. In this case they are floating point, so we pass in gl.FLOAT. There's also gl.BYTE and gl.SHORT (and others, but let's not overcomplicate things). This tells WebGL two very important things: First, how to expose the values to the shader, and secondly how big each value is. Now that's important, and for those of you who aren't familiar with byte packing let's do some quick review:

gl.BYTE = 1 byte (obviously!), can hold values from -127 to 127
gl.SHORT = 2 bytes, value range of -32767 to 32767
gl.FLOAT = 4 bytes, value range of a whole lot.

For the most part, unless you have some VERY specific needs, you'll want to be sticking with gl.FLOAT. Using Float here also lines up with our use of Float32Array when creating the buffer. If we were to use one of the other types you would want to create the buffer with a different array type.

The important thing to take away from that is that according to our vertexAttribPointer call above we are passing in 3 Floats. 3 * 4 = 12, so each position takes up 12 bytes. And, hey! We have a 12 in that call too! That's called the "stride", and it tells the system how far apart (in bytes) the start of each vertex is. This is very useful if our vertex contains more information than just a position. Lets say that for some reason our vertices were defined like this instead:

var values = [
    -1.0, -1.0, 0.0, 999.0, // Vertex 1
    0.0, 1.0, 0.0, 999.0, // Vertex 2
    1.0, -1.0, 0.0, 999.0 // Vertex 3
];

Those 999's might represent some other part of the vertex data, but they don't have anything to do with position. If we continued using the same vertexAttribPointer values from above, our vertexes would look like this:

[-1, -1, 0] [999, 0, 1] [0, 999, 1] [-1, 0, 999]

We can easily "ignore" that 4th value, and hence go back to our desired vertex positions, like so:

gl.vertexAttribPointer(shader_attrib_position, 3, gl.FLOAT, false, 16, 0);

Notice that we are still instructing WebGL to read 3 floats, but the stride has changed to 16 to account for the 4th, unneeded float. (4 values * 4 bytes per value = 16 bytes).

Finally, we have that 0 at the end, which tells WebGL how far into the array to start reading values. This is also given in bytes, and in this case is 0 because we want to start at the first value. If we wanted to start, say, on the second number we would do this:

gl.vertexAttribPointer(shader_attrib_position, 3, gl.FLOAT, false, 16, 4);

Now our vertex positions would look like this:

[-1, 0, 999] [1, 0, 999] [-1, 0, 999]

This is very useful for packing multiple meshes into a single buffer, since we can start rendering at any point, but it's also useful for interleaving data. "Interleaved" means that you have multiple types of data packed into a single array, like position and texture coordinate data. Most tutorials will show them as separate arrays like so:

var pos = [1, 1, 1, 2, 2, 2, 3, 3, 3]; // (x, y, z), (x, y, z)...
var tex = [0, 1, 0, 1, 0, 1]; // (s, t), (s, t)...

But it's often times more efficient and easier to have them all as one array, like so:

var verts = [
    1, 1, 1, 0, 1, // (x, y, z), (s, t)...
    2, 2, 2, 0, 1,
    3, 3, 3, 0, 1,
];

var vertBuffer = gl.createBuffer();
gl.bindBuffer(gl.ARRAY_BUFFER, vertBuffer);
gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(verts), gl.STATIC_DRAW);

When we set up the vertexPointers for this array, we need to make two calls to vertexAttribPointer.

gl.bindBuffer(gl.ARRAY_BUFFER, vertBuffer);
gl.vertexAttribPointer(shader_attrib_position, 3, gl.FLOAT, false, 20, 0);
gl.vertexAttribPointer(shader_attrib_texcoord, 2, gl.FLOAT, false, 20, 12);

Let's examine what that means: The first attribute (our position), is comprised of 3 floats spaced 20 bytes apart, starting at byte 0. The second attribute (our texcoord), is comprised of 2 floats spaced 20 bytes apart, starting at byte 12 (or 3 floats in). And that's it! Two attribute types in one buffer! And we can do this for as many attributes as we need (within reason). It's not uncommon to have a single buffer that contains position, texcoords, normals, binormals, and vertex weights in it!

I sincerely hope that all made sense, and wasn't just a lot of graphics gibberish. I haven't talked about the shader side of things at all here, but that's something that other tutorials cover pretty well. Good luck, and happy WebGL-ing!

Announcement: onGameStart

2011-04-30T10:32:00.000-06:00

I've been sitting on this for a little bit, but now my face is on the website so I guess there's no sense in waiting any longer. I've been asked to speak on my Quake 3 demo at onGameStart, an HTML5 Game development conference in Warsaw Poland this September!

I'm still working on the exact subject matter of my talk, but it will most likely involve discussing the optimization challenges I faced while building the Quake 3 demo and things to consider when building a map or model format for use on the web. I'm also working on another surprise project that I want to be able to show off at the conference to contrast with the Quake Demo.

I'm extremely excited by this opportunity to mingle with the HTML5 game development community and share whatever I can about working in this exciting new environment! Hope to see you there!

Teaser time

2011-04-17T18:31:00.003-06:00

So I'm taking a bit of a break from my super-secret game project to do a super secret WebGL project! :) I don't know if I'll be able to get this to a functioning point any time soon, but the following screenshot has me SUPER excited!

Hopefully I have more to show in another weeks time or so. Wish me luck!

Update: There's now something a lot more interesting than dots to look at!

WebGL Starter Package

2011-04-13T22:33:00.001-06:00

[Edit: Now with 100% less jQuery dependency!]

I've had some sudden urges to jump back into WebGL land again lately, and while gearing up for another project it struck me how much time I was wasting trying to copy one of my older projects, strip out all the project-specific stuff, and get down to a really basic starting point.

Realistically, that starting point is one of the hardest hurdles to jump for any graphically-based program, doubly so for 3D programs. There's just so very many silly little things that might go wrong!

Is the context and viewport set up properly?
Is your shader compiling?
Are your vertices right?
Are your indicies right?
Are your matrices right?
Did you give the right strides and sizes to your vertex layout?
Is your geometry rendering in front of the "camera"?
Is it rendering in a color other than the background color?
Are you certifiably insane yet?

I decided to save myself some greif and put together a quick and dirty WebGL page that I can use as a jumping-off point for future projects. The goals here were to start with something that was putting geometry on the screen and allowed me to move around the scene, nothing more. This was the result:

Toji's WebGL "Sandbox"

Like I said, very simple. Just enough geometry on screen to know that you're rendering properly and to give you a sense of space. This is certainly not meant to be the foundation of a complex demo, but it sure beats starting out with a blank page! Of course, while I built this for my own use I certainly hope that some other aspiring WebGL developer out there finds it useful, and to that end I've packaged it up in a convenient downloadable bundle:

WebGLSandbox.zip

A couple of quick notes, for those that end up using this: Most of the formats I work with are designed with Z as the "up" axis, so Y is the one that actually points "out" of the screen. I'm using requestAnimationFrame (with fallbacks to setTimeout) for the core animation loop, so hopefully that doesn't cause any issues. I'm also not setting anything like blend modes or geometry culling states, you're on your own there. The page relies on my glMatrix library (included) and also makes use of webgl-debug.js, though you can easily turn that one off.

If anyone finds this useful or uses it as the basis for their own projects I would love to hear about it! Also, if you have any suggestions on how to improve it send them my way! Happy coding!

requestAnimationFrame

2011-04-12T23:07:00.001-06:00

On the suggestion of @mrdoob today I reworked the animation loops for my Quake 3 and Doom 3 demos to use requestAnimationFrame (if it's available). This won't really produce a visible difference for most people, but it should utilize the browser event loop more efficiently. Paul Irish gives a good explanation of it at his blog.

A side effect of this that may be of interest to other developers is the simple little jQuery plugin I wrote to support this functionality in a cross platform manner that also provides a few perks to the user. You use it like so:

$('#canvas').requestAnimation(function(event) {
// Draw frame here...
});

The "event" passed into the callback function contains the following values:

timestamp: Current timestamp, equivalent to new Date().getTime()
elapsed: Milliseconds elapsed since the animation started
frameTime: Milliseconds elapsed since the callback was last called
framesPerSecond: Rough count of the number of times the callback has been called over the last second. Only updates once per second.

If you wish to stop animating, return false from your callback.

I recognize that this may not meet everyone's needs, and probably is a little buggy at the moment, but it should provide a quick and easy way to do a basic animation loop in a way that plays well with your browser. If you have any suggestions for improving it let me know!

Hellknight demo works again

2011-04-12T08:16:00.000-06:00

I've had something come up this last weekend that encouraged me to go back and clean up a couple of my Demos. Hopefully I'll have something more tangible to talk about in that regard soon, but the practical effect of it is that I got the Hellknight demo working again!

The problem was exceedingly stupid, and the only reason I hadn't solved it earlier is because frankly I had never bothered to look. When doing the update to move everything over to the new array models I apparently started pushing the mesh indicies in as a Uint8Array, then later called drawElements with UNSIGNED_SHORT (which, of course, is 16 bit). Obviously these two don't get along very well. I changed over to a Uint16Array and everything worked again! No one to blame but myself on that one. :)

Oh, and I had to un-invert the texture coordinate V component that I had previously been inverting. No idea why that happened. *shrug*

First impressions of Gingerbread for the Droid X

2011-03-28T08:06:00.000-06:00

So I posted this review on the Motorola support forums (here), but apparently even though I didn't give out any details about where to get the leak Motorola still felt that me talking about Gingerbread (even if it was mostly positive) was unacceptable and they locked down my post and removed the review. Fortunately, I saved a copy, and I'm reposting it here. Also, since this is my site now and not theirs, I have no qualms about linking to the leak pages:

Get your Gingerbread goodness over at My Droid World! Huge thanks to P3Droid and the crew for pulling this together for us!

I've spent the last half hour or so browsing through my newly Gingerbread-ed X, and I wanted to let the community here know my initial impressions. It's still early, so I probably won't cover everything but I'll update this thread as I find new things. For the sake of reference I was using Liberty 1.5 just before I updated, so I'm going from Blurless to full on Blur. (Quite the switch!)

So, the obvious one first: The new Blur. Anyone familiar with my previous posts on the forum will probably be surprised to hear that I (gasp!) actually don't mind the new Blur! Please note: "Don't mind" is a far cry from "Love it", but whereas I found the old Blur borderline offensive I see the new one as usable and even decent looking. As such I'm going to use it for the next week or so, just to give it a fair shake. I'll probably end up going back to old reliable (LauncherPro) eventually, but it speaks volumes that I didn't switch right away.

It also helps immensly that the Blur widgets don't produce a noticable slowdown simply by being present anymore. I don't know if that's a side effect of Gingerbread's optimizations or Motorola's hard work (probably a bit of both) but it's appreciated regardless. I did remove them after a few minutes, simply because they don't really mesh with how I use my phone, but it's nice that I did it out of preference rather than a desire to keep my phone from chugging. The Blur widgets look nicer now too, though they have been tweaked in much smaller ways than the rest of the home screens.

There are several other things about the home screens that deserve special mention. I had raged against Motorola in a previous post for apparently removing the "Manage Apps" option from the home screen, as that is a major Gingerbread feature. I'm happy to report that the Manage Apps button is back where it should be, much to my relief! (Maybe the Blur devs heard my impassioned cry? I can dream! ) I'm also very happy that the first three icons in the bottom dock can be set to whatever the user wishes. The third icon is the app drawer, and is non-replaceable (for obvious reasons), nor can it be moved. The placement of it (bottom right corner) is taking me a bit to get used to, but otherwise it's fine. The app drawer itself is actually one of the nicer parts of the skin, as it now supports user-defined groups (helpful for ignoring the ever-present crapware) and has a handy link to the market at the top. You're still stuck with 7 home screens, which is overkill for me (I typically use 3), but having the extra screens off to the sides doesn't hurt anything. It's worth a special mention that swiping between screens is smoother than any other homescreen that I've used on my phone to date. Kudos to the Moto Devs for that!

Of course, I have some gripes about the home screen too: In order to accomodate the "replacable dock icons" feature, the trash can that appears when dragging apps or widgets has been moved. It now shows up in place of the notification bar. I get why they did this, but the placement looks awkward and it feels weird to drag things onto it. If the bar was a bit thicker, I think it wouldn't feel so strange. Alternatively, since the app drawer icon can't be replaced, why not turn it into the trash when dragging? Another bit of weirdness was the behavior when adding apps to the homescreen. You still open the dock and long-press an app as usual, but instead of returning to the homescreen so you can place the icon it brings up a menu asking if you want to put the app on the homescreen, in a group, "share" it (WTF?), or uninstall. Choosing "home" then awkwardly plops the icon in the middle of the current homescreen where you can resume your regularly scheduled dragging. Again, I get why this was done (primarly to support the grouping feature) but it turns populating my homescreens from a two step process into a three step per app. That's not huge, but 15 apps later it gets really old. I don't know how this one could be mitigated, but it feels awkward nonetheless. I also was very annoyed to see the the unlock slider is still a clunky mess, despite all the optimizations made elsewhere. Liberty had an unlock slider that rivaled the iPhone in smoothness, so to see this one chugging along is a total letdown. You can do better, Motorola! Come on!

Next, the theming: The overall color scheme is a fairly nice muted grey/blue that I'm actually somewhat fond of. It looks nice and feels classy and clean. The notification bar has a very nice transparency effect to it, (and it has the ability to clear individual notifications, which is awesome!) and the notification icons generally looks very nice, following the same clean blue look. The exception is 3G icon, which is a big grey block who's letters light up blue when in use. It's ugly and out of place, and I hope that it's just a temporary icon that has yet to be replaced in this leak. Outside of the blue headers, though, all of the menus are a very bright grey (not white, but only a few steps off from it). It fits the theme decently enough, but I'm not a huge fan of them being so bright. I much prefer stock Gingerbread's slick black menu backgrounds, especially if I'm using the phone in a dark room (which is common for me, I like to read Kindle books on my phone at night.) They're not as ugly as I thought they would be, but still not my favorite. I'm also somewhat dissapointed in the much lauded "bouncy" effect when you reach the ends of a list that Gingerbread introduced. I've played around with the Nexus S a bit and quite liked the effect, but here it's extremely muted to the point of almost going unnoticed. You do get a very faint blue highlight effect when you hit the end of a menu but you have to be going pretty fast to see any real bounce. It's difficult to explain, but the fact that it's so subtle is a ding for someone like me who's always love Apple's elastic menu effect and longed to see an equivalent in Android.

Overall I like the theme, it's not nearly as scattershot as the old Blur, but at the end of the day I still prefer the standard Gingerbread look and feel. At least this one is tolerable, but I expect that the first chance I get to go back to a ROM that gives me the AOSP theme.

Now while I do like the theme, that goodwill does not apply to the dialer. Stock Gingerbread has hand's down the nicest, cleanest dialer theme I've seen. It's understated, simple, and beautiful. GingerBlur, on the other hand, has a dialer that is flat out ugly. It's got this wierd mix of radial gradients, glossy highlights, and funky grey borders that make it looks like the product of a Photoshop tutorial gone wrong. The area showing the dialed number also has a totally different background and look than the keypad itself, further lending to the confused and messy look. It's downright atrotious, and I cannot get rid of it fast enough. At the very least the contact list looks nice, with a slick alphabet bar running down the side so you can jump to the right name quickly.

A few last notes about the apps themselves and then I'm done for the night. The Camera and Camcorder apps have been tweaked a bit, and I still like them better than the basic Android camera, but they're still pretty slow and the pictures don't look much better so it seems to be primarily a slight re-skinning. The gallery is effectively unchanged outside of the new color theme, and it's passable but I still vastly prefer the cool 3D AOSP gallery. This one does give you HDMI out capabilities, but I don't feel it's worth the tradeoff. The music app is about on par with the standard Google one, which is to say that they're both pretty clunky. Use DoubleTwist or Winamp instead. I don't use email (I'm a Gmail guy), so I can't comment on that one. Nor have I ever given much attention to the News, Places, Media Share, or Messaging apps, so I don't know what's changed. Sorry. As mentioned earlier, I don't make use of the Blur widgets eitehr so I can't comment beyond saying that they look better and no longer bog the system down.

VZNavigator, Blockbuster, Skype, and crew are all still here and still unremovable. That's a slap in the face to Motorola's customers, but frankly we don't expect anything better from them at this point so no surprise. There's a special place in hell, though, for whomever it was that decided that they were going to have CityID on here and not let me remove the little parasite. [begin rant]This app - this single, unremovable app - is what convinced me to buy my wife an iPhone instead of an Android device. It's outright criminal that this thing pops up demanding that you pay for it after every phone call. No, I don't care that it goes away eventually. No, I don't care that I can tell it I don't want it. It's predatory and malicious, and doesn't belong on my phone or anyone elses. Take notice, Verizon, Motorola, and anyone else who cares to listen: I will never buy another phone that has CityID pre-installed, removable or not. If there are no Android phones that meet that description when it comes time to buy, I will turn to Apple. That's how much I loath this thing.[/end rant]

So, final verdict? I'm still not a fan of this rabid desire by the manufacturers to "brand" Android every chance they get. I would still much rather have a stock Google experience phone. If Motorola is going to insist on pushing Blur, though, at the very least they're getting better at it. With Gingerbread Blur has gone from "intolerable mess" to "not offensive", and aside from the awful bloatware I'm not scrambling to scrub it from my phone anymore.

How's that for high praise?

glMatrix 0.9.5 released

2011-02-27T11:17:00.000-07:00

Quick post to say that I've just posted a new version of glMatrix up on the Google Code repository. This is primarily a bug fix release, but I've also squeaked in a few new functions at the suggestions of some users including: mat3.transpose, vec3.lerp, and quat4.slerp.

I know it's been a while since I've updated the library, but frankly I haven't had much motivation to do so lately. I haven't had a chance to do much with WebGL lately (much to my dismay!) and it didn't seem like anyone else was really using the library. A few days back, though, I received word that the awesome tutorials at LearningWebGL.com have been updated to use glMatrix! Needless to say, this provides a bit more motivation for me to keep the libraries up to date!

I'll be paying a bit more attention to items on the library issue page for the next little while, so please direct any bugs you find or feature requests you have over there. Thanks!

Why I love my Android, but bought my wife an iPhone anyway

2011-02-08T22:27:00.000-07:00

Yes, I'm one of those crazy blokes that was up at 3AM (well, 1AM in my timezone) to pre-order an iPhone as soon as it came out on Verizon. But not for me, for my wife. Her old feature phone was dying and she had made certain wistful comments about how nice a smartphone would be, so when given the chance I jumped at it. It arrived yesterday, and so far she seems to be in love. Yay!

There was a funny moment when I gave it to her though, one that caught me off guard. After an initial moment of shock and excitement (it was a surprise) she turned to me and said: "But wait, why didn't you get one for you?"

I laughed in her face.

It was a totally involuntary reaction, and I felt bad for it, but in all honesty the idea of getting an iPhone for myself seemed a little absurd. I absolutely love my Droid X, and wouldn't trade it for anything Apple has to offer. At the same time, I would never buy my wife an Android phone (or at the very least not any of the ones Verizon offers at this point). I feel it's worth examining the reasons why:

I don't want to give my wife a phone crammed with bloatware. I don't want to try and explain why that stupid Verizon bookmark will never go away, or why VZ Navigator is stuck there, even though she'll never use it.
I don't want to ever be concerned about wether or not she's going to get the latest software and OS updates. I don't want to tell her that the cool new feature that they just announced may not be coming to her phone at all because the manufacturer is too lazy to update it.
I don't want her to deal with a buggy, bloated skin. I don't want to have to explain why my phone looks and acts different than her phone which looks and acts different than her parents phone, even though they're all on the same OS.
I don't want to EVER tell her that she needs to pull her battery to get her phone to respond again. I've needed to do that weekly in the past with my Droid X (before I started using custom ROMS).
I don't want her to worry about wether or not an app in the store will actually work on her phone. I don't want her to pay for something only to have it crash and burn when she tries to run it, because it was developed on Phone X and she has Phone Y.
Basically: I don't want to give her a phone that needs maintenance, by me or anyone else. It's a freaking PHONE! If she has to keep running to her geeky husband just to keep it running, it has failed in the most fundamental way possible.

Say what you want about walled gardens and draconian policy, you have to admit that Apple puts the rest of the mobile world to shame when it comes to making a smartphone that just plain works. There's a lot to be said for that, and I honestly believe that that is the core reason why they still sell like mad.

Of course, on the flipside there's MY phone, which comes with an entirely different set of qualifications:

I don't want to ask permission (much less pay) for the "privilege" of running a program that I built on my phone.
I don't want to be told that I can't run something just because I didn't get it through their "official channels." If I find a cool project online, who are you to tell me I can or can't try it?
I don't want to ship my phone off for a week because my battery died.
I don't want to pay through the nose just to get more storage.
I don't want to have to use some proprietary cable when I have several perfectly good micro USB cables lying around.
I don't want to be forced to use a particular music, email, browser, or messaging app just because the phone maker doesn't like competition.
I don't want to lose my widgets! Holy crap, how do you people live without them?
Basically: I view my phone as a small computer, and I want to treat it as such! I don't mind a bit of tweaking and fiddling in order to have more control over my device.

Of course, these two viewpoints (Absolute stability vs. absolute control) are somewhat opposing ideals, but it is certainly nice that there's enough choices out there to satisfy both parties. Granted, I think both sides could certainly be improved by trying to meet somewhere in the middle: There's no good reason why Apple can't free up their platform a little more, and there's no excuse for me ever needing to pop my battery because my OS locked up! Until we hit that point, though, it's a bit sad to say that the iPhone really is the only sane option for users that want a reliable device without dealing with a lot of crap from the carrier and manufacturer.

Android still has a long way to go in that regard.

A few other random notes before I go, after observing my wife with her phone:

No matter how much I try to delude myself into believing that my phone has a snappy UI, that illusion disappears the moment I interact with any iOS device. It's so much more responsive that it makes me want to cry. This is important, Google! Fix it!!!
It's unfortunate that many of the Android apps out there are mere shadows of their iOS counterparts. (Pandora comes to mind immediately.) Many Android apps look like amateur knockoffs in comparison, even when developed by the same company! And that's not even considering the multitude of apps that have no Android equivalent! (Netflix! I'm looking at you!)
It's completely baffling that Apple would do something as clumsy as sticking an actual temperature (73 deg) on the weather app icon and not make it update! The first big question my wife had about the phone was "Why isn't the temperature right?" It took me several minutes of googling to discover that the value shown is static. Same goes for the clock. Seriously?!?
I didn't realize how much difference two features really make in how I use my phone: The notification bar, and the app drawer. The fact that I can get notifications about anything on my phone in a spot that I can easily see (AND easily ignore) from pretty much anywhere is something I totally took for granted. Likewise, the ability to keep infrequently used apps hidden away in the app drawer while reserving my homescreens for the things I use all the time is absolutely invaluable.

Cutting the Cable (TV)

2011-01-09T17:06:00.001-07:00

My wife surprised me a little while ago by approaching ME with an idea that I had been toying with for a while but never suggested to her because I thought she'd hate it:

"Could we save money if we canceled our cable subscription and just did Netflix instead?"

Well, yes! Yes we could. With our special "we want to keep you as a customer so we'll lower your cost for 12 months" rate having just expired a decent internet connection and standard cable package was now running us about $120 a month. Ouch. The plan was then to sign up for Netflix and Hulu Plus, get a Roku player, and see how it worked out. This was a bit of a leap of faith for us, since we'd never used Netflix before, and I'd only tried Hulu online. If we stuck with it, though, we'd be keeping an additional $45 or so each month, so it was well worth it!

Technicality: Obviously we couldn't cut out the cable company entirely, since our internet connection needs to come from somewhere, and here in Utah that pretty much means Comcast. (Note to local officials: Free market only works when there are no monopolies involved and the government doesn't actively block competition.) Also, we kept the "basic" cable plan for local channels since it pretty much nulls itself out by keeping the internet "bundled" price and prevents me from buying antennas and converter boxes for all our TVs.

Though I was worried at first (mostly for my wife's sake, I already watch most of my TV on my computer anyway) I found this transition surprisingly painless and even a step up from before in many ways. I can't tell you how many times we've been browsing our Netflix recommendations and my wife has said "why would anyone need cable with all this?" Good question indeed!

I have to admit that I had no idea how much I would like Netflix. There's a TON of stuff on there, and the backlog could keep my happily watching new material till I'm 60, and I haven't even queued up a single DVD! I've especially enjoyed watching some of the material that you just don't see here in the states unless you're buying DVDs: My guilty pleasure lately has been the supremely silly BBC series Robin Hood. I had always shied away from Netflix because I never considered myself enough of a movie buff and now I'm regretting that. It's an excellent service, and it's no wonder that it's putting the brick and mortar stores out of business.

But while Netflix has been a pleasant surprise I must say that I've had the most unexpectedly disappointing experience with Hulu Plus. Going into this I thought that we'd use Hulu a lot more than Netflix, because I had been using it quite a bit on the computer to watch my weekly shows. I didn't care so much about the episode backlog (Netflix to the rescue again!) but I wanted the Plus service simply so we could actually watch our shows on the TV set, which is supposedly one of the big "Plusses" of Hulu Plus, right? Well, subscriber beware: start reading the fine print.

Hulu has a pretty good selection of shows, and prior to the switch I was happy to see a lot of my favorites would still be available. I'm not talking "popular" garbage like Glee or Desperate Housewives (gag me!) I'm much more interested in some of the more niche material like House Hunters. What I missed, though, was the little label under the show description:

We do not have the rights to make it available on TV or mobile devices at this time — we continue to work on securing these rights.

And what's more, that message is on the vast majority of their content. Essentially you can only watch on your TV (or mobile device) if you really like popular network TV series. I recognize that this is somewhat beyond Hulu's direct control, but it does make me question wether or not it's worth the subscription at all. What exactly am I paying for?

Commercial free episodes? Oops! Sorry! They don't offer that for any price.
Backlog of shows? That's nice, but Netflix handles it better. I want Hulu for new episodes.
Ability to play on my TV? Yeah, we covered that. This is pretty much useless to me, since the 10% or so of shows that they can actually stream to my Roku player aren't the 10% I want to see. Also, unlike Netflix very streamlined interface the Hulu channel for Roku is clunky and slow. Text is designed for HDTVs only, and simple things like seeing a list of "recommended" shows or searching takes 5 or 6 very. slow. clicks. It's nearly unusable.
Ability to play on mobile device? This might be nice, except it suffers from the same problem as above PLUS they don't have a player for Android devices.

Actually, that last point deserves special mention: Both Hulu and Netflix will happily stream to your iWhatever, but they're also apparently content to ignore arguably the most popular smartphone OS of the day: Android! Netflix has some lame excuse about DRM being hard and the hardware being fragmented (yet they managed Windows Phone 7 just fine, practically on launch day) and Hulu is basically ignoring the issue all together. I can't say that I want my phone to be the primary device I use for media consumption, but the option would be nice! This is an area where everyone fails equally. Wake up and get with the program, people!

I'm not sure if I'm going to cancel Hulu just yet. The subscription program is still young and I have a lot of hope that they'll improve it, and they need money to be able to do that. I will say that my wife has pretty much ignored the service, though, and if I can't see some forward momentum I will drop it within a few months.

So to sum up my experiences in giving cable the boot: Netflix is far better than expected, Hulu isn't worth your time anywhere but a computer monitor, and Comcast hasn't been missed at all!

My Mac experience: Awesome, and occasionally awesomely frustrating.

2011-01-06T22:45:00.000-07:00

I mentioned in my last post that I replaced my old (extra crispy) PC with a Mac. I've been using it pretty heavily for more than a week now, and wanted to post some of my initial impressions. Just FYI: I'm fairly new to this whole Apple thing, so some of my comments may be the result of ignorance on my part. If they are, feel free to correct me. (Note: "You just don't get it, it's better that way!" is NOT a valid form of correction.)

The great:

Out of box experience is FANTASTIC. Plug it in, turn it on. Done. Windows is physically incapable of providing an experience like this (though that's not always a bad thing. More on that later.)
NO CRAPWARE! The things that come pre-installed (iMovie, Garage Band, etc) are actually full, useable programs that I want to keep! What a concept!
I love the program installation model. 90% of apps are "Drag into folder to install, drag to trash to uninstall." Beautiful! And it's only possible because...
No registry! I know that this can have it's downsides (the windows registry does address some issues) but most of the time it's just a badly maintained disaster waiting to happen.
Everything is... shiny! It's silly, but all those smooth little transitions make more of an impact than you know. Apple seem to be the only software company that really, truly appreciates how much a slick UI matters.
It's silent! At first I was upset by how load the magic mouse was dragging across my desk, but when I switched back to my G500 I realized that it was just as loud. The problem was, apparently, that this was the first time I could actually hear my mouse movements over the cooling fans. Wow. The loudest part of my computer now is the external hard drive.
Drivers, or lack thereof. I plugged in my printer and it just worked (didn't even tell me it was "Installing new hardware."), same goes for pretty much everything else. The most surprising so far: NO additional drivers needed to sync, develop for, and debug on my Android phone! There was one notable exception, which I'll mention in a bit.
I can actually let my computer go to sleep now and not have to worry that it may not wake up again. That's really REALLY nice.

The quirky (things that are weird to a newbie, but not necessarily "bad"):

It kinda weirds me out how programs don't actually "close" when I hit that little X in the corner. Well, most of the time, anyway. Some programs do. The inconsistency takes a bit to get used to.
All those little shortcut symbols used in the menus are greek to a new user. Command is easy to figure out (the same symbol is on the keyboard), and Shift can be guessed with a bit of thought, but that Option symbol is terrible. I only figured it out through careful experimentation. Same goes for Escape and Control.
The mouse ballistics feel all wrong to me. I simply can't get them to a comfortable point. Not to mention that the highest mouse movement speed is way too slow using the magic mouse. I'm glad I can crank it up with my G500.
Dual screen use is odd. It works okay, but to have the menu for all windows stuck on the main screen is inconvenient at best.
My nice 5.1 speaker system is useless now, since it was the "3 analog jack" type. (Yes, you can get converters. For the price you may as well get new speakers, though.) Not really Apple's problem, since they provide an optical out. Makes me sad though.
Had to buy a DisplayPort to VGA adapter after finding out that the DVI port on my old monitor apparently wasn't the right kind of DVI, rendering my DisplayPort to DVI converter useless. More ViewSonic's fault than Apples, but annoying nonetheless.
The magic mouse served to very quickly remind me of how much I depend on the middle mouse button (wheel click). It's annoying that there's no way to simulate that with some gesture. (Three finger click? I don't know.) I want to love the magic mouse, but I can't use it without going crazy due to this.

The maddening:

The hardware is limited and expensive. As a graphics developer and a gamer I was forced to buy the second tier of iMac hardware JUST to get a decent graphics card. I love the computer, don't get me wrong, but there's no question that it cost more than an equivalent PC (sorry, it does. You can't convince me otherwise) and that the choices are very slim. I would LOVE the ability to get a 3Ghz machine with the 5670 GPU, but I can't. I know this is part of what allows Apple to make such a stable OS and machine, but it doesn't stop it from being a major frustration. As much as Windows gets dinged for compatibility issues, the fact that they don't have such an iron grip on the hardware they are compatible with means that ANYONE can find a Windows machine that suits their needs. The same cannot be said about Apple hardware. (Nope, not even if you consider the Mac Mini.)
Finder is painfully primitive compared to Windows Explorer. There's some nice touches (like playing music files directly from the thumbnail) but way too many omissions: There's no obvious way to go up to your parent folder (had to look up a keyboard shortcut). Opening an image in Preview doesn't let you navigate between any other images in the same folder. No built in way to open a terminal window from your current folder. No way to manually enter a file path. No "New file here" option. Oh, and to rename a file you press "Enter" but to open it you press "Command+O"? Really? I'm sorry, but that's just stupid.
What, exactly, is that stupid little green "+" in the corner supposed to do? SOMETIMES, if I'm lucky, it maximizes a window like I wanted. Other times (like in Google Chrome! Augh!) it just makes the window "a little bigger" (for seemingly random values of bigger). Is it really that weird to want a consistent method for making my window fill my screen? Oh, and on the subject of resizing...
Sorry guys, but the lower right corner just ain't cutting it for me. There's really no reason why I shouldn't be able to grab ANY side or corner of the window and stretch it out. Yes, Windows did it first, but that doesn't mean it was a bad idea. Suck it up, admit they did it right, and make all of our lives a little easier.
With all the peripherals that just works, the fact that I had to go pull a random driver built by a hobby developer off of Google Code to get my XBox 360 gamepad to work at all was a massive disappointment. I know that it's Microsoft's product, and I know that Apple isn't a big fan of games, but after everything else worked flawlessly this was a huge let down, and there's really not much excuse for it.
The fanboys. There's a reason why people dub it "the cult of Mac." You guys are worse than the Linux zealots out there sometimes! I'm sure that someone out there is reading this post right now and preparing to explain to me in great detail why all of the flaws I listed above are defects in ME, not OSX. You know what: I just don't care. If I'm fighting with the machine to get it to do what I need it to, it has failed. Windows failed a lot. OSX fails less, but it still fails, and it's all the more annoying because of how much they managed to get right.

Despite all my gripes however (and I didn't even list them all here) the fact remains that the thought of booting into my Windows partition now makes me cringe. I still work with Windows at my job every day, and I don't have a problem with that, but when I'm at home I stay in OSX as much as possible. Even with some real head-scratchers in the design department it blows away anything Microsoft has to offer. I love my Mac, and I'm glad I made the switch.

Now can someone give me a working "Maximize" button?

Back up your data! Now!

2011-01-01T11:31:00.000-07:00

So, wow... this has been a crazy last week for me. I managed to luck into having the entire week off of work, so I was looking forward to a nice break that I could spend working on my game project (which I mentioned a couple of posts back) to try and get it ready for the Mozilla Game On competition.

[Quick side note here: Apparently Mozilla decided to go and drop WebSocket support in Firefox 4 due to security concerns with untrustworthy proxies, something which I utterly fail to understand as being a WebSocket concern. If your proxy is lying to you, aren't you pretty much screwed anyway? Whatever the reason, though, that put a serious cramp in my entry as is because I relied heavily on WebSockets to get performance that didn't, you know, suck. I was rather annoyed about the whole thing, but that's not much of a concern for me now...]

Before I jumped into my little coding spree, however, I did a little post-Christmas shopping with some gift cards I received. I used one of these to purchase a new hard drive, since I was running a bit low on space on one of mine. Took the new drive home that night, plugged it in (SATA, so the plugging part was ridiculously simple) and flipped the computer on...

...and all hell broke loose.

My UPS immediately began to wail because I was apparently drawing too much power. The fans on my machine were spinning, but the machine wasn't coming up. Then I caught that delicious scent of burning ozone. (Anyone who has ever worked with computer hardware knows and fears that scent). Switches were flipped and plugs were pulled quickly, but I knew that this had just turned into a long night.

My first instinct was that the power supply must have fried, but upon further inspection imagine my surprise when I found that the burning smell was actually coming from the area of my CPU! Oh Joy. Attempts to turn on the machine again were met with absolute silence. No fans, no hard drive spinning, nothing. I called up a nearby friend and ran over to his place to borrow a spare power supply real quickly, but my hopes of making a quick recovery were sliding downhill fast. Upon plugging THAT one in, I got the case fans spinning again (Dead power supply: check!) but also started getting that nasty burning smell from the CPU once more, and no POST beeps whatsoever. (Dead CPU and/or motherboard: check!)

So now I was faced with a decision: Do I even bother to rebuild my PC? It was about 4 and a half years old (with one Motherboard replacement mid way through it's lifespan), so I had gotten a good deal of use out of it, and frankly it had been acting a little wacky lately. I did some quick window shopping at Newegg and figured that if the video card and hard drives were still good I could probably do a decent rebuild for about $400 and end up with a reasonably up-to-date machine. That's all well and good, but there was part of me that actually cringed at the idea of cobbling together the parts and hoping it all worked again.

That surprised me, to be honest, but after giving it some more thought I realized that it was true: I didn't WANT to tinker around inside my machine anymore. It was great to fiddle like that when I was a hobby developer in high school, or a poor college student trying to squeak in an upgrade and still make rent, but at this point I'm a professional developer with 7 some years under my belt and priorities have shifted a bit. I still enjoy playing with the hardware and putting machines together, no question, but when it comes to my personal computer I just want it to freaking work!

Of course, once I made up my mind to get a boxed PC, there was no question in my mind where I would get it from. I would never even consider a Dell or HP or Sony or what have you. I've worked with too many of them over the years and seen far too often the shortcuts made to cut costs. No thank you, my choice was quite clear.

So I got a Mac. Specifically the 21.5 in 3.2 Ghz iMac. Yes, it's stupidly expensive for the hardware, (and no, I refuse to concede otherwise. Apple hardware IS overpriced. Period.) but their warranties are very good and the hardware IS nice. They don't take shortcuts out in Cupertino. Plus the fact that I had long wanted the ability to develop for OSX and iOS was a huge driving factor as well. There are a lot of development opportunities in the Mac world now, far more than even a few years ago, and professionally it makes a lot of sense for me to familiarize myself with the environment and APIs, even if I'm not using them at my work currently. I'll probably do another post about my experiences in the Mac world soon, but outside of a few quirks I've been quite happy with my decision.

Anyone, on to the REAL point of this post: After getting my shiny new toy and setting it up, I went back to my old drives to pull over my photos, music, documents, etc. Except, when I plugged them into the USB adapter nothing showed up on screen. And I got that funny burning smell again. Well.... crap. (Dead hard drives containing years of family memories and important documents: Check!)

And NOW the panic sets in. Just think about all the stuff thats on your computer that you would absolutely hate to lose. Now imagine that your PC died catastrophically right this second, and none of that data was recoverable. Would you have any way of getting it back? Scared yet? I should have been. (Yes, I know that you can do data recovery. Unless you're a big business, though, it's not worth it. That's expensive stuff.)

So now the fact that I rarely ever did backups comes around to bite me hard. As a developer there's no excuse: I should have known better. I scrambled, and tried to retrieve a backup that I did to a free Mozy account about a year and a half ago, but it appears that they've gone and deleted my account without notifying me (Note to everyone: Avoid Mozy at all costs! They're obviously not to be trusted with maintaing your data!) After a bit more panic (and consoling a wife who believed we had just lost wedding and honeymoon photos) I did manage to dig up a drive from mid 2008 that I had actually done a backup to! YAY!

That gave me back a lot of my data, and some more recent things like photos of a Disneyland trip were still on my phone, so it wasn't an absolute disaster. There's still a good two year gap of photos and documents, though. Most critically: The game that I had been working on is now completely gone. I had been committing to a local Mercurial repository, but never synced it out to a server. (Stupid developer! No Cookie!) That's a good month or so of my own code that's completely gone. I'm going to re-build it, probably starting on Android this time, but I will be starting completely from scratch. Not. Happy.

Moral of the story: Back up your data! As in: Now! And frequently thereafter! My new Mac has a feature called Time Machine that does backups to an external drive for you automatically, and you can bet that I'll be making good use of it! Don't be a stupid developer like me, don't assume that your computer will be running tomorrow, and don't get stuck digging through your closet in a panic for two year old data! Just backup!

Tron: Legacy (super short!) review-ish

2010-12-20T22:01:00.000-07:00

Just got back from Tron: Legacy. It was, well, Tron. Very pretty with a very silly back story. I enjoyed it, but more for geek factor than anything else.

There was one MAJOR letdown though, and that's that the film makers had the chance to include the most epic XKCD inside joke EVER and they totally blew it! Shame on you!

I'm not dead yet!

2010-11-17T22:16:00.001-07:00

Just wanted to drop in and leave a quick note to say that I haven't abandoned the blog, and I actually have a fun little project under way! Unfortunately I don't want to talk about it much at the moment, since I feel like this is something that could actually be marketable (probably via the Apple App store/Android Market). I'm also considering submitting it in it's web-based form to the Mozilla Game On competition. But that depends on how quickly and how well I can polish it up.

In any case, it is a game and I'm developing/prototyping it using HTML 5 tech like Canvas (no WebGL this time around, sorry!), Audio tags, and WebSockets. Great fodder for blog posts, but once again I would very much like to have a more complete version before I go chatting it up. Expect some updates in the next few weeks though! Even if I'm not talking about the game itself I'll probably want to talk about my experiences with the HTML 5 stuff. (I feel a rage post about the Audio tag brewing as we speak...)

Till then!

"The Social Network" Review-ish

2010-10-19T08:17:00.000-06:00

So I had a chance to see "The Social Network" the other day. I'll admit that it wasn't a movie that I was really intent on seeing but a Hollywood blockbuster about (of all things) a programmer/website struck me as an odd little beast, so I was somewhat intrigued.

Now that I've seen it I think that It's probably worth at least one viewing by every individual who has ever built a piece of software. That's not because I really liked the movie, though. True, I found it to be an entertaining (albeit highly fictionalized) tale, and the actors do an admirable job portraying their overblown characters. I wasn't a big fan of the way they tried to portray the whole thing as this wild "sex, drugs, and rock and roll" lifestyle, though. I'm sure that there are programmers out there who live like that, or at least want to, but that's a pretty big disconnect from most of the developers I know. To me that whole aspect of the movie felt heavy handed and overdone and as a result it was probably a one-time viewing for me. So while I thought it was a good flick it certainly wasn't my favorite. (I'd easily go see Inception 3 more times before before bothering with this one again).

Anyway, the thing that really struck me about the movie, the thing I think is important about it, is that it is the first movie that I have EVER seen that actually portrays programming/hacking/development almost completely accurately. There's a fascinating monolog at the beginning of the show in which Zuckerberg (the screen version, anyway) narrates some of the steps he's going through to collect pictures for a prank website that he's building. It's a fairly lengthy scene, and shockingly doesn't try to gloss over the technical bits. It's packed with terms like "apache server" and "wget" and "bash shell" and all sorts of authentication odds and ends, and every single one of them is used in the correct context! This type of portrayal continues throughout the movie: Every single computer screen displays real OSes, real sites, and real code running on real hardware (the OSes are even date-appropriate!) They talk about real languages and real algorithms and generally act like real programmers. What a concept!

Of course, a lot of this is just incidental stuff and most of it is done rapidly enough that audience members who don't know their Perl from their Python will simply pass it off as techno-babble and still be able to focus on the story. But for an industry that has long portrayed "hackers" as semi-mystical beings who mash random keys while staring at psychedelic screens and suddenly have access to anything the very grounded and realistic approach taken here is very surprising and refreshing. It's worth a viewing if only for that: to bear witness to the day that Hollywood recognized that programmers were real people.

One final recommendation for the show: The soundtrack is great, and makes for (ironically) great music to code by. But, hey! It's Trent Reznor! Would you honestly expect anything else?

Itty Bitty WebGL

2010-09-29T22:02:00.001-06:00

File this one under "Cool but useless"

I stumbled across js1k.com yesterday and was quite amused by the concept: Use 1024 bytes or less of javascript in a minimalistic shell page to create a cool demo. The contest is over now, and browsing through the winning entries is surprising and somewhat awe inspiring. They managed to get a full Chess AI and graphics in 1k of javascript?!? Awesome! It also got me thinking: None of the demos used WebGL (primarily for compatibility reasons. The rules state that demos must work in Safari, Chrome, Firefox, and Opera), but were the standard further along, how much COULD you do with a 1k WebGL app? I decided to find out!

Of course, all you have to do to be considered a "WebGL app" is create a valid context:

<!DOCTYPE html>
<html>
 <body>
  <canvas id="c"<>/canvas>
  <script>
   g=document.body.children[0].getContext("experimental-webgl");
  </script>
 </body>
</html>

Hey! WebGL in 61 bytes of script! YAY! But how interesting is that, really? We want actually SEE something, right? As every good graphics developer knows, the very first thing to try is clearing the viewport to some easily identifiable color. I took it a step further, and made the canvas pulse red:

g=document.body.children[0].getContext("experimental-webgl");
i=0;
setInterval(function(){
    g.clearColor(Math.sin(i+=0.1),0,0,1);
    g.clear(16384)
},1);

Okay, so now we're at 144 bytes (if you subtract the whitespace) and a red pulsing screen! That's ALMOST small enough to fit in a tweet! In fact, once the browser vendors drop the "experimental" from the context name it WILL be small enough to tweet. (Though you can do it right now for Firefox by using "moz-webgl" instead.)

A note about the above code: Some of you probably caught the g.clear(16384) call and said "Huh? 16384?!? Where did that come from?" Well, that just happens to be the integer value assigned to the WebGL symbol COLOR_BUFFER_BIT, which of course is how we indicate that we're clearing the color buffer. By using the numeric equivalent instead of the symbol we save 11 bytes, which in this case is invaluable!

Please note: I would NEVER EVER condone doing such a thing in a "real" app! Not only is it horrendous to read, but if the constants change for some reason (and they may. It's not set in stone) or if one browser does it differently than the others then our little demo blows up and dies. In this case both Firefox and Chrome define their constants the same way, so it works. Just don't do it anywhere else, ok?

So, that's great and all, but WebGL is all about geometry, right? We should be able to get a triangle on screen without much fuss, right?

Well....

Consider everything that you need to set up to show any geometry in WebGL: You need to set up the shaders, compile the shaders, activate the shaders, define the verticies, push them into a buffer, bind the vertex buffer pointers, and finally draw the arrays. And ALL of those function calls are verbose things like "enableVertexAttribArray". It's a lot to do, so can we fit it all in?

Of course!

c=document.getElementById('c');
c.height=300;
g=c.getContext('experimental-webgl');
g.clearColor(0,0,1,1);
g.clear(16384);

p=g.createProgram();

function l(x,y){
    s=g.createShader(35633-x);
    g.shaderSource(s,y);
    g.compileShader(s);
    g.attachShader(p,s);
}
l(0,'attribute vec2 p;void main(){gl_Position=vec4(p,1,1);}');
l(1,'void main(){gl_FragColor=vec4(1,0,0,1);}');

g.linkProgram(p);
g.useProgram(p);
g.enableVertexAttribArray(0);

b=g.createBuffer();
a=34962;
g.bindBuffer(a,b); 
g.bufferData(a,new Float32Array([0,1,-1,-1,1,-1]),35044); 
g.vertexAttribPointer(0,2,5126,false,0,0);

g.drawArrays(4,0,3);

And now we're getting somewhere! Minus whitespace we're now at 576 bytes, so a little over half our limit (plenty of room to play around in). Of course, we're also introducing a lot more nasty tricks. For example: it's probably a really bad idea to just assume that you know what index a vertex attrib is going to be! We're also neglecting to set up a viewport and not bothering with perspective matricies, and assuming that this will produce a coordinate system of [-1,-1] to [1,1] But when all is said and done this DOES give a single red triangle in the middle of a 300x300 blue square (FYI: canvases have a documented default size of 300x150, so assuming the browser follows that we only have to set the height to get a square.)

But, of course, our result is static and could easily be outdone by a poorly made GIF. This is WebGL, we need movement! That complicates things further, but we can still stay pretty slim:

c=document.getElementById('c');
c.height=300;
g=c.getContext('experimental-webgl');
g.clearColor(0,0,1,1);

p=g.createProgram();

function l(x,y){
 s=g.createShader(35633-x);
 g.shaderSource(s,y);
 g.compileShader(s);
 g.attachShader(p,s);
}
l(0,'attribute vec2 p;uniform float t;void main(){float s=sin(t);float c=cos(t);gl_Position=vec4(p*mat2(c,s,-s,c),1,1);}');
l(1,'void main(){gl_FragColor=vec4(1,0,0,1);}');

g.linkProgram(p);
g.useProgram(p);
g.enableVertexAttribArray(0);
u=g.getUniformLocation(p, 't');

b=g.createBuffer();
a=34962;
g.bindBuffer(a,b); 
g.bufferData(a,new Float32Array([0,1,-1,-1,1,-1]),35044); 
g.vertexAttribPointer(0,2,5126,false,0,0);

t=0;
setInterval(function(){
 g.clear(16384);
 g.uniform1f(u,t+=0.01);
 g.drawArrays(4,0,3);
},1);

And now we spin! We're at 721 bytes now, which leaves us a paltry 303 more to play with in our 1k bounds. That would be tough to utilize, but I'm sure someone could do it! Granted, this isn't really going to win any competitions, and frankly I think WebGL is simply too verbose to really compete with a decent canvas app in a space like this, but it's a fun and ultimately useless experiment that I wanted to share!

Raytracing in Javascript

2010-09-24T08:28:00.000-06:00

So I've been pretty quiet lately, but not for lack of activity on my part. I'm in one of those funny stages right now where I've got about a thousand ideas I want to try and things I want to learn and not enough time to pull any of them off properly. So I experiment, whip up several dozen cool little proof-of-concept programs, and move on to the next idea. This kind of "brain dump into code" mode is pretty fun, but has the severe downside that none of your projects reach a point where their worth actually showing to anyone else. Recognizing that my blog has been sorely neglected as of late I determined that I would bring at least ONE of my playthings to a semi-showable state and post it, and it so happens that my raytracer was the closest to being presentable. :)

Yes, it's a browser demo (but not a WebGL one, sorry!) It's a Monte Carlo tracer, which means that the image starts out really ugly and gets better as more passes are applied. (Let it sit longer and it becomes prettier.) The image above is the result of letting it sit for 400 passes, and even then you can see a fair bit of noise. It's a very slow way of going about it, but it can produce spectacular results under the right conditions (and, you know, when you're not relying on Javascript to do it...)

Okay, so admittedly this is probably the least impressive demo I've done so far. It's your standard old boring Cornell box, and not even a really cool one at that because it lacks a refractive (glass) sphere. (More on that in a bit). There's a whole bunch of things "wrong" with it, and I may be tempted to come back and update it at some point, but the point of the thing was that I wanted to learn more about raytracing, and this little guy has served that purpose admirably.

A few fun "techy" notes: This started life as a Python port of smallpt, which I eventually gave up on when I realized that the GIL makes threading for this type of thing pretty useless. So I ported my port to Javascript (wait, aren't we moving backwards here?) mostly so I could use Web Workers, but it has a nice side effect of letting me demo it online. :)

The demo uses 4 web workers (though it's super easy to scale to more or less), each of which renders their own jittered version of the full scene. All passes are then send back to the main thread and composited into the final image. I'm actually passing the image data back and forth in an ImageData object, which means that some browsers may not support this yet. It also means that there's the potential for some pretty severe precision loss, since all of the color values are compressed into a 0-255 range, sent to the main thread, and blended with the main image, which does another 0-255 compression. It would be far better to store and composite all of the color components as floats (or doubles, but Javascript doesn't give you the option. It falls somewhere in between), and copy those values over to the image after compositing. I tried this at one point, but the Web Workers kept dying and spitting out a strange "Not enough arguments" message. This is apparently a Chrome bug, but I have no idea how to get around it.

I also eventually dropped the idea of having a refractive sphere, simply because after many MANY iterations I still end up with either a black ball or a clear ball with an ugly black ring around the edges. You can still see the material class for it in the worker script, and if anyone wants to point out where I'm being stupid be my guest!

Anyway, as unimpressive as it is I did have a lot of fun putting this together, and hopefully someone out there at least finds it mildly interesting! For now, though I'm moving on to my next project, which may or may not be any one of the following: (haven't decided yet)

A WebGL accelerated version of this demo, to compare speeds
A SketchUp to JSON exporter for quick prototyping of WebGL scenes
An OpenGL ES 2.0 demo on Android (very tempting)
A WebGL Material system
Or possible something completely different! I have a bit of an itch to do something non-graphical for a bit, just to shake things up.

I've also got a couple of blog entries I want to do about some subjects I've found interesting lately, so hopefully there won't be as long of a gap before my next post.

Setting up an OpenGL ES 2.0 dev environment for Android (The horror!)

2010-08-29T23:33:00.002-06:00

It is a well known phenomenon that if you give a software developer (a REAL developer) any device which contains a CPU (and even a great many that don't) you will also install in them a great and uncontrollable urge to put their own programs on the damn thing. This urge cannot be satisfied by any old "Hello World" app either, oh no. It has to be something significant. Something that uses the device in a non-trivial way. Something that makes your peers say "Woah!" (The "Woah" factor is incredibly important, ask anyone.)

As such, being a developer and recently coming to posses a shiny new smartphone I've been dying to do some code for it for a while now. Finishing up some of my WebGL stuff and keeping up with my normal work (you know, the one that actually pays me) has prevented me from acting on it so far, and I still may not get a chance to code for it for a little while, but I finally had enough time lately to sit down and actually set up the SDK and play with some of the examples.

Now, I have a history of managing to choose the absolute worst case scenario as my first attempt at any platform I develop for. One of my very first large-scale .NET apps had me writing dynamic MSIL injection to create arbitrary event signatures, which is a really harsh way to introduce yourself to the language, let me assure you! In this case, hearing that Android can run OpenGL ES 2.0 was quite exciting to me, especially since that's essentially what WebGL is anyway so in theory it shouldn't be a huge change in direction. Thus being the graphics geek that I am that was my first target. (After shouting my salutations to the world, of course. Tradition!) This turned out to be a Bad Idea™ of epic proportions.

I was able to get the OpenGL sample app (a couple of spinning cubes) up and running on the emulator immediately, so it all seemed to be going well. Looking at the code, however, I realized that the sample was using ES 1.0, which was less-than-desirable. Why would I want to trudge through fixed function madness when I have a fully programmable GPU at my disposal? So I set out to find an ES 2.0 example.

I found it. Yes, that's a link to the NDK (Native Development Kit). Turns out that ES 2.0 is only accessible via JNI and C code. That's a bit of an annoyance, but in many ways it makes sense. Java isn't really designed for things like high performance graphics, and I actually prefer writing native code anyway so a little bit of Java/C interfacing wasn't a huge turn off. So I downloaded the NDK, got it set up, and tried to build the program... and found out that the native bits need to be compiled separately. In cygwin. *sigh*

Note for aspiring developers: Make sure you click the checkbox for the "make" package when installing cygwin. It'll save you time and grief. Trust me!

Okay, so now cygwin is installed, the native bit is built, I can link it to the Java project, and launch it on the emulator... all's well, right? Not exactly. You see, I use the word "launch" loosely because while the app does start up it crashes within moments without ever showing anything on screen besides a nasty error message. Poo. Only after much toy around with settings trying to get this to work do I discover that the emulator that you use for debugging doesn't support OpenGL ES 2.0. Absolutely fantastic. This means that in order to develop against that API I need to constantly be deploying unstable code to my phone. Color me thrilled. That fixed function madness is starting to look really attractive about now...

In any case, that means I have to be able to actually push developed apps to my phone, which I hadn't done yet. The method for doing this is via a simple little command line app called ADB. You call adb install app/path to push your freshly built app out to your phone via USB, easy-peasy. Well, except when it starts yelling that there's no phone attached. (Of course there is silly computer! See! The cable is right here!) So a couple of things need to happen in order for ADB to "see" the phone:

The phone needs to be set to allow debugging (settings->Applications->Development->USB Debugging)
The phone needs to be set to "Charge Only" mode when connected, which I found slightly counter intuitive. (This may be Droid X specific. I've heard conflicting stories online)
You need the appropriate ADB driver installed.

Item three seems obvious, and in fact the first time you plug your phone in there's a whole slew of drivers installed. With my X, however, it was complaining that it couldn't find the right ADB driver. Fortunately the driver you need is included with the SDK if your phone doesn't install it automagically. (/usb_driver) Unfortunately when I attempted to point my device manager to this driver manually it just shrugged and told me it couldn't find anything. So... back to the forums I go in search of help.

As it turns out there's an issue with Motorola's phones and the SDK drivers under Windows 7 64 bit (take a wild guess what I'm running!) that requires you to dive into the .inf and replace some device ID's manually. For me, I ended up needing to change the Motorola device ID's from this:

[Google.NTamd64]
;Moto Sholes
%SingleAdbInterface% = USB_Install, USB\VID_22B8&PID_41DB
%CompositeAdbInterface% = USB_Install, USB\VID_22B8&PID_41DB&MI_01

to this:

[Google.NTamd64]
;Moto Sholes
%SingleAdbInterface% = USB_Install, USB\VID_22B8&PID_428C&REV_0216&MI_01
%CompositeAdbInterface% = USB_Install, USB\VID_22B8&PID_428C&MI_01

Your mileage my vary.

So now, after much pain and suffering (this whole process spanned several days of random poking) I was finally able to send my ES 2.0 sample app to my phone and see it running for the first time! YAY! The reward for my perseverance? An ugly green triangle against a pulsing grayscale background. Hm... ever-so-slightly-less-than-impressive. But it's a start, and now that I've got all this garbage sorted out I should be able to make some actual headway developing something cool. Something with a good "Woah" factor. :)

I have come away from this experience with a couple of thoughts: Primarily that if Google EVER wants to compete with Apple in the mobile gaming market this process absolutely must to get better! Granted, many of the driver issues and whatnot were just bad configuration for my phone (and I seriously doubt Motorola cares about developers working with their phones in the slightest) but the very idea that I have to deploy my code to my phone with every change just to run the thing is appalling. Yes, you will probably get some very motivated people that do a few cool tech demos with it but many serious development studios are going to balk.

Second: This entire process feels so incredibly... Linux-ish. I know, I know, that's what the thing is built on so of course some of it is going to shine through, but the entire snobbish "if you can't figure out how to set up a make file by hand then you're obviously not worthy to develop for our OS" attitude is something that we don't want to inherit! Even the hardcore amongst us appreciate a good drag and drop UI builder sooner or later. After playing around with the Android SDK/NDK it's increasingly difficult to ignore the siren's song of Apple's nicely packaged development environment. (I mean, come on? Xcode vs Eclipse and cygwin? Is that even a contest?)

Still, I don't have an iOS device, and even if I did I don't have a Mac to develop with (not for lack of wanting...) In the end I'm going to develop against what I've got, no matter how painful the process might be. But I'm a hobby developer with a limited budget, those excuses don't apply to big publishers and that's where platforms like this are really built or buried. There's a lot of potential here, I'd hate to see it passed by because of some crummy development tools.

Oh, and since this is a post about my phone: MotoBlur still bites. And no, I'm not going to let that one go.

Communication across language barriers

2010-08-28T11:26:00.000-06:00

Had a neat experience while out walking my dog today. We came across a very kindly and very lost elderly Hispanic gentleman that flagged us down to ask for help. Since the first thing out of his mouth was "I no speak-a English very good." I was concerned that I would be fairly useless with whatever problem he was having. (Two years of Jr. High Spanish and all I can say fluently is "I don't speak Spanish".) Luckily, however, he was able to point to an address scrawled on a bus schedule and indicate that he was trying to reach that location. I knew where the address was, and fortunately he wasn't more than a couple of blocks off, but I was somewhat at a loss for how to communicate directions to him outside of vague hand gestures.

Then, in a moment of fleeting intelligence, I recalled the trusty little Android phone in my pocket that I had been listening to music on. Pulling it out, in under a minute I was able to call up Google Maps, punch in the address, have it map out the directions, and show this gentleman the results. Even though the map was in English a big blue line guiding you from where you are now to where you want to be is pretty universal. After studying the map for a moment a look of understanding and relief swept over him. He thanked me several times, a sentiment that I had no problem understanding, and headed off to his destination no longer lost and much happier for it.

For my part I was left with a mix of the warm fuzzies you get from helping another human being and wonder that we live in a day and age where I can hold in my pocket a tool that can completely transcend language and cultural differences and allow me to help someone in need. I don't care who you are, that's just cool!

As a programmer it's very easy for me (and, I think, others of my profession) to loose sight of why we surround ourselves with all this technology. We can easily fall into the trap of pursuing technology for the sake of technology. (I'm certainly guilty of that.) It's worth reminding ourselves once in a while that all of these spectacular devices and innovations are really there to serve one singular purpose: To make our lives better in some little way. Whether it does that by making some task a little easier, bringing us the information we need, making us safer, or just providing a few moments of entertainment, it's all in the name of improving our day to day existence. If we keep that in mind, magical things can happen.

Random WebGL related gunk

2010-08-26T18:40:00.000-06:00

There's a few things I've come across in the last few days that I feel are worth pointing out to the WebGL community:

Newer builds of Firefox have started validating the textures passed to generateMipmaps according to the OpenGL ES 2.0 spec. This means that attempting to generate mipmaps with non-power-of-two textures will start failing on you very soon! Please update you code/resources accordingly. (This already bit me with my Quake demos, which have now been tweaked accordingly.)
WebGL___Array types are going, going, gone! Newer browser builds don't even have them defined, so make sure you are using the appropriate analogs from the Typed Arrays. Again, my demos have been updated to account for this.
In an odd twist, whereas Firefox has been speeding up lately apparently Chrome is cracking down on crazy fast loops now. With the latest dev releases (7.xx) I've noticed that my Quake 3 demo is basically locked at 30fps, regardless of whether or not you check "Cap FPS". Curiously, this does not seem to be the case for my Quake 2 demo, which means that Chrome is actually limiting the internal message pump. Granted, it's a very stable 30fps, and I'd rather have a lower but stable framerate than a high but erratic one, so I'm not complaining. It did strike me as an interesting change, though. A bit of background on this one: For the Quake 3 demo in order to get the best framerates possible I used a hack picked up from Vladimir Vukicevic that continually posts messages to the page as a signal to draw (demo), which attempts to avoid some of the resolution limitations of setTimeout/Interval. This apparently causes severe performance issues if allowed to run rampant, however (as commenters on this site have pointed out) and so it's possible that this apparent capping may be Google trying to prevent abuse of the system. (Which is probably a good thing!) I'd love to know a bit more about the actual changes that took place in this regard, but in the end it appears that the timer-based method is the better way to go.
Finally, in a bit of news that has nothing to do with WebGL whatsoever, I find this to be supremely amusing. Chrome envy, perhaps? :) It's especially funny in light of some Google-bashing that Microsoft did a few months back about how a single address/search bar was a bad thing for users privacy.

Firefox 4 beta 4 - Hardware Acceleration FTW!

2010-08-24T18:24:00.003-06:00

I just wanted to jot down a quick note about something that made me VERY happy today! Firefox 4 beta 4 was released today, and one of the big features that it includes is (On Windows Vista/7 anyway) hardware acceleration via Direct2D. It's disabled by default, but doesn't take much to turn on (and certainly shouldn't be a problem for anyone who's been following WebGL).

First off, let me say that text rendering looks 1000% better when hardware acceleration is enabled. Seriously! I'm usually not one to care about aliasing in my text and whatnot but... WOW! The really exciting bit for me, though, was booting up my Quake 3 demo. With acceleration turned off I'm getting about 20-23 FPS (on par with previous betas), but with acceleration enabled I'm seeing 60-70 FPS! OMGWTFBBQ!?!

A 300% bump in render performance? That's insane! Awesome, but insane! Great work, Firefox crew! I can't wait to get this out of beta and into the average users hands.

(And now Chrome needs to get with the program! Seriously, looking at text in any other browser is gonna bug the crap out of me now!)

UPDATE: Ask and it shall be given, I suppose? I've turned the associated flag on, but can't see any difference in performance due to the apparent message pump cap I mentioned in a newer post. I'll see if I can find some better way of performance testing this later.

Spring (er.. late Summer?) Cleaning

2010-08-24T07:52:00.001-06:00

As a heads up, some of my older demos (well, basically everything but the Quake 3 demo) are probably broken on newer browser builds at this point due to changes in the WebGL's API. (Hey, that's what you get for developing against experimental libraries!) Over the next couple of days I'm going to go back and fix them up (may even try to optimize the code a bit), so hopefully everything should be working again soon.

Until then, sorry for any broken code you may stumble across!

Update: So I've got all of the demos to stop throwing errors, but the Spore and Hellknight demo's aren't showing anything, even though they appear to load correctly. I'll continue looking into it, but in the meantime at least the Quake 2 and 3 demo's are running fine.

Droid X Android 2.2 (FroYo) Micro-review

2010-08-21T23:20:00.002-06:00

So anyone patiently waiting for Motorola to release Android 2.2 for the Droid X has probably heard by now that an official build of it was leaked and can be installed on any X, rooter or otherwise. I was personally quite excited by this news, since I had been looking forward to the new release to see if it addressed some of the software shortcomings that I had found in the phone. As such, it took me all of about half an hour after I found out about the leak before I was running 2.2! (Disclamer.... voided warranty... not supported... blah blah blah...)

Now, before I continue I should note that there is a chance that this is not the final update that will be released OTA. That said, I doubt that Motorola is planning on making any major changes if they still intend to hit an early September release.

If you want to skip my earlier review essentially I think that the X is a beautiful piece of hardware that is badly hampered by Motorola's terrible software. (MotoBlur, uninstallable services, etc.) So as much as I doubted it I kept a little spark of hope alive deep down in my heart that Motorola would hear the impassioned cries of their faithful userbase and use the 2.2 update as an opportunity to strip away the bloatware that we all hate so much (or at least give power users a way to do so).

Wish granted? Nope. Actually, it seems to have gotten somewhat worse! For one, there are even MORE bloatware apps now: VZ Navigator comes pre-installed with 2.2, even though the default Navigator app outperforms it at every turn. Quickoffice has also appeared in my app list, though I haven't really tried using it. And yes, both new apps prevent uninstallation, which instantly sours my opinion of them. It's a shame, really, because in the case of Quickoffice it may even be a great app, but the fact that I have no choice about it being on my phone makes me instantly assume that it's crap (otherwise why would they prevent you from removing it? No one removes software they find useful).

The core OS is speedier and more responsive now, but this hasn't done the terrible Blur widgets any favors. They still cause homescreen transitions to lag like crazy whenever they're added. Additionally, the unlock screen has been updated to look, um... more metallic? I personally think it's pretty ugly (I like the original Droid's 2.2 unlock screen, a classy transparent version of the Droid X's old one), but that's a matter of taste. The real problem is that it's really jumpy and laggy. Swiping my finger across the bar produces 4 or 5 very visible "jumps" in the bars position, which gives the appearance of very poor performance. The main homescreen also retains the Droid X's original minimalistic "dock" theme instead of 2.2's nice looking and more functional dock, which is a disappointment. Fortunately Launcher Pro continues to work just fine, so you can avoid most of Motorola's bad design. In the end the story of Motorola's custom skin seems to be that after hearing loud and clear the communities dislike of it they did the sensible thing and made it worse. Good job, boys! Way to show your customers you're listening. *sigh*

So, the good news? Android 2.2 is a very nice update at it's core. Other sites have covered in detail the major new features like voice commands and Flash support. Those are all here, and working well, so I won't cover them in detail. One interesting thing to note in that area, though, is that I cannot seem to find tethering options anywhere, only the 3G Hotspot service app. This struck me as odd, since my friends with the original Droid and the official 2.2 update have a tethering menu underneath their main settings list. I hope that I'm just missing something or the final update will add the feature, because otherwise it feels like one of the most useful features of the phone is being suppressed so that Verizon can profit. Not cool.

For the record, I have tried a few flash sites, and found that it works pretty well all things considered. The performance is still lacking in some area's (it struggles with smooth, high-res video), but it's still perfectly usable.

There are a lot of niceties buried in the various apps. For one, the gallery app is MASSIVELY improved, in terms of visuals, performance, and usability. Its an app that I want to use now, which is certainly an improvement. In conjunction with that, the camera and camcorder apps have also been updated. There doesn't seem to be any new features outside of the ability to tap the screen to capture (I prefer the physical button but it's nice that the option is there.) but the UI is better looking and a bit less obtrusive, and the startup time for both apps feels much better. I've heard other people say picture quality has improved too, but I haven't had a chance to look. I do wish the music app had been updated a bit more, but it looks like I'll still be using Double Twist for that.

Some other goodies include an improved Battery Manager, the ability to disable data access while still allowing the phone to receive calls, a brightness setting that auto-adjusts depending on use, and Swype keyboard that's not hideously ugly! There are a few nice additions to the contacts list, such as how clicking on a contact's photo now brings up "quick tasks" for that record such as text, email, or call. I never personally had any of the issues that some people reported with mail syncing, but I've heard that those are resolved now to. It's also nice that I can switch to landscape mode by turning the phone either clockwise or counter-clockwise (like an iPhone) but I will say that I seem to get quite a few false-positives with that one, where the screen will jump to landscape mode at odd times. Oh, and the volume for pretty much everything is louder now. Yay!

That's everything that I can recall for now, though I'm sure I'm missing some things. All in all I would say this looks like a good update with plenty to be excited about. It's an absolute shame that Motorola didn't use this as an opportunity to respond to some of it's critics in regards to their sub-par software, but perhaps that's asking a bit much from a company that thought MotoBlur was a good idea in the first place. Maybe Google can finally knock some Sense into them when they come out with Gingerbread (3.0).

A man can dream, right?

Rendering Quake 3 maps with WebGL: Tech talk

2010-08-14T10:01:00.001-06:00

So, I promised I would talk more about the the development side of the Quake 3 demo and I'm here to deliver. Warning! This is going to be somewhat long and technical, so steer clear if you're just here for the shiny graphics!

So first off, why Quake 3? After all, CopperLicht has already been there and done that, and I think many people won't recognize this as much of a step up from the Quake 2 demo I already did. Not to mention that along with my Doom 3 model loader this is my 3rd id-related demo in a row, which may seem a bit boring to some people.

For me it mostly comes down to what I wanted to achieve personally. Thus far each of my demos have had been the result of a specific goal on my part: My Spore demo was just to familiarize myself with WebGL. The Hellknight model was to toy around with animation and game-oriented file formats. The Quake 2 demo was a (somewhat failed) stab at large scenes that the user could navigate. I had intended next to move on to some special effects (I still want to try Megatexture-style streaming, for example), but the problems with the Quake 2 demo left me itching for some closure. So I redirected my efforts and decided that I wanted to do a project that created a "real world" game environment in WebGL. Something that looked and acted like an actual game, running at framerates that would actually be considered playable, without "dumbing it down" for the browser. Partially for the challenge, and partially to demonstrate that yes, WebGL is (or at least will be when released fully) a viable platform for game development.

Given that criteria, Quake 3 very quickly became a natural fit. For one, I own the game and therefore had the appropriate resources at hand. Two, the formats it uses are well documented and easy to extract. Three, the format is similar enough to the Quake 2 maps that I could hit the ground running with the previous projects code. Finally, Quake 3 is very well known. People know what it looks like, what it FEELS like. If I could capture even some of that feeling and bottle it up in a canvas tag I knew it would catch people's attention. Oh, and Quakecon was coming up and I thought it would be really cool to release a demo right around the same time! :)

So, with that in mind the first and foremost concern I had was speed. Quake is fast. Quake is twitchy. Quake is not a 5 FPS slideshow because we can't convince javascript to do any better. I knew that I had to produce something that ran smoothly or the internet at large would dismiss it as "WebGL just isn't fast enough". This performance oriented mentality led to some interesting optimizations, some of which are a bit counter-intuitive at first glance.

The biggest thing I did was to try and reduce the number of draw calls as much as possible. To this end I packed all lightmaps into a single texture so that we could avoid switching textures where possible. I also pre-calculated all curved surfaces in the level (using a fixed number of segments per patch) and stored them as static geometry. All geometry in the level is stored in a single Vertex/Index buffer pair, so I can bind the VBO once at the start of the draw routine and never touch it again. Additionally the indexes were pre-sorted by shader. This allows me to draw all geometry that shares a single shader with one call, and subsequently means that I never make more than one draw call per shader stage.

Along those lines it may surprise some people to know that I am doing absolutely no geometry culling in the demo. The entire map, visible or not, is rendered brute force every frame. This probably sounds like a bad idea but the fascinating part is that I was never able to implement a culling scheme that actually improved performance! Brute force rendering was always faster. And when you think about it that makes a lot of sense.

When a WebGL draw call is made the actual rendering of geometry happens for all intents and purposes at native speeds, since most of it is simply spinning around on the GPU anyway. The only thing that really slows it down at all is jumping back and forth to Javascript, which is many orders of magnitude slower than your GPU and always will be. Thus the more processing we do in Javascript the slower we will go, and the more we can push of the the GPU the better. As such, in the time that it would take us to (in script) figure out where our camera is, trace through the BSP tree, find the potentially visible node set, test all the node geometry for visibility, flag it for rendering, and then actually do the rendering your graphics card could probably have rendered all of that unseen geometry 10 times and not have broken a sweat. So it makes a lot of sense (in this case) to just let the card do it's thing and not worry about whether or not you can see that particular triangle.

Now, that's probably not going to hold true for all WebGL apps, so there's a couple of pointers to keep in mind if you are trying to implement visibility culling for your project:

Don't break your geometry up for the sake of culling! Unless you are dealing with many millions of polygons you will be very hard pressed to gain any benefits from doing so. If you do cull, do so based on state: If you can determine that no geometry using a certain shader will be visible then skip that shader entirely, but otherwise try to draw all geometry using a specific shader in one call.
Only calculate visibility when you must. For example, when I was still trying to do some culling I would only recalculate the visible shaders if I detected that I had changed nodes in the BSP tree. Otherwise there was no chance that the visible set would have changed, and recalculating it per-frame would just be wasted cycles.
If possible, offload your visibility calculations to a web worker. Anything that can avoid blocking the UI (render) thread is a good thing, and most people will be pretty forgiving if a small chunk of a wall isn't visible for a fraction of a second after turning a corner.

Speaking of web workers, it's worth noting that all of the level loading and pre-processing takes place in a worker thread in the demo. This allows the page to remain responsive while the map loads, which is a problem that plagues a lot of WebGL demos. It does take a bit more planning to break up the work in this way, but in the end it makes a big difference in how your app will be perceived. There are parts of this that could be improved with better browser support (For example, allowing typed arrays in a web worker would be beautiful) but even without those niceties the performance is noticeably better. In fact I would like to move more logic to worker threads, such as the collision detection, but just didn't have the time.

Oh, and on the subject of collision detection: there really isn't much to say. A lot of my code is just a tweaked version of the Quake 3 movement code (it's about the only part of the demo that could legitimately be called a "port") because there's not much more that could be done in the way of optimization. Don't fix what isn't broken, I guess. The quirks of web-based controls cause the movement to be a bit more unstable than I would like, but overall it gets the job done.

The other big thing I wanted to talk about was shaders, and it's probably the part that will be the most interesting to the graphically minded amongst you. When Quake 3 was launched the cutting edge cards of the day were all fixed function with, if you were lucky, cool new features like multitexture and register combiners! *Oooooh!* The Quake 3 engine (Or, idTech 3 I guess) was built with this in mind and features a "shader" system that was targeted at the capabilities of the day. Consisting of a plain text set of instructions about things like blending functions, texture coordinate manipulations, and multiple render stages it's a primitive system by today's standards but is amazingly robust in the effects that it can produce.

It's also a reasonably complicated format, with a lot of keywords to track and a bunch of variables that can look fairly obscure until you go digging through the source to find out what they mean. For that reason almost every Quake 3 loader I've seen outside of some commercial ones simply ignore the shaders altogether (including CopperLicht, BTW). And since most of the time the shader names also correspond to a texture name this works pretty well and you can get about 80-90% of you level to look right. That last 10-20% stands out pretty badly, though, and since I was determined to capture that Quake feeling as I mentioned before I couldn't avoid them. In the end I'm very glad I didn't because it makes a huge difference on how people perceive the level as they walk through it.

Initially I was reading in the shaders and translating them into a series of state and uniform changes that were fed into a single global GL shader. This worked for a while, but as the number of shader keywords I was processing increased and I encountered more and more special cases it quickly became apparent that that approach was going to become unworkable. In the end I switched the whole system over so that each stage is now compiled into a custom GL shader that handles all of the texcoord transforms, color modulation, and other effects that don't involve setting GL state. This gives me a couple of benefits in that the shaders are now very flexible and the state I have to bind for each stage is pretty minimal. The code took a bit of tweaking to get right, but I'm very happy with how it ended up. The code to do this is in http://media.tojicode.com/q3bsp/q3shader.js which is used by the worker thread, and http://media.tojicode.com/q3bsp/glq3shader.js which is used by the render thread.

Now if anything I do feel that this is the biggest area for potential improvement in the demo. Right now I'm creating a new GL shader program for each stage of each Quake 3 shader I process. This could probably be reduced quite a bit if I was: A) checking for duplicate stages and re-using the same shader and B) checking to see if multiple stages could be collapsed into one multitextured stage. There's also the fact that although at this point I am able to render most of the game's shaders correctly there are still some parts of the shader format that I'm not handling, and a couple of parts that I could be handling better. For example: I'm not processing alphaGen lightingSpecular properly at all, and I'm not doing anything to handle volumetric fog (which is probably the demo level's biggest omission.) But while there's room to grow I think the basic framework laid down is pretty solid, and I'm happy with it.

The last think I wanted to touch on is that I did make some changes to the original resource files in order to make the demo work online. It was my goal to alter as little as possible about the game files in creating the demo, and I feel like I largely succeeded in this area. I few concessions I did make to the browser are pretty small in the end:

All texture files (JPG and TGA) were converted to PNGs. Obviously this worked out well for the TGAs (which are basically used any time the artists wanted an alpha channel), but in the case of the JPGs it actually increased the filesizes, which wasn't exactly the happiest thing. Considering that WebGL will use a JPG image as a texture just fine this may seem like an odd choice, but it was primarily done to avoid multiple hits against the server. The problem is that Quake never specifies the format of the image it's trying to load. It either has no extension or a "default" extension of TGA, and internally the engine will attempt to open a TGA version and if that fails it will fallback to a JPG. That's all well and good, but that "fail and fallback" process gets a whole lot longer and uglier when the texture file is sitting on a server at the other end of the country somewhere. It certainly would be possible with a few server hacks to allow the engine to request a TGA and receive a JPG back, which may in the end be a better solution. I wanted to have my demo code be 100% client side, however, so the unifying the texture formats was the better plan.
Some textures were resized to ensure their dimensions were powers of 2.
For any surfaces that have some sort of special effect the engine parses and pulls the effect definition from one of several shader files, each of which describes multiple shaders. The map files don't tell you which shader file the one they're looking for is in, though, and the game simply loads all of them into memory on startup. Once again, this isn't exactly optimal when talking about a web environment so when I was ready to post the demo I hunted down all of the shaders the map I chose used and compiled them by hand into a single shader file. The shaders themselves were not changed, and it's worth noting that demo code has the ability to load and run with multiple shader files (if you look at the source you can see the list of shaders I used to develop locally commented out). In this case it's just much more efficient to have the user download a single file which they will use all of instead of multiple files that they may use part or none of.

And that's about it! Everything else is pulled straight from the PAK files. That means that aside from some minor image conversions (which many programs can do in batches) everything can be dropped in from presumably any Quake 3 level and it will just "go".

So that's my brain dump on the subject, hopefully someone out there finds it at least mildly interesting! This demo has been another great learning experience for me, and has given me some new subjects that I want to investigate in the future. Hopefully the next demo won't take as long as this one did!

Rendering Quake 3 maps with WebGL: Demo

2010-08-09T08:13:00.002-06:00

So, after a good long delay it's finally WebGL demo time again! This one is pretty elaborate, but I feel like it was worth the effort:

WebGL Quake 3 - Q3TOURNEY2

A quick note: I've heard reports that Chrome 5 has trouble with the demo because I'm using the newer texImage2d signature. Your mileage may vary, but if you're having trouble give it a try with the Chrome dev channel.

Now, there are SOOO many things to talk about in regards to this demo: Optimizations, remaining issues, changes made to resources for web use, etc. Unfortunately I don't have the time to elaborate on them right now but I wanted to get the demo out there ASAP to get some feedback on it. Expect another post (or maybe an addendum to this one) within the next couple of days about my thoughts on Quake 3 bsp maps under WebGL.

(UPDATE: My ramblings about the demo are online now)

Have fun!

EDIT: Quick update. I've tweaked the code a little bit to reflect gero3's suggestion of combining the multiple event loops into one. It worked pretty well, and the movement in particular feels better for it (well, not the mouselook. Not much I can do there though). Not to mention the code now reflects a "traditional" game loop much more.

Droid X: First Week Impressions

2010-08-04T22:19:00.001-06:00

Things have been a bit quiet on the blog lately, but not for lack of activity on my part! I've been working on a fairly involved WebGL demo that I'm hoping to post soon (I'd like to get it out by Aug. 12...) that I feel is pretty dang awesome. Hopefully the rest of the interwebs will feel the same. :)

For the moment, though, I want to talk about something non-WebGL. I bought a new phone last week after my EnV Touch started randomly powering off... again. (For the record in the last two years I have cycled through 6 EnV phones: 3 EnV 2s and 3 EnV Touches. ALL of them had power issues within about 4 months or so of receiving a replacement phone. Don't get phones from the EnV line. Ever.) This time I decided to make the jump to smartphone land, and since Apple has yet to dump AT&T (and since I like to, you know, HOLD my phone) I went with the Android powered Droid X. I will admit that it was a close race between that and the HTC Incredible, though. Better battery life on the X won me over in the end.

Well, I've had it for about a week now and thought I would do a brain dump of my initial impressions of the device. Please keep in mind that this is the first Android device that I've owned or even used extensively, and as such I'm not 100% certain of where the line is between stock Android and the Droid X's software.

So, first things first: I absolutely LOVE Android. There are bits of it that are clunkier that I would like, and there are still some things that I'm trying to figure out how to do (can I get a contact link with photo on my home screen? Please?) but overall it's been a joy to use and become familiar with. Everything is super snappy too (once the default widgets are removed, more on that later), and I can jump around from app to app without a second thought. I very rarely, if ever, run into lag of any sort and if I do it's usually the result of something that I'm doing that I probably shouldn't be. (For the record, running Pandora, the stock music player, and the YouTube app all at once is just asking for stutters!) I have a friend who is very much in love with his iPhone, and he insists that menu transitions and the like are smoother on Apple's device but I'm pretty hard pressed to see the difference. Even if the iPhone does out-transition the Droid X I'm confident that the performance here will be good enough for pretty much anyone.

Aside from the processor, the rest of the hardware is pretty awesome too. Obviously the screen is the main attraction here, a whopping 4.3 inches. It's stunning to look at, and very crisp. I could easily see myself watching movies on it next time I fly somewhere. It's worth mentioning, however, that this phone sits somewhere between "very large" and "obscenely large" on the size scale, and which way that needle tips depends entirely on the user. I'm a pretty big guy with pretty big hands, and this phone is probably at the very limits of what I could use comfortably. My wife, on the other hand, is somewhat overwhelmed by the massiveness of the thing. I doubt she could ever use it as her personal phone, and would probably be far more comfortable with an Incredible. Otherwise it's slim form factor lets it fit well in my jeans and it doesn't weigh any more than my previous phone so the size isn't an issue for me. Your mileage may vary. Battery life is quite good too, and with a bit of smart management (no GPS if I'm not using it, etc) I can easily have Pandora running almost non-stop during my work day (~8hrs) before needing a charge. My bosses Incredible, by comparison, gets about half that.

I was less than enthused about the camera and buttons. The camera's pictures and video are really just on par with most other phones I've used, and that's not a good thing. It's sad, really, that a phone which proudly advertises the ability to take 720p video can't deliver image quality worthy of the resolution. It's not a deal breaker for me (I expect all phone cameras to be terrible anyway) but it's definitely an area that's lacking. The button are similarly lackluster. the volume rocker and unlock buttons are nice and solid, but the rest of them feel rather cheap. They wiggle around a bit and don't really seem to mesh with the rest of the otherwise very solid construction. I would have preferred touch sensitive buttons like the original Droid for the front but that's just me. It also would have been nice to have a trackball of some sort, but I haven't found that particular omission to be a problem so far (the huge screen usually makes it easy enough to hit what I want.) In the end these few bumps aren't anything that makes me want to trade it in.

The real frustration comes from Motorola's software. Pre-loaded on the phone are a bunch of MotoBlur-themed widgets covering everything from contacts to Twitter and, to put it as kindly as possible, they suck. They take up a ton of screen real estate, don't mesh with the rest of the phone's look and feel, constantly try to pull down data, and cannot be uninstalled. On top of all that, they're slow as dirt. Upon first firing up the phone and flipping between home screens there was a very noticeable stutter to any animation, and interacting with any of the moto-widgets was usually accompanied by a pause. This is the kind of performance you get from a Gigahertz processor? The saving grace here is that you can remove all of their custom crap from the home screens and use just the default Android widgets, at which point all of the performance issues disappear.

That doesn't completely clear them out of the phone, though. As I said, these custom widgets and apps can't be removed, only taken off of the home screen. As a result you get a lot of little annoyances like the fact that if I want to use the (superior) Google Twitter app I will forever more have two nearly identical Twitter icons in my Accounts page, forcing me to guess which one I really want to use. Also, services like "Skype mobile" and a Social Networking "FeedReceiverService" start up with the phone every time, whether you want them to or not. For the most part it's easily ignored but the idea that my Android phone, a system build on the idea of openness, refuses to allow me to remove the things I don't want grates on me. To add insult to injury on top of all that, Motorola has a bootloader protection mechanism installed on the phone (eFuse) that will lock it up if you try and modify the preinstalled OS. They must really really REALLY want you to use their ugly Facebook widget!

Now, I get that they want to brand the phone somehow and make it their own (cause the big "MOTOROLA" stamped across the top apparently just isn't enough) but is this seriously how they want customers to experience their powerhouse hardware? Stuttering and lagging and draining the life out of the battery? It's kinda like trying to show of your new Ferrari by driving it through a speed bump infested parking lot. It baffles me, honestly.

Now that's a lot of griping over an aspect of the phone that I ignore 99% of the time, but when everything else about the device is so stellar the clumsy Motorola software stands out like a sore thumb. In the end I'm keeping the phone and am thrilled with it, but I'm also crossing my fingers and wishing with all my might that the upcoming updated to Android 2.2 lets me clear off some of the crud.

Final Verdict:
Hardware - 9.8/10 - Cheap buttons mar an otherwise solid package. Might be too big for smaller hands.
Software - 7/10 - Android is awesome but everything that Motorola has added to it fails miserably. Extra points taken off for forcing me to keep said garbage on the phone.

More pardonings of continued dust, if you please...

2010-07-15T21:45:00.002-06:00

I'm going to be doing some more server shuffling over the next day or so, so I apologize if things are a bit up and down again. I'm working on it!

EDIT: Sorry for the downtime. I'm having some annoying issues getting my DNS entries sorted. In the meantime I've got tojicode.com redirecting to my blogspot address so everyone should at least be able to continue seeing the blog while I try and get a more elegant solution going.

EDITING THE EDIT: Alright, so it looks like the current configuration is stable so I'm sticking to it. blog.tojicode.com is going to be the official address, and for the time being tojicode.com will redirect to it.