Quake 2 BSP - Quite possibly the worst format for WebGL ever

So, as hinted at by my last post lately I've been working on a Quake 2 map viewer for WebGL. I'm not as far along with it as I would have liked, but I've decided to stop developing it and move on to other formats/pursuits for various reasons that I'll get into in a bit. I'm sure the majority of you are here just to see the pretty demo, though, so without further ado:

Quake 2 BSP WebGL Demo

(Linkified because trying to scrunch the viewport into the blog would make it difficult to use.)

It's certainly not perfect, but hopefully you guys like it. I would be thrilled to see someone expand on what I've done here... oh wait.

So, let's talk about why I've decided to move on from this project. This bit is gonna get rather long and technical so feel free to ignore it if subjects like binary parsing and lightmap calculations make your eyes glaze over.

Coming up...

So I've been a bit quiet on the blog lately, primarily because my next WebGL plaything is a bigger project than the others I've taken on so far and has undergone a couple of false starts. Just to prove I'm still working on something, though, here's a teaser:

Not sure when I'll be at a point where I want to post the live version. It's pretty sloppy at the moment. Should be a pretty cool demo when it's done, though!

In other news, I've noticed lately that my Spore and MD5 demo's don't seem to be rendering correctly for me when I view them on this site. It seems as though the normal maps aren't being applied (makes it look like the light is always behind the model). I'm not sure what that is, but I suspect that it's been a change in Chrome recently. Is anyone else seeing this?

glMatrix 0.9.2 Released

glMatrix 0.9.2 has just been released. Included are even more optimizations, some new functions (like mat3.toMat4 and mat4.toMat3), and a smattering of quaternion code to boot! (Developed for use in my md5 loader).

If you've been using the library, or simply want stupidly fast javascript matrices, be sure to grab the latest version!

It's Alive!!! idTech4 models with skinning.

So, not content to just show a static Spore creature I decided that my next project in WebGL was going to be something a bit more dynamic. After a little bit of toying around, I decided to give the MD5 mesh format (Doom 3, Quake 4, etc.) a try. I think it turned out pretty well...

Click here to run

(I hope Mr. Carmack and company don't mind me using one of their models to demo with. If anyone should take issue with it I'll remove that model and swap it out for another, but you've got to admit that seeing a Hellknight chilling out in a browser is pretty cool!)

[EDIT: Interleaved arrays have been fixed in the latest Minefield releases, which allows the demo to run correctly.]
As of right now I've confirmed that the above demo runs with Chrome, but not Minefield. I'm pretty certain it's because I'm using interleaved arrays, which is still horribly broken in Mozillas implementation last I heard. Your mileage may vary, and in any case I should have a more compatible version up in the next day or so.

I've worked with the format before, so I knew how it was laid out reasonably well and (crucially) I knew that the format was plain text. One of the biggest drawbacks to WebGL (and web development in general) is a near complete lack of ways to efficiently work with binary data outside of a select few media types. I must say, though, even given that the files were text based the actual parsing was far easier and faster than I expected. Certainly the file size and loading speed blows COLLADA out of the water. Guess that just a happy side effect of using a format intended for realtime rendering!

Of course, there are a few things that I know could be done better/faster that I might want to tackle in the future (and would love to hear about if anyone else improves on my methods!):

• Material loading is a huge, ugly hack right now. Problem is that I can't figure out a happy method of loading the actual material definitions used by the game in a web-friendly way. So for the time being the loader simply looks for 'shader-name.png', 'shader-name_s.png', and 'shader-name_local.png' for the diffuse, specular, and normal maps respectively. It also ignores any special effects (hence the odd looking drool)
• Skinning is done on the CPU right now, but I would like to try GPU skinning and see if there are any performance gains. In all honesty, though, the format isn't intended for GPU skinning and I'm quite pleased with the speed of the javascript based animations, so I'm not sure how necessary that is.
• I'm not interpolating between frames at the moment. Playback is at 24 FPS, defined by the animation file. Certainly it would be nice to smooth that out a bit but I'm worried that the extra overhead would murder the framerates. It's worth playing with though.
• I'm not 100% confident in my skinning methods of the normals and tangents. The results look correct enough, but may fall apart with more extreme movement. Also, I'm not taking mirrored uv's into account when calculating tangents (I think the models are built to account for this, but I could be wrong) and I'm ignoring texture seams when calculating the normals which leads to some sharper-than-I-would-like normal differences.
• I'm hauling around a lot of unnecessary information after the initial load, and as such memory use could be much better.
• Animating two independent instances of a mesh would require some changes as well, but wouldn't be too difficult.

Aside from those complaints, though, the library is reasonably fast and robust and should work with most of the MD5 meshes out there (something that certainly cannot be said about my COLLADA loader). It requires my glMatrix library, but is otherwise standalone.

I'd love to hear any comments or suggestions for improvement, and if anyone else builds something cool on top of the loader please let me know!

Pardon our dust...

I'm in the process of shuffling servers and changing domain names right now, so I apologize if anything weird happens over the next day or so. Everything should be fine soon, though!

Stupidly fast WebGL Matrices

I've made a matrix library for WebGL. It's fast. Really fast. It's linked right here: glMatrix
[EDIT: And just been updated! Now even faster!]

And, lest some scoff at my claims of fastness, I've also made a benchmark: glMatrix Benchmark

Don't bother with that last link if you're browser doesn't have WebGL enabled. None of the comparison libraries really work without it. (Though glMatrix will function with any javascript sequence.)

Okay, so let's get a little more serious. Why in the world would I want to make yet another matrix library when there are already a good selection of them?  That's a tough one, especially because I'm a very firm believer in not reinventing any wheels. What it really comes down to is this: I tried all the other matrix libraries I could find and didn't feel like any of them were meeting my needs.

• Sylvester is a very nice and robust library that I've seen some people using with WebGL, and I can't really blame them. It's probably the most complete one that I've seen, and is aimed at people doing heavy duty math in their browser. The biggest problem with it is simply that Sylvester was built for robustness, not speed, and as such is ill suited for realtime graphics.
• Khronos recommends using CanvasMatrix in their introductory tutorial. It's apparently from Apple (according to the comment at the top), and has a nice interface, but once again isn't really built with an eye towards performance.
• mjs was built to be far more speed conscious, and it shows. Done by the guy who did the original Spore WebGL demo, it's a nice little library and it's what I've been using so far. I only have a few complaints with it: I'm not a huge fan of the syntax ('M4X4' is awkward to type over and over again), and you end up creating a lot of temporary matrices to store intermediate values in (passing a single matrix as both the source and result matrix can corrupt the value). There's also some odd bits of missing functionality (like a full, generic inverse) that may or may not be a problem. Still, it's very workable.
• EWGL Matrices appear to be the latest of the bunch, and the idea behind them was very specifically to be extremely fast. To that end they did a great job, but there were still a couple of things that I was annoyed by. All operations take place on the source matrix, which can lead to some unnecessary duplications in certain scenarios. (Basically the inverse of mjs's problem) And it's object oriented nature tends to force you to use, for example, their vector types when passing translation or axis data, which I find somewhat cumbersome.

Now, all of these complaints are, in reality, quite petty. If that were the extent of it I'd just use one of the latter two and be done with it. But there was one feature that was missing from all of them (except Sylvester, which doesn't count for speed reasons) that killed it for me: none of them had facilities to multiply a vector by a matrix! To me that seems like a rather obvious one, because while certainly in an environment like WebGL we want to let the GPU do as much work as possible there's some times where you simply want to rotate a point in memory.

So, of course, noticing a small deficiency in the existing options I did what any normal programmer would do: Wrote my own. :) 

There were a few key things I was aiming for when I wrote it:

• The interface needed to be clean and consistent
• Operations needed to be able to happen in-place OR written out to a destination array, leaving the original untouched.
• The library should not lock you into using a certain type or set of classes. (ie: It should work just as easily with Arrays as WebGLFloatArrays)
• And for crying out loud it needs to be able to transform vectors!

Believe it or not, speed was a completely secondary concern while building it. Once I got all my desired functionality in there and started doing some benchmarks, however, I realized that I wasn't too far off from the faster existing libraries. So I spent another evening optimizing the crap out of it to make my little matrix library "stupidly fast". A few key optimizations were:

• Unroll EVERYTHING. There's not a single loop in the code.
• Take advantage of javascript's variable caching. If a matrix element was read more than once in any function it was stored in a local variable first. I was honestly surprised at just how much of a difference this one made!
• Inline anything that made sense. ie: Although I have a vector normalize function, I did an inlined version in the matrix inverse to cut down on call overhead and additional variable creation.
• Take shortcuts when possible! If the source and destination matrix for a transpose are the same, we don't need to alter the elements on the diagonal. Or when rotating if we notice that it's along the X, Y, or Z axis we can cut out a lot of calculations that would just end up as 0 anyway.

And the end result is that glMatrix outperforms even EWGL in every scenario I've tested! No small feat!

Of course, "stupidly fast" is probably pushing it. After all, this is Javascript we're talking about. Even the most naive of C matrix libs would run circles around the best javascript libs. But when it comes to WebGL we really don't have much of a choice now do we? At the very least I feel confident in saying that this library is one of the fastest (if not THE fastest) javascript matrix libs available today.

This being a first release, I fully expect a few bugs here and there and welcome any feedback on how to improve things. But hopefully it can serve as a meaningful contribution to the WebGL development scene.

Happy coding!