Friday, May 6, 2011

Interleaved array basics

I got a question from Jon in the comments on the WebGL sandbox project yesterday:

Hi Brandon, Using your demo as a starting point, I try to display a pyramid, but this far I've only been able to see one of its four faces. If I use gl.LINES instead of gl.TRIANGLES, then I see only one triangle (i.e. on face). I'm also a bit confused by the way you mix texture coordinates into the vertArray. Can you explain how these coordinates get sorted out in the shader?

Honestly I don't know if I'm the best guy in the world to be explaining this, but I'll give it a try, since there seems to be remarkably little WebGL-specific information about this. Most tutorials prefer to keep the arrays separate for simplicity, but that's not optimal for performance. The concepts all work pretty much the same as OpenGL, but it's nice to see them put to use in the environment you are using. It requires some basic understanding of subjects that don't normally apply to Javascript, like counting bytes, but it's not too hard once you get the hang of it.

The first thing to wrap your head around is that the WebGL API doesn't know anything about vertices or texture coordinates or normals or anything else like that. What it does know about is lists of numbers. It knows how to save lists of numbers in your graphics memory, and it knows how to tell the shaders to start rendering those numbers, but really that's about it. YOU are the one that tells the system what the numbers mean, via the shader code and gl.vertexAttribPointer().

var values = [
    -1.0, -1.0, 0.0, // Vertex 1
    0.0, 1.0, 0.0, // Vertex 2
    1.0, -1.0, 0.0 // Vertex 3

You've probably all seen that before, right? The thing to notice this time is that aside from the comments and the way that we've structured our line breaks there's nothing that separates the vertices. That line can just as accurately be written like this:

var values = [ -1.0, -1.0, 0.0, 0.0, 1.0, 0.0, 1.0, -1.0, 0.0 ];

Of course, it's a lot harder to read for us human beings now, but to the computer it's exactly the same. In either case we push the data into a vertex buffer like so:

var vertBuffer = gl.createBuffer();
gl.bindBuffer(gl.ARRAY_BUFFER, vertBuffer);
gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(values), gl.STATIC_DRAW);

Take note of the Float32Array bit, we'll be talking about it later. End result of this, though, is that vertBuffer now points to an array of numbers somewhere on the graphics card. So how does the computer know how to interpret these seemingly random values? gl.vertexAttribPointer!

Before we render, we have to call code like this:

gl.bindBuffer(gl.ARRAY_BUFFER, vertBuffer);
gl.vertexAttribPointer(shader_attrib_position, 3, gl.FLOAT, false, 12, 0);

This tells the system everything it needs to know about how to read your vertices. (Well, more accurately how to tell your shader about the vertices...) There are 4 points that we want to pay attention to, here: The "3" tells WebGL how many values make up each point. Since a position is defined by x, y, and z coordinates we tell it "3". If you were doing 2D rendering in your WebGL code you could easily pass a 2 in here, and you're allowed to give it anything from 1 to 4 depending on the situation. Next, we tell it what kind of numbers we're passing. In this case they are floating point, so we pass in gl.FLOAT. There's also gl.BYTE and gl.SHORT (and others, but let's not overcomplicate things). This tells WebGL two very important things: First, how to expose the values to the shader, and secondly how big each value is. Now that's important, and for those of you who aren't familiar with byte packing let's do some quick review:

  • gl.BYTE = 1 byte (obviously!), can hold values from -127 to 127
  • gl.SHORT = 2 bytes, value range of -32767 to 32767
  • gl.FLOAT = 4 bytes, value range of a whole lot.

For the most part, unless you have some VERY specific needs, you'll want to be sticking with gl.FLOAT. Using Float here also lines up with our use of Float32Array when creating the buffer. If we were to use one of the other types you would want to create the buffer with a different array type.

The important thing to take away from that is that according to our vertexAttribPointer call above we are passing in 3 Floats. 3 * 4 = 12, so each position takes up 12 bytes. And, hey! We have a 12 in that call too! That's called the "stride", and it tells the system how far apart (in bytes) the start of each vertex is. This is very useful if our vertex contains more information than just a position. Lets say that for some reason our vertices were defined like this instead:

var values = [
    -1.0, -1.0, 0.0, 999.0, // Vertex 1
    0.0, 1.0, 0.0, 999.0, // Vertex 2
    1.0, -1.0, 0.0, 999.0 // Vertex 3

Those 999's might represent some other part of the vertex data, but they don't have anything to do with position. If we continued using the same vertexAttribPointer values from above, our vertexes would look like this:

[-1, -1, 0] [999, 0, 1] [0, 999, 1] [-1, 0, 999]

We can easily "ignore" that 4th value, and hence go back to our desired vertex positions, like so:

gl.vertexAttribPointer(shader_attrib_position, 3, gl.FLOAT, false, 16, 0);

Notice that we are still instructing WebGL to read 3 floats, but the stride has changed to 16 to account for the 4th, unneeded float. (4 values * 4 bytes per value = 16 bytes).

Finally, we have that 0 at the end, which tells WebGL how far into the array to start reading values. This is also given in bytes, and in this case is 0 because we want to start at the first value. If we wanted to start, say, on the second number we would do this:

gl.vertexAttribPointer(shader_attrib_position, 3, gl.FLOAT, false, 16, 4);

Now our vertex positions would look like this:

[-1, 0, 999] [1, 0, 999] [-1, 0, 999]

This is very useful for packing multiple meshes into a single buffer, since we can start rendering at any point, but it's also useful for interleaving data. "Interleaved" means that you have multiple types of data packed into a single array, like position and texture coordinate data. Most tutorials will show them as separate arrays like so:

var pos = [1, 1, 1, 2, 2, 2, 3, 3, 3]; // (x, y, z), (x, y, z)...
var tex = [0, 1, 0, 1, 0, 1]; // (s, t), (s, t)...

But it's often times more efficient and easier to have them all as one array, like so:

var verts = [
    1, 1, 1, 0, 1, // (x, y, z), (s, t)...
    2, 2, 2, 0, 1,
    3, 3, 3, 0, 1,

var vertBuffer = gl.createBuffer();
gl.bindBuffer(gl.ARRAY_BUFFER, vertBuffer);
gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(verts), gl.STATIC_DRAW);

When we set up the vertexPointers for this array, we need to make two calls to vertexAttribPointer.

gl.bindBuffer(gl.ARRAY_BUFFER, vertBuffer);
gl.vertexAttribPointer(shader_attrib_position, 3, gl.FLOAT, false, 20, 0);
gl.vertexAttribPointer(shader_attrib_texcoord, 2, gl.FLOAT, false, 20, 12);

Let's examine what that means: The first attribute (our position), is comprised of 3 floats spaced 20 bytes apart, starting at byte 0. The second attribute (our texcoord), is comprised of 2 floats spaced 20 bytes apart, starting at byte 12 (or 3 floats in). And that's it! Two attribute types in one buffer! And we can do this for as many attributes as we need (within reason). It's not uncommon to have a single buffer that contains position, texcoords, normals, binormals, and vertex weights in it!

I sincerely hope that all made sense, and wasn't just a lot of graphics gibberish. I haven't talked about the shader side of things at all here, but that's something that other tutorials cover pretty well. Good luck, and happy WebGL-ing!