Friday, May 6, 2011

Interleaved array basics

I got a question from Jon in the comments on the WebGL sandbox project yesterday:

Hi Brandon, Using your demo as a starting point, I try to display a pyramid, but this far I've only been able to see one of its four faces. If I use gl.LINES instead of gl.TRIANGLES, then I see only one triangle (i.e. on face). I'm also a bit confused by the way you mix texture coordinates into the vertArray. Can you explain how these coordinates get sorted out in the shader?

Honestly I don't know if I'm the best guy in the world to be explaining this, but I'll give it a try, since there seems to be remarkably little WebGL-specific information about this. Most tutorials prefer to keep the arrays separate for simplicity, but that's not optimal for performance. The concepts all work pretty much the same as OpenGL, but it's nice to see them put to use in the environment you are using. It requires some basic understanding of subjects that don't normally apply to Javascript, like counting bytes, but it's not too hard once you get the hang of it.



The first thing to wrap your head around is that the WebGL API doesn't know anything about vertices or texture coordinates or normals or anything else like that. What it does know about is lists of numbers. It knows how to save lists of numbers in your graphics memory, and it knows how to tell the shaders to start rendering those numbers, but really that's about it. YOU are the one that tells the system what the numbers mean, via the shader code and gl.vertexAttribPointer().

var values = [
    -1.0, -1.0, 0.0, // Vertex 1
    0.0, 1.0, 0.0, // Vertex 2
    1.0, -1.0, 0.0 // Vertex 3
];

You've probably all seen that before, right? The thing to notice this time is that aside from the comments and the way that we've structured our line breaks there's nothing that separates the vertices. That line can just as accurately be written like this:

var values = [ -1.0, -1.0, 0.0, 0.0, 1.0, 0.0, 1.0, -1.0, 0.0 ];

Of course, it's a lot harder to read for us human beings now, but to the computer it's exactly the same. In either case we push the data into a vertex buffer like so:

var vertBuffer = gl.createBuffer();
gl.bindBuffer(gl.ARRAY_BUFFER, vertBuffer);
gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(values), gl.STATIC_DRAW);

Take note of the Float32Array bit, we'll be talking about it later. End result of this, though, is that vertBuffer now points to an array of numbers somewhere on the graphics card. So how does the computer know how to interpret these seemingly random values? gl.vertexAttribPointer!

Before we render, we have to call code like this:

gl.bindBuffer(gl.ARRAY_BUFFER, vertBuffer);
gl.vertexAttribPointer(shader_attrib_position, 3, gl.FLOAT, false, 12, 0);

This tells the system everything it needs to know about how to read your vertices. (Well, more accurately how to tell your shader about the vertices...) There are 4 points that we want to pay attention to, here: The "3" tells WebGL how many values make up each point. Since a position is defined by x, y, and z coordinates we tell it "3". If you were doing 2D rendering in your WebGL code you could easily pass a 2 in here, and you're allowed to give it anything from 1 to 4 depending on the situation. Next, we tell it what kind of numbers we're passing. In this case they are floating point, so we pass in gl.FLOAT. There's also gl.BYTE and gl.SHORT (and others, but let's not overcomplicate things). This tells WebGL two very important things: First, how to expose the values to the shader, and secondly how big each value is. Now that's important, and for those of you who aren't familiar with byte packing let's do some quick review:

  • gl.BYTE = 1 byte (obviously!), can hold values from -127 to 127
  • gl.SHORT = 2 bytes, value range of -32767 to 32767
  • gl.FLOAT = 4 bytes, value range of a whole lot.

For the most part, unless you have some VERY specific needs, you'll want to be sticking with gl.FLOAT. Using Float here also lines up with our use of Float32Array when creating the buffer. If we were to use one of the other types you would want to create the buffer with a different array type.

The important thing to take away from that is that according to our vertexAttribPointer call above we are passing in 3 Floats. 3 * 4 = 12, so each position takes up 12 bytes. And, hey! We have a 12 in that call too! That's called the "stride", and it tells the system how far apart (in bytes) the start of each vertex is. This is very useful if our vertex contains more information than just a position. Lets say that for some reason our vertices were defined like this instead:

var values = [
    -1.0, -1.0, 0.0, 999.0, // Vertex 1
    0.0, 1.0, 0.0, 999.0, // Vertex 2
    1.0, -1.0, 0.0, 999.0 // Vertex 3
];

Those 999's might represent some other part of the vertex data, but they don't have anything to do with position. If we continued using the same vertexAttribPointer values from above, our vertexes would look like this:

[-1, -1, 0] [999, 0, 1] [0, 999, 1] [-1, 0, 999]

We can easily "ignore" that 4th value, and hence go back to our desired vertex positions, like so:

gl.vertexAttribPointer(shader_attrib_position, 3, gl.FLOAT, false, 16, 0);

Notice that we are still instructing WebGL to read 3 floats, but the stride has changed to 16 to account for the 4th, unneeded float. (4 values * 4 bytes per value = 16 bytes).

Finally, we have that 0 at the end, which tells WebGL how far into the array to start reading values. This is also given in bytes, and in this case is 0 because we want to start at the first value. If we wanted to start, say, on the second number we would do this:

gl.vertexAttribPointer(shader_attrib_position, 3, gl.FLOAT, false, 16, 4);

Now our vertex positions would look like this:

[-1, 0, 999] [1, 0, 999] [-1, 0, 999]

This is very useful for packing multiple meshes into a single buffer, since we can start rendering at any point, but it's also useful for interleaving data. "Interleaved" means that you have multiple types of data packed into a single array, like position and texture coordinate data. Most tutorials will show them as separate arrays like so:

var pos = [1, 1, 1, 2, 2, 2, 3, 3, 3]; // (x, y, z), (x, y, z)...
var tex = [0, 1, 0, 1, 0, 1]; // (s, t), (s, t)...

But it's often times more efficient and easier to have them all as one array, like so:

var verts = [
    1, 1, 1, 0, 1, // (x, y, z), (s, t)...
    2, 2, 2, 0, 1,
    3, 3, 3, 0, 1,
];

var vertBuffer = gl.createBuffer();
gl.bindBuffer(gl.ARRAY_BUFFER, vertBuffer);
gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(verts), gl.STATIC_DRAW);

When we set up the vertexPointers for this array, we need to make two calls to vertexAttribPointer.

gl.bindBuffer(gl.ARRAY_BUFFER, vertBuffer);
gl.vertexAttribPointer(shader_attrib_position, 3, gl.FLOAT, false, 20, 0);
gl.vertexAttribPointer(shader_attrib_texcoord, 2, gl.FLOAT, false, 20, 12);

Let's examine what that means: The first attribute (our position), is comprised of 3 floats spaced 20 bytes apart, starting at byte 0. The second attribute (our texcoord), is comprised of 2 floats spaced 20 bytes apart, starting at byte 12 (or 3 floats in). And that's it! Two attribute types in one buffer! And we can do this for as many attributes as we need (within reason). It's not uncommon to have a single buffer that contains position, texcoords, normals, binormals, and vertex weights in it!

I sincerely hope that all made sense, and wasn't just a lot of graphics gibberish. I haven't talked about the shader side of things at all here, but that's something that other tutorials cover pretty well. Good luck, and happy WebGL-ing!

15 comments:

  1. Wow, I just realized how badly Blogger's "Compose" view messed up the formatting on this. I'll try and fix that soon. Sorry!

    ReplyDelete
  2. Okay, rebuilt the html for the post by hand. Still not pretty but at least it's consistent. It's astounding to me how much cruft Blogger crams into your post when using compose mode.

    ReplyDelete
  3. Hi Brandon,
    Thank you for a very good explanation of interleaved arrays. I would probably have taken me quite a while to figure out this. ;-)
    It seems my pyramid model may not be too bad; I've managed to catch a glimpse of two faces together. If I could only find out how to rotate it ... I would also like to display several identical pyramids together, just rotated differently. Could you give me a hint?
    You've said that most of the formats you work with are designed with Z as the "up" axis. Where in your demo is that decision coded?

    ReplyDelete
  4. Nice article, most blogs/tutorials don't mention those things, i am very excited to implement it in my own engine (in progress).

    When you have a animated mesh with 1000 frames would it then still be handy to interleave ALL frames of the mesh?

    And how do you tell it where to stop?
    Lets say i join 3 meshes with all same amount of points, how does webgl know that when i start from 0 he should stop on a third of all points?

    ReplyDelete
  5. Sorry, that would be a fairly important part, wouldn't it?

    When you call either drawArrays or drawElements you give it the number of vertices to render. It will process that many beets and then stop. So if you have 300 verts in your buffer and only want to render the first third of them, tell your draw call to render q
    100 verts.

    ReplyDelete
  6. offcourse ;) stupid of me not to look there.
    Thanks for the quick response.

    ReplyDelete
  7. Thanks for the clear explanation. My stumbling block was offset and stride are in bytes, not items. I keep thinking that WebGL will do (size * offset) and (size * stride) for me.

    ReplyDelete
  8. This post makes sense, thanks!
    I have some questions regarding it:
    I don't see how opengl sees what buffers to iterate through, especially if you have multiple ones, for example in a draw method:
    (in comment what I understand to happen)

    // draw object1

    gl.bindBuffer(gl.ARRAY_BUFFER, obj1PosBuffer); // selects obj1PosBuffer as current array buffer
    gl.vertexAttribPointer(shaderPosAttr, obj1PosBufferItemSize, gl.FLOAT, false, 0, 0); // create an attribute pointer for the buffer

    gl.bindBuffer(gl.ARRAY_BUFFER, obj1ColorBuffer); // same as above just with colors
    gl.vertexAttribPointer(shaderColorAttr, obj1ColorBufferItemSize, gl.FLOAT, false, 0, 0);

    gl.drawArrays(gl.TRIANGLES, 0, obj1PosBufferNumOfItems); // now at this point, how does it know that it needs to use both the obj1PosBuffer and the obj1ColorBuffer too to iterate through and pass the current values to the vertex shader?

    // draw object2

    basically the same methods here, so here opengl should not take the buffers used above into consideration. How does it know it know that it shouldn't? Is it something like that the draw call resets the selected buffers?

    I'd really appreciate if you could give me a hint with this.
    Btw do you have a good recommendation for diving into webgl? Like books, tutorials, etc?

    ReplyDelete
  9. I think I found the answer to my question, so when I call gl.vertexAttribPointer I'm actually setting that I want the given attr index to load that buffer, and that's what gets overridden when calling it for the second object.

    ReplyDelete
  10. thanks, that was very helpful!

    ReplyDelete
  11. That wrapped up my code. Thanks. I remember reading this a bit ago and found it. So I am warning you that someone better start making you a millionaire as of the visible webgl knowledgeable people on the planet, you re wearing the kitten's pajamas.

    -------------
    I was writing a parser for an obj file to a json and ran through that three level index fields of obj files , (with opengl stuck with one). And maybe I took a wrong turn, but it seems that adding more and more indices to cover multiple uses of uv's (or normals) would take up more final memory then what I remember you putting down here in this blog.
    So it worked like magic ..

    ReplyDelete
  12. That wrapped up my code. Thanks. I remember reading this a bit ago and found it. So I am warning you that someone better start making you a millionaire as of the visible webgl knowledgeable people on the planet, you re wearing the kitten's pajamas.

    -------------
    I was writing a parser for an obj file to a json and ran through that three level index fields of obj files , (with opengl stuck with one). And maybe I took a wrong turn, but it seems that adding more and more indices to cover multiple uses of uv's (or normals) would take up more final memory then what I remember you putting down here in this blog.
    So it worked like magic ..

    ReplyDelete
  13. Thank you, there really aren't any good articles that explain this rather confusing topic.

    I have a question, though. Suppose I want to draw a bunch of triangles that each have their own alpha level. Do I _need_ to include the alpha attribute for every vertex in my VBO, or can I somehow specify that the attribute applies to the following 3 vertices?

    In other words, if this were possible, here's what my VBO might look like:

    [
    ...

    0.5, // Alpha attribute for the next 3 vertices:
    0, 0, 5, // Vertex coordinate 1
    10, 0, 5, // Vertex coordinate 2
    10, 10, 5, // Vertex coordinate 3

    ...
    ]

    ReplyDelete
    Replies
    1. No, you'd have to specify the alpha for each vertex. The only way to get around that would be to create some sort of lookup table that you pass to the shader as a uniform array, but that's far more trouble than it's worth and probably won't perform very well.

      Delete