WebGPU/glTF case study

Over the last year or so I've been working on Chrome's implementation of the new WebGPU API, a successor to WebGL based on more modern graphics API patterns. While working on the API I've also been busy building a lot of demos and other community resources with it, and one topic has been brought up by people I've talked to probably more than anything else: How do you keep the API state manageable when loading something like a glTF model?

So I wrote about it!

If you're interested in WebGPU, glTF, or simply graphics on the web, check it out!

Efficently rendering glTF models: A WebGPU case study 

WebXR Scene Optimization

I posted a walkthrough of some basic steps for optimizing scenes for WebXR/WebGL on GitHub. Take a look!

https://toji.github.io/webxr-scene-optimization/

Early access to the WebXR Device API in Chrome Canary

Hey, are you a web developer that wants to play with bleeding-edge browser VR APIs and influence their development? If so read on!

The WebXR Device API is the next iteration of WebVR, and the API that we intend to release in stable browsers at some point and support going forward. It’s still a work in progress, but the first (mostly) working implementation is now available in Chrome Canary for Android and Windows behind a flag, and so it’s the perfect time for motivated developers to start playing with the API and giving the Immersive Web Community Group feedback!

New experimental WebVR builds for Android

It's an exciting time for WebVR on mobile right now! Today the Chrome team announced that WebVR is available as an Origin Trial in the Chrome 56 Beta for Android. Oculus has also recently launched their Carmel browser preview, and Samsung is continuing to improve their implementation. On the desktop side Mozilla is continuing to make great progress, and the spec is making huge strides as all of the interested parties (like Microsoft, whose input has been invaluable!) identify the rough points and flesh out the edge cases. The future of VR on the web looks bright!

And because we're so excited about that future, a few of us on the Chrome VR team wanted to give developers a peek at what's coming for WebVR performance beyond the Chrome 56 release. Browser release schedules being what they are we weren't able to get all of the optimizations for WebVR into Chrome 56 that we wanted and still hit the release date. We've already got some exciting proof-of-concepts for significant performance enhancements that we intend to make part of future builds though! So in the tradition of the experimental desktop builds that have bootstrapped the WebVR desktop development community we're making an experimental Android Chromium build available today on https://webvr.info/get-chrome (Look for ChromePublic-webvr-native-WIP-201*****.apk)

This build carries with it all the caveats of the previous experimental Chrome builds for Android. It's not intended for normal browsing, it's definitely a bit unstable, it doesn't have various patent-encumbered media decoders baked in, and it comes with exactly zero support. What it DOES have is our latest and greatest rendering pipeline research that we've found to yield pretty impressive performance wins.

I should take a moment and call out my colleague on the Chrome VR team Klaus Weidner for his excellent work in this area. I may have the public reputation for being the WebVR guy, but Klaus is responsible for a great deal of the WebVR rendering optimization on Android. He gave a great lightning talk about the work he's being doing at the W3C VR workshop this October, which you should check out if you're interested in that kind of thing!

There's some known issues outlined in the Release Notes you should know about, the primary one being that this is currently only for Daydream Ready devices (though a Daydream View is not required.) But if you're a WebVR developer with the right hardware take a peek and give us some feedback!

[EDIT: I also wanted to clarify the Origin Trial thing, since it appears to be a point of confusion. Origin Trials are for sites that want to make a feature available to all of their visitors by default. It's intended for use on pages like the ones linked below that are public facing and don't want to make users flip obscure flags to see the VR content.

If you're just doing development or testing of WebVR pages, though, you do NOT need an Origin Trial token! You can access the exact same features by turning on the "WebVR" and "Gamepad Extensions" flags in about:flags. (The Origin Trial covers both features).]

A couple of pages I recommend trying out with the new build

• https://webvr.info/samples (of course)
• https://www.clicktorelease.com/code/cruciform/vr/
• https://threejs.org/examples/?q=webvr
• https://media.tojicode.com/q3bsp/ (If you've got the stomach for it the touchpad on the Daydream controller will let you run around the level w/ click to jump!)

Happy WebVR-ing!

Update on WebVR spec, Chrome, and HTTPS

(This post was originally sent to the public-webvr mailing list, which you should totally follow if you're interested in the latest and greatest WebVR news!)

Hello WebVR community!

The Chrome team has been a little quiet on the WebVR front recently, and I apologize for that and thank you for your patience. We’ve been heads down and working hard to deliver the best experience we can for developers as soon as we can. Several things have happened recently that will affect what we’re delivering and how. As such, we wanted to give developers an update.

Oculus Rift and HTC Vive review

By virtue of the WebVR work I've been doing I've been lucky enough to have a fair amount of access to VR hardware, and at this point feel like I've got a pretty good handle on where each of the newly released devices stand in relation to one another. Since it seems to be a popular topic lately I figured I'd give a brief overview of my thoughts on the Vive and Rift.

Moving towards WebVR 1.0

Consumer VR is at our doorsteps, and it’s sparking the imagination of developers and content creators everywhere. As such it’s no surprise that interest in WebVR is booming. Publications like the LA Times have used WebVR to explore the landscape of Mars, and a doctor was able to save the life of a little girl by taking advantage of Sketchfab’s VR features. The creativity and the passion of the WebVR community has been incredible!

Those of us who have been defining and implementing the API feel a responsibility to make sure that it keeps pace with the current state of VR hardware and software. That’s not always easy with the breakneck speed with which the field has been evolving, and as we look at the API as it exists today there’s some significant disconnects from the realities of modern VR.

A quick refresher about how we arrived at the point that we're at now: When WebVR was first conceived by Vlad Vukicevic (April 2014) Oculus had just barely announced the DK2 and the only VR headset most people could get was the DK1. The Vive, 6DoF controllers, Hololens, and GearVR were still behind closed doors at this point. Cardboard had only just been announced when we first started making builds available. The APIs used to interact with the hardware that was available looked very different than it does today. That’s why in my initial blog post about the API I said “Keep in mind that these interfaces absolutely, without question, WILL change”.

We’re taking that sentiment to heart, and in the interest of keeping WebVR relevant and (hopefully) a bit more future proof we’re proposing some major, backwards-compatibility-breaking changes. You can see the new proposed spec here, but I wanted to cover some of the changes in a bit more detail and go into the rational behind them.

Bringing VR to Chrome

[Update: WebVR has evolved quite a bit since this post! Read about the latest changes.]

The Good Stuff

Let's get all the links out of the way upfront, because that's what you're really here for, right?

WebVR-enabled Nightly Chromium Builds
Go here for builds

Source
Chromium WebVR experimental branch

Demos (Should work with Chrome and Firefox's WebVR implementation)
Quake 3 level renderer (Source) - Click the "VR" button in the lower right
WebVR test page (Source) - Simple test apps

Related Links
blink-dev "Intent to Implement WebVR" thread
Vladimir Vukićević blog post about Mozilla's WebVR implementation
web-vr-discuss mailing list - If you have feedback about WebVR in general, this is the place for it
Chrome WebVR bug tracker - If you have issues with Chrome's WebVR implementation, log them here.

WebVR: What it is and what it isn't.

To start out, I'm going to highly recommend that you go read Vlad's blog post about Mozilla's WebVR plans first. Chrome's current implementation is very close to Firefox's, so it doesn't make sense to re-state what's already been written. (I'll discuss the differences in implementations in a moment.)

Crowdsourcing Unreal Tournament

I spent an awful lot of time from my middle school years forward learning everything I could about the mechanics of making video games. The goal being, of course, to make my own. And not some silly small asteroids game or anything like that. No! I wanted to build Shooters and Platformers and RPGs. The epic kind! With lots of story and cut scenes and hours of gameplay in massive open worlds! (Ah, the joys of blissfully and unknowingly throwing oneself at impossible tasks.)

As you might imagine, I failed spectacularly at this effort. Luckily for me I never quite viewed it as a failure. It was just a matter of not yet knowing enough about subject X or software Y or technique Z. So my quixotic quest actually propelled me to learn a tremendous amount about programming and graphics development and all manner of related subjects, a fact which I am enormously grateful for today as it has directly led to my current employment. And even today I still have a tendency to chase after projects that are too big for me to reasonably handle in the free time I have available. But that's okay, because this is one of those cases where shooting for the moon really can land you among the stars, and it gives me a wonderful excuse to try all manner of fun, crazy things that would never come up in the course of my day-to-day job but still prove useful anyway.

Given this background, I couldn't help but feel the tiniest bit sad that Epic didn't announce their latest engine pricing or their new crowdsourced path for the next Unreal Tournament about 15 years earlier. Teenage me would have been all over that! I spent a decent amount of time hacking with the Quake 3 SDK and other similar code bases, mostly in an effort to learn more about how they worked and how "real" programmers did things. The idea that I could have access to the full source of a modern game engine for a few dollars a month would have been a dream come true! And beyond that, the fact that I could have not only watched but participated in, from day 1, the development of a AAA game would have probably killed me with giddiness. That was exactly what I wished I could have back in the day: A perfect little portal through which I could view the professionals at work creating something real, large, and "cool". I think I would have poured many weeks of my life into following the project and trying to contribute in any way I possible. (I shudder to think of 15 year-old me's code actually ending up anywhere public though. I was better than a lot of people gave me credit for at that point but I wasn't nearly as good as I thought I was.)

Oculus, Facebook, and the dreaded "Ecosystem"

It's taken me several days to get over my knee-jerk, emotional, Vader-esque "NOOOOO!!!" response regarding Facebook's purchase of Oculus. I wrote up a "Please cancel my DK2 pr-order" email immediately after hearing the news, but resisted hitting the send button because I knew that anything I did at that point would be out of spite and not because I truly no longer believed in the product. Now that I've had some time to think on the matter, though, I feel like I can actually enumerate my concerns pretty clearly.

Xbox One controller in Chrome on OSX

I've been working lately on better Gamepad API support in Chrome (call it my "20% project"). It's been a fun change of pace from WebGL, primarily because it's much more straightforward code. In the process of learning the ropes, I decided to try something a little crazy: I noticed that our code path for supporting the 360 controller on OSX didn't rely on any drivers, but instead used IOKit to parse the data packets manually. (Check out that code here.) So I figured, hey! Why not try the same thing with the Xbox One controller, ignoring the fact that it's currently supported on exactly zero non-console platforms.

Somewhat unexpectedly, it really wasn't too hard!

 

How Blink has affected WebGL, Part 2

In a previous post I detailed some of the ways that the migration to Blink has affected the WebGL pipeline. The short version is that we were able to remove some layers of abstraction without changing the code which was executed, which cleaned up our dependency diagram a bit. At the time we ended up with a data flow that looked roughly like this:

That was 8 months ago, and I've been busy in the mean time! So let's take a quick peek at the current state of affairs.

Notch, WebGL, Dart, and ramping up quickly

Just a little something I thought was worth pointing out. @notch, the creator of Minecraft, has been posting updates on his latest toy project to his Twitter account recently. I've found this very interesting because he's doing a WebGL app in Dart, which just makes me all sorts of happy. Ignoring that, however his tweets illustrate something that I feel is very important.

failIfMajorPerformanceCaveat: With great blacklisting power comes the need for self restraint

Chrome is gaining a new WebGL context creation attribute: failIfMajorPerformanceCaveat. In the words of the spec, when you've set this useful little boolean to true:

Context creation will fail if the implementation determines that the performance of the created WebGL context would be dramatically lower than that of a native application making equivalent OpenGL calls.

Essentially, if there's any reason why the browser feels that the requested WebGL context will have significant performance overhead setting this flag allows the browser to simply not give you a context. It's a way for you as the developer to say "This context needs to perform really well, and if you can't deliver that then I'd rather not get one at all." In Chrome's case specifically, there's only one reason we will currently fail to return a context and that's if the SwiftShader software renderer is being used. (We may add more cases in the future.)

I think the kneejerk reaction many developers will have upon learning about this is: "Hey! I always want my WebGL to run fast! I'm going to set this flag on every context I ever create forever!"

If that sounds like your thought process, I give a you a stern look of disapproval: ಠ_ಠ

Using WebGL on Chrome (AKA: "Why don't I have the WebGLs?!?")

It occurred to me the other day that although WebGL is now available on more devices with Chrome than ever before, Google actually hasn't said a whole lot about who has it, how to get it, and what to look for if you don't have it. Today I'm going to try and fix that!

The TL;DR version is that with some predictable exceptions most devices that run Chrome can get to WebGL at this point, and an increasing number of them have it turned on without any action on the part of the user. If you don't have WebGL on your device of choice it's most likely because we've explicitly identified problems with your hardware or your device hasn't implemented the appropriate safeguards, but in most cases you can still turn it on manually if you really want to.

At last! Chrome D3D11 day has come!

As of revision 223716, Chrome Canary will have the ability to use Direct3D 11 (via ANGLE) as the rendering backend! Woot woot! The new backend can be enabled using the "Enable D3D11" flag in about:flags. To tell which backend you are using visit about:gpu and look for the GL_RENDERER string.

That's cool, but should have no impact whatsoever on your day to day Chrome use (and if it does, we want to hear about it!) So why do I bring it up? Because along with the shiny new backend comes a much awaited WebGL feature: Multiple render targets! This extension has been available to Mac and Linux devs for a little while now (it's still behind the Draft Extensions flag), but had some spec issues that prevented it from being implemented in the ANGLE D3D9 backend. Now with the new D3D11-based backend the extension should be available to a much wider range of developers.

(Want to know if you have it? Look for "WEBGL_draw_buffers" in the supported extensions list on http://webglreport.com/)

So go, brave WebGL developers, and defer all your renderings for great justice!

[UPDATE: I initially said that the D3D11 backend may be enabled by default on some systems. Unfortunately that's not the case just yet, and I apologize for the error.]

What's coming in WebGL 2.0

The WebGL working group has just released a public draft of the WebGL 2.0 spec. Hooray! Of course, being a public draft things are still subject to change and there are still plenty of TODOs, so don't be too surprised if things get chopped, tweaked, added or completely reworked before WebGL 2.0 reaches a browser near you.

The nice thing is, however, that since the entire goal of the spec is to bring OpenGL ES 3.0 capabilities to the browser the chances of things deviating too much from that spec are pretty minimal, and it's probably safe to assume that most ES 3.0 features will make their way into your browser soon. (With a few notable exceptions that I'll discuss in a bit.)

So what is new in 2.0? I realize that reading specs is rarely fun, so allow me to summarize for you. While this won't be a complete rundown of the upcoming features it should provide a good sense of where things are headed.

Holistic WebGL

ho·lis·tic  

Adjective

Characterized by comprehension of the parts of something as intimately interconnected and explicable only by reference to the whole. Characterized by the treatment of the whole person, taking into account mental and social factors, rather than just the physical...

Definition from Google Search

As I've established way back on the very first post on this site, WebGL is awesome. So awesome that one might be tempted to start seeing opportunities everywhere around the web to throw some 3D goodness at it. After all: when your hammer is a 3D API, every problem looks like a vertex array... er, or something like that. Now never let it be said that I discouraged the use of WebGL, but as developers we should also be aware that every technology has a time and place.

What is holistic WebGL? It's application of the technology in a way that takes in to account the entire web experience, not just the content of your canvas tag. Holistic WebGL content is considerate about it's context and respectful of the user's time, attention, and battery life. In essence, it's the exact opposite of this:

Now I'm guessing that very few visitors to this site were actively working on "Punch the Polygonal Monkey" banner ads, but it's not hard to envision such a thing. After all, we all know that "eye catching" is an advertiser's friend. What's a poor ad designer to do now that their beloved Flash is being slowly smothered? Why, hellllooooo there HTML5! What lovely WebGL you have! And now we have pages where a deluge of banner ads all rendering beautifully shaded cars, watches, and pharmaceuticals makes your battery commit suicide, your fans squeal, and your page chug.

But wait! It doesn't have to be like that! WebGL can be used in a way that's both exciting and engaging while simultaneously being non-obtrusive and (crucially) non-rage-inducing. Much of it involves simply being aware of what place your WebGL content in the page as a whole.

WebGL instancing with ANGLE_instanced_arrays

And still no Shakespeare...

Ever find yourself in a position where you absolutely, positively have to get a ton of monkeys on screen with a single draw call?

Yeah, okay, me neither. (At least, not until I put together the demo for this post.)

But there's certainly occasions when it makes a lot of sense to redraw the same mesh many times with some minor tweak like position, orientation, color, etc. Simple examples of this are trees and other vegetation, streetlights,  boxes in a warehouse, soldiers in an army, and so on.

Traditionally with WebGL the optimal way to draw those repeated meshes, commonly referred to as instances, would be something like the following pseudocode:

bindMeshArrays(gl);
for (i = 0; i < meshInstances.length; i++) {
  var instance = meshInstances[i];
  gl.bindUniform3fv(meshPosition, instance.position);
  gl.bindUniform4fv(meshColor, instance.color);
  gl.drawElements(gl.TRIANGLES, indexCount, gl.UNSIGNED_SHORT, 0);
}

This is good, because we only set up the vertex attributes (implied to be in bindMeshArrays) once,  then make many draw calls in quick succession, only changing the properties that are different from instance to instance. In this case just position and color. This makes for a near minimal amount of overhead for each draw call, and in an environment like Javascript where each call is expensive that's important!

(This example is pretty simple, and in fact you can actually optimize it further by getting creative with how you provide the GPU with the data. Check out Gregg Tavares' wonderful Google I/O talk for more ideas about how to render lots of geometry very quickly.)

Even in this scenario, however, you have to make at least three WebGL calls for every instance of the mesh. That doesn't sound terrible but in the real world that would probably be quite a bit more, and the fact is that in Javascript every call into native code (Like the WebGL API) carries a certain expense. Thus while drawing repeated meshes in this way works, and generally works well, it could still be better.

And that's exactly why we have ANGLE_instanced_arrays.

No more Google+ comments

I tried out the new Blogger Google+ comments for a while, but after experiencing some weird behavior (like comments not displaying on mobile) and hearing from a couple of people that avoided leaving comments because they didn't want a Google+ account I've decided to turn them off. This may cause a couple of comments to disappear, and I apologize for the inconvenience, but I'd rather have a reliable commenting system.