WebGL – working with GLSL source files

If you are doing anything THREE.js/WebGL related, sooner or later you are going to start spending a significant amount of your time working in GLSL, rather than in JavaScript.

This post is going to cover the workflow I have adopted, to make development faster and more enjoyable. The context here is a THREE.js app that uses Require.js to structure the code. You can see a simple example of such an apphere.

EDIT: I’ve since updated my workflow, so you might want to check out the new post here.

Shaders

Shaders are programs written in GLSL, which looks very much like C, except that it has some extra functionality built-in, like vector and matrix operations, or support for textures. Here’s how some GLSL code might look:

void main() {
  float h = length(position);
  vec3 transformedPosition = transformPosition(position);
  transformedPosition = transformedPosition + vec3(1.0, 2.0, 3.0);
  // ...
}

To get the graphics card to execute this when we’re running in an WebGL-enabled browser, we need to take this entire program, as a string in JavaScript, and send it to the graphics card using the WebGL APIs, for compilation. When we’re using THREE.js, this is abstracted away from us, as we create a Material, however we still need to pass the Material the shader as a string.

Approaches

A common technique I’ve seen people use <script> tags to house this code, and then use DOM methods to get at the content.

<script type="x-shader/x-vertex" id="vertShader">
  void main() {
    float h = length(position);
    vec3 transformedPosition = transformPosition(position);
    transformedPosition = transformedPosition + vec3(1.0, 2.0, 3.0);
  }
</script>

// Later in JavaScript
var vertShader = document.getElementById('vertShader').textContent;

Or another way is to directly put the shader together in JavaScript:

var vertShader = [
  "void main() {",
    "float h = length(position);",
    "vec3 transformedPosition = transformPosition(position);",
    "transformedPosition = transformedPosition + vec3(1.0, 2.0, 3.0);",
  "}"
].join("\n"),

The first method works fine, but as I’m using Require.js to modularize my code, it didn’t seem to fit to pull in content from <script> tags.

The second method works, and allows me to encapsulate the shaders into a Require.js module, however it is an absolute nightmare to edit, as everything is wrapped in double quotes and it is very easy to forget to add a trailing comma at the end of each line.

Shader compilation

To address this, I put together a simple converter, which takes shader files (ending in .frag or .vert) as inputs and combine them into a Require.js module, which I could then easily import in the rest of my application.

The advantage of writing the shaders in separate files like this, is that we get syntax highlighting, and a nicer way to organise our shader code.

Despite calling this process compilation, the GLSL code isn’t actually compiled. However, in the future I hope to exapnd this script so that it can also perform validation of the GLSL code, by compiling it.

Example

As an example, lets put together a simple ShaderMaterial that we can apply to a Mesh. First we’ll create the vertex and fragment shader files in /js/shaders/, the same location that our converter is in.

// file: /js/shaders/simple.vert 
void main() {
  gl_Position = projectionMatrix * modelViewMatrix * vec4(position,1.0);
}

// file: /js/shaders/simple.frag 
uniform vec3 uColor;

void main() {
  gl_FragColor = vec4(uColor, 1.0);
}

These don’t do very much, the position of each vertex is left untouched and the object is draw as a solid color uColor, a uniform that will be passed into the shader. When we run /js/shaders/compile.py, it will produce a Require.js module /js/app/shader.js with the following content:

define( [], function() {
  return {
    vertex: {
      simple: [
        "void main() {",
        "  gl_Position = projectionMatrix * modelViewMatrix * vec4(position,1.0);",
        "}",
      ].join("\n"),
    },
    fragment: {
      simple: [
        "uniform vec3 uColor;",

        "void main() {",
        "  gl_FragColor = vec4(uColor, 1.0);",
        "}",
      ].join("\n"),
    },
  }
} );

This will now allow us to create a material using this shader, like so:

define( ["three", "shader"], function( THREE, shader ) {
  return {
    simple: new THREE.ShaderMaterial( {
      uniforms: {
        uColor: { type: "c", value: new THREE.Color( "#ff0000" ) }
      },
      vertexShader: shader.vertex.simple,
      fragmentShader: shader.fragment.simple
    } ),
  };
} );

To see a full working example, check out the code on github. This example doesn’t use the custom shader by default, but if you modify /js/app/app.js, you can easily load in material.simple, rather than material.grass.

Here’s how a cube looks with the simple material applied:

Red cube

Automation

I have my editor (vim) setup such that when the shader source files are updated, the compile.py script automatically runs and updates /js/app/shader.js, which means that I can forget about this intermediate file altogether, and just edit the GLSL code and have the app always be up to date in the browser. The structure (e.g. indenting) of the files is preserved so it is easy to see where errors are, if there are bugs in your GLSL code.

Future

In the future, I’d actually like my compile.py script check for errors in the GLSL code, so I don’t have to wait for the whole app to be loaded in the browser, the GLSL shader sent to the GPU, just to be notified that I’ve missed a semicolon. I’ve had some success integrating with GLSL Unit, although it still isn’t perfect, so I’d be curious to hear what others are doing.

Finally, debugging GLSL code is pretty painful, as it runs on the GPU, however a recent development that should make things better is the inclusion of live GLSL in the Firefox Developer Tools.

  • Roy Williams

    I am the author of GLSLUnit, Please ping me (my gmail address is on the google code project), happy to hash out how to integrate!

  • http://coryg89.github.io/ Cory Gross

    This is cool. I never liked loading GLSL shaders from <script> tags. Maybe it would be nice to have an option to output a module that supports CommonJS implementations as well.

  • http://ganbarugames.com Nathan

    I have absolutely no experience with GLSL, but couldn’t you just use the ‘text’ plugin with RequireJS? (https://github.com/requirejs/text)

    • pheelicks

      Yes, I actually found an answer on StackOverflow saying the exact same thing just after writing this post – I wasn’t aware of the plugin functionality. My long term plan is to actually have the GLSL code compile, which I didn’t think I could do from a Require.js environment. Ideally I would be able to wrap it up into a Require.js module, which would act like the ‘text’ plugin, but also validate/minimize the GLSL

  • Gianluca Guarini

    great article thanks! I have just released recently a simple threejs bootstrap that allows you to use the shaders in separate files right out of the box check it out:

    threejs amd bootsrap

    https://github.com/GianlucaGuarini/threejs-amd-bootstrap

    I think it could be really useful for beginners and pro threejs coders

    If you have any feedback do not hesitate to open feature requests directly on Github https://github.com/GianlucaGuarini/threejs-amd-bootstrap/issues and I will be glad to keep enhancing this small project

    • pheelicks

      This really nice, thanks for sharing. I made something similar (https://github.com/felixpalmer/amd-three.js) a couple of months ago, although it lacks the bower integration and builds.

      Have you looked into linting GLSL code at all? This is something I’d like to get working, as it is quite inefficient when bugs in the GLSL code only show up once the app is loaded in the browser.