Automatically validating GLSL files

If you are doing anything THREE.js/WebGL related, sooner or later you are going to start spending a significant amount of your time working in GLSL, rather than in JavaScript.

In a previous post I detailed how I work with GLSL files, specifically loading them into the app itself in an convenient way.

One notable missing feature was automatically validating the shaders before they are used the browser.

This post will detail how you can do this directly from your editor, to help you spot stupid mistakes while editing, rather than only alerts when testing in the browser.

Frustration

For me, it was a huge frustration to make a change in a GLSL file, reload my app in the browser and then after waiting for it to load to be told (by a pretty obscure message) that I’d missed a semicolon in my code.

Originally, I was using the GLSL Compiler, which alerted me when I made silly mistakes, such as missing a semicolon. However, it could not spot all types of errors. In particular I was forever doing things like this:

float r = 10;

Only to be greeted by the following message in the browser (usually obscured by a dump of my entire shader):

cannot convert from 'const mediump int' to 'float'

For a while, I thought that I would have to live with this, as Googling around didn’t seem to yield anything. The breakthrough came when I found a plugin for Sublime Text, GL-Shader-Validator. While I don’t use Sublime, the guts of this plugin were interesting, namely ANGLE was used to actually compile the code and detect any errors.

ANGLE

What the ANGLE executables let you do is pass in a fragment or vertex shader file, and it will try to compile it for you, reporting back any errors. E.g.

Script

Never have I been so happy to see a list of errors!

The output format wasn’t quite what I wanted so I have wrapped this in a python script, which gives me an easier to parse output:

Script

THREE.js Integration

Now the above works great for self-contained shaders, however I’m working with THREE.js, which passes in helpful variables into the shaders for me, like the cameraPosition or modelViewMatrix. The trouble is, ANGLE doesn’t know anything about these variables and hence whenever I use them, they are reported as errors. Not great.

To solve this I’ve included prefix files that the glsl-validate.py script will automatically prepend to any shader it passes to the ANGLE compiler. This basically mocks out the things that we expect to have passed in.

This approach could easily be extended to support other libraries, if you feel like doing so, please submit a pull request!

#includes

Extending the library integration idea, I’ve been working on adding a rudimentary form of #include to my shaders, so I can effectively share code between them. The implementation is pretty simple, basically whenever the validator encounters a statment like #include shader.glsl, it replaces the include statement with the contents of the included file.

Editor integration

I use vim for editing and set it up so that whenever a file is saved, glsl-validate.py is run over the file, reporting any errors. In the future I’ll probably wrap this up into a Syntastic plugin, so that the error messages appear directly in vim.

Performance

So far, I’ve been very pleased with the accuracy of detected errors, I’ve yet to hit an occassion where my shaders wouldn’t compile in the browser if they passed through the validator.

Script

The project is up on Github, hopefully others will find it useful.

  • http://codeflow.org/ Florian Bösch

    I’m doing it differently. I usually have shader snippets in the sourcecode (I use CoffeeScript, so multiline strings). Each shader string is denoted by a prefix (like ”’es). A preprocessor goes trough these files and substitutes each snippet’s line with a pseudo #line directive listing linenumber and source file.

    Then when I paste snippets together on the client, my clientside shader parser picks these apart and builds a mapping of shaderline to sourceline. So when WebGL throws a shader error, I parse that error, and translate it back to a trace citing my original source location and file.

    • pheelicks

      That sounds great from a debugging in the browser perspective, have you shared the code anywhere?

      The thing that bugged me about including strings inside js files was that I lost syntax highlighting, and I don’t see how I could nicely validate the code before sending it to the browser. The application I’m working on takes about 3 seconds to load, and then another couple of seconds to display an error in GLSL, which is quite painful, compared to vim pointing out the errors on save.

      By the way, glad to see you here, I’ve come across your site in the past and it was very interesting and helpful.

      • http://codeflow.org/ Florian Bösch

        It’s floating about in some of my released projects in various iterations.

        You can teach vim to parse subsyntax, after all it’s enclosed in ”’es ”’ bits.

        I agree that you couldn’t really do pre-validation using snippets. However being able to mash together a variety of snippets and to produce them programmatically but keep them at least debuggable is also a great boon. Everything’s a tradeoff.

  • http://greweb.fr/ Gaëtan Renaudeau

    I think you guys should also give a try to https://npmjs.org/package/glslify which is like browserify but for GLSL.

    Very powerful.