I took a look at the type inference code in guile-prescheme and I'm impressed with how elegant it is. However, it's an imperative implementation. My algorithm is functional and I'd like to keep it that way. Once I have something that produces correct output for all of my existing shaders I will try to refactor the code into something nicer.
I now have basic struct types working. The qualified type logic was extended to support them and overloaded function generation was extended to deal with structs as arguments. I also added function signatures for a number of built-in GLSL functions. I can once again compile the simple vertex shader I use for 2d sprites:
(top-level ((in vec2 position) (in vec2 texcoords) (out vec2 frag-tex) (uniform mat4 mvp)) (outputs (vertex:position (* mvp (vec4 (-> position x) (-> position y) 0.0 1.0))) (frag-tex texcoords)))
The interesting bit is the 'for-all' expression that generalizes the function 'f' to accept any arguments as long as they satisfy the predicate (the 'and' expression). In this case the only valid type for 'x' is an integer.
The compiler is currently unable to emit GLSL code for the function because it doesn't yet know how to translate a 'for-all' into a series of functions for each valid type combination. That's the next step.
What might be useful is allowing for a signature like 'a, b -> c' and associate substitutions with predicate conditions. If 'a' is a vector and 'b' is a matrix then substitute 'b' for 'c'. If 'a' is a matrix and 'b' is a vector, substitute 'a' for 'c'.
One limitation of this approach seems to be (meaning I haven't found a paper/article stating otherwise yet) that it cannot express all of the possibilities of function overloading. For example, in GLSL it is valid to multiply a matrix by a vector. It is also valid to multiply a vector by a matrix. Both forms return a matrix, but the signatures are different. The first is 'a, b -> a' and the second is 'b, a -> a'. This would require two differently named functions in a qualified type system.
Qualified types introduce a type predicate that needs to be checked after inferring each expression. For example, let's say the signature of '+' is 'a, a -> a' where 'a' is any type. Addition only makes sense for certain types of data, so we can then specify which types are valid with a predicate. For example: where 'a' is a float or an int. Adding two booleans is valid as far as the function signature is concerned, but it will then fail the predicate check.
Seagull is feeling pleasantly Schemey! There are two big missing features in the language: Loop syntax (GLSL *severely* restricts looping and recursion is *not* allowed so a loop form is essential, unfortunately) and user defined structs. I already have define-shader-type syntax that I've been using with my existing GLSL shader code so I just need to a way to import those into Seagull code and have the compiler generate the GLSL struct declaration.
The next big step was hacking this compiler into my existing shader code so I could actually use the shaders and test that they really work. This required a linking pass that does a bunch of renaming on the separately compiled vertex and shader stages to ensure that vertex output names match up with fragment input names (GLSL needs the names to be the same, ugh!) and that uniform variables between both stages use unique names because they share a global namespace.