TwoYearsWithTruffle
From APIDesign
In May 2015, when I joined OracleLabs and got a task to turn Truffle from a research system for writing fast AST interpreters into an industry ready framework, I wrote an essay called Domain Expert - asking whether one can design API as a service or whether one has to be a Domain Expert when designing an API. I feel it is a time for reflection and time to describe how my attempts to design as a service ended up.
Polyglot Beginnings
One of the unique features of Truffle is its polyglot nature. The ability to freely mix languages like JavaScript, Ruby or R at full speed (as demonstrated by my sieve project) is clearly amazing. Most of that had already been written and published by Matthias Grimmer - when I joined, I just got a task to polish it into a formal API - exactly aligned with the vision of design as a service. Matthias had done a lot of work - he was able to run benchmarks written in all of the three scripting languages and also mix them with his own interpreter of C. That meant there was a lot of tests to run - just booting up the languages was complicated. Every language needed its own proprietary setup.
It was clear what I had to do: design a system of registering Truffle languages in a declarative way and initializing them in a uniform style. That was great, as my previous NetBeans_Runtime_Container API experience was all about registering and discovery of some services.
I just had to decide whether it is better to fit into an existing standards API - e.g. ScriptEngine - or whether it is better to design something from scratch. The dilemma is always the same - is it better to follow the standard and compromise on the things that don't fit in there (in the case mostly polyglot features) or design a new API that would match our needs 100%? Given the fact that polyglot features were the most important selling point, I went for a new API. That was probably good choice, but the missing support for ScriptEngine is still biting us from time to time and there is an open issue to address it.
Anyway TruffleLanguage and PolyglotEngine were created and all Matthias's interop tests could be rewritten to use just these API and don't talk to each language directly. Good example that design as a service can work.
Testing Compatibility
Another thing that was more or less obvious was the need for compatibility between languages. There are two approaches to tackle that:
- testing from bottom
- testing from the top.
Testing from the top simply means to write simple programs that mix and match languages and assert some output. Again the sieve project contains examples of that approach. However this can only test the languages one knows about (e.g. one is on top the chain of dependencies and knows them all). Thus it is a lot of work and the coverage isn't equally distributed among all languages.
Thus I prefer testing from bottom: one writes a test compatibility kit (a [[TCK]) that contains abstract test definitions with tasks that each language author needs to finish by writing suitable code snippet in own language to perform the given task. This approach scales much better and gives consistent coverage across all language implementations. Each time we detect a deviation from the expected polyglot behavior and add a test to TruffleTCK the language writers are notified and can implement the desirable functionality.
I'd say this part of my work went well even I still wasn't a Truffle Domain Expert.
Follow
