CompileTimeCache

From APIDesign

Current revision

Caches are usually created during installation time or during first execution/computation. Just like the CacheForModularity, they provide another store for information available elsewhere. This store is easily accessible, faster to read and use which then results in more optimal execution.

However caches can also be created during compilation. With JDK6 and the help of AnnotationProcessors, it is easier than ever. This article shows general principles and describes few tools that can make the task even simpler. Let's describe yet another variant of Component Injection, by rewriting the Anagram Game as introduced in Chapter 7.

Define the Annotation

Although not necessary, it is good to start with definition of an annotation in your API. At the end AnnotationProcessors are optimized to process annotations. To allow API users to easily define list of words to feed into the Anagram game, let's introduce @Words annotation:

Code from Words.java:

@Retention(RetentionPolicy.SOURCE)
@Target(ElementType.METHOD)
public @interface Words {
}
 

Any user will be allowed to annotate its own static method with this annotation instructing the API to call this method whenever somebody needs a list of words to play the anagram game. Here is a sample use:

Code from WordsFactory.java:

public class WordsFactory {
    @Words
    public static String[] myWords() {
        return new String[] {
            "microprocessor",
            "navigation",
            "optimization",
            "parameter",
            "patrick",
        };
    }
}
 

Please note that although this example is quite simple (and there are usually more complicated annotations), it already simplifies the use of the API compared to original ServiceLoader based alternative - one does not need to implement any factory interface, just provide a method and annotate it.

Compiler Plugin

The @Words annotation has RetentionPolicy.SOURCE. This means that it shall not propagate to resulting bytecode or be present in runtime (via reflection API). Instead it needs to be processed during compile time via AnnotationProcessor. Is it complex to define one? Not at all, just write:

Code from WordsProcessor.java:

@ServiceProvider(service=Processor.class)
@SupportedAnnotationTypes("org.apidesign.anagram.api.annotations.Words")
@SupportedSourceVersion(SourceVersion.RELEASE_5)
public class WordsProcessor extends LayerGeneratingProcessor {
 
    @Override
    protected boolean handleProcess(
        Set<? extends TypeElement> set, RoundEnvironment env
    ) throws LayerGenerationException {
        Elements elements = processingEnv.getElementUtils();
 
        for (Element e : env.getElementsAnnotatedWith(Words.class)) {
            Words w = e.getAnnotation(Words.class);
 
            TypeElement te = (TypeElement)e.getEnclosingElement();
            String teName = elements.getBinaryName(te).toString();
}
}
}
 

The processor needs to be present in the same JAR as the @Words annotation and it has to be registered via java extension mechanism. The code above is slightly NetBeans specific (use of @ServiceProvider instead of plain META-INF/service/ registration and subclassing of LayerGeneratingProcessor), yet it still shows the common properties of every processor. Each has a method to process its annotations which is called by the compiler as soon as these annotations are used in the compiled code. Then one needs to iterate over all those annotations and somehow process them.

The good thing is that since JDK6, the javac compiler invokes these processors by default. All that is needed is to include a library which contains their registrations on the compile classpath. Processors will be discovered and used during compilation. Natural, zero configuration needed.

Generating Caches

Annotations have gained quite a lot of popularity since Java 5. However people usually use RetentionPolicy.RUNTIME or at most RetentionPolicy.CLASS and then scan classes during execution. This is an approach used by Jersey or Spring. It is not really effective. Why? Well, imagine that the first thing your framework does is to open all JARs on classpath, seek what classes their contain and inspect some of them. If they contain the desired annotation, do something, however as majority of the classes likely does not use your annotation, just skip them (but leaving them uselessly loaded in memory). Not really effective.

Is there a better way? Indeed: Generate caches during compilation and use them to effectively load just the needed classes instead of scanning whole classpath. All that is needed is to register an AnnotationProcessor for important annotations and during compilation generate META-INF/jersey.classes or META-INF/spring.classes with list of class names to inspect. Then just use

for (Enumeration<URL> en = getClassLoader().getResources("META-INF/jersey.classes"); en.hasNextElement(); ) {
  URL config = en.nextElement();
  // read and process classes named in config
}

Even this simple optimization can dramatically boost start up performance of your framework.

Namespace

NetBeans configuration mechanism is based on concept of layers of virtual filesystems written in XML. It is quite flexible, yet slightly errorprone. When one makes a typo, it is quite hard to find where the problem is. And this is where annotations help quite a lot. Annotations are natural Java language elements. They have associated Javadoc, code completion works when you use them in an IDE, one can seek for their usage, etc. All the Java comfort is available for free.

Then, with use of layer generating AnnotationProcessors we can convert them into the flexible NetBeansLayers format. No typos, no inconsistencies in the XML definitions. Everything done automatically and even with compile time verification that the registration is sane (e.g. classes are public, methods are static, etc.). We have a builder like API to generate the XML quite easily:

Code from WordsProcessor.java:

File f = layer(e).file(
    "Services/" + teName.replace('.', '-') + ".instance"
);
f.methodvalue(
    "instanceCreate",
    "org.apidesign.anagram.impl.annotations.WordsImpl",
    "create"
);
f.stringvalue(
    "instanceClass",
    "org.apidesign.anagram.impl.annotations.WordsImpl"
);
f.stringvalue(
    "instanceOf",
    "org.apidesign.anagram.api.WordLibrary"
);
f.methodvalue(
    "words",
    teName,
    e.getSimpleName().toString()
);
f.write();
 

Lookup Integration

Nice feature of NetBeansLayers is that they provide integration with Lookup. By generating files into Services folder, one can influence results returned by Lookup.getDefault(). As a result an application that was written to do Component Injection via Lookup, can smoothly work with @Words and its AnnotationProcessor. Just by dropping the JARs on classpath, everything gets wired together and libraries registered via the annotation are available to clients expecting WordLibrary interface. How is that possible? Because the bridge class that converts the static method into appropriate interface:

Code from WordsImpl.java:

public class WordsImpl implements WordLibrary {
    private Map<String,Object> map;
 
    private WordsImpl(Map<String,Object> m) {
        this.map = m;
    }
 
    public static WordsImpl create(Map attributes) {
        return new WordsImpl(attributes);
    }
 
    public String[] getWords() {
        return (String[])map.get("words"); // NOI18N
    }
}
 

The NetBeans XML filesystem makes sure that once somebody calls map.get("words") the appropriate static method (registered under the name words by the AnnotationProcessor previously) is called. This way we can beautify our API while keeping compatibility with clients expecting old good interfaces.

Too NetBeans Specific

Parts of this article may sound a bit too NetBeans specific. Although based on our NetBeans experience, they are not specific to NetBeans at all! First of all, annotation processors are available to every API writer targeting JDK6 (and as Java 5 is about to be end of lifed soon, this shall soon be almost everyone). Then there is the question of generating caches. One can always choose the simplest possible format - e.g. a plain text file and then there are no dependencies on NetBeans either. However even if you like our NetBeansLayers format and our LayerGeneratingProcessor, don't dispair - support for this is available as two separate JAR files (org-openide-util.jar and org-openide-filesystems.jar which are also available from NetBeansMavenRepository), fully independent from the NetBeans IDE or even NetBeans Runtime Container.

Anyway, regardless you use these extensions or just plain AnnotationProcessors, do it! Generating CompileTimeCache really pays off and can simplify a lot of API as well as make it much more efficient.

<comments/>