Building Web Applications With Apache Ant

The importance of having a solid build process

Modern web applications are large and complicated pieces of engineering, using many different technologies, sometimes deployed on hundreds (or even thousands) of servers throughout the world, and used by people from dozens of locales. Such applications cannot efficiently be developed without relying on a solid build process to do all the dirty and repetitive work of reliably putting all the pieces together.

Apache Ant

Apache Ant

Many tools (make, gnumake, nmake, jam, etc.) are available today to build applications. However, when it comes to building a web application, my personal favorite is definitely Apache Ant (here is a good explanation of Ant’s benefits over the other tools) This short tutorial will assume you already have some basic knowledge of Ant (if you don’t, you should flip through its user manual beforehand) This article will focus mainly on building front-end code using Ant.


Build types

It is often useful to build an application differently at different stages of its life cycle. For instance, a development build will have assertions and tons of logging code, while a production version will be stripped of all that. You may also need some intermediate build to hand off to QA, and still be able to debug things easily while running actual production code. With Ant, you can create one target per build type, and have a different dependency list for each target. Properties can also be used to customize specific build types. Here is an example:

<target name="dev" depends="load.properties.dev, js.preprocess, js.check.syntax, copy.jsp, copy.image.files, copy.css.files, copy.js.files, compile.jsp, copy.properties.files, compile.webapps, copy.libs"
    description="Development build">
</target>
<target name="prod" depends="load.properties.prod, js.preprocess, js.check.syntax, js.concatenate, js.minify, copy.jsp, copy.image.files, copy.css.files, copy.js.files, compile.jsp, copy.properties.files, compile.webapps, copy.libs"
    description="Full production build">
</target>
<target name="load.properties">
    <property file="ant.properties"/>
</target>
<target name="load.properties.dev" depends="load.properties">
    <property name="js.preprocess.switches" value="-P -DDEBUG_VERSION=1"/>
    <property name="js.compressor.switches" value="--nomunge --line-break"/>
</target>
<target name="load.properties.prod" depends="load.properties">
    <property name="js.preprocess.switches" value="-P -DDEBUG_VERSION=0"/>
    <property name="js.compressor.switches" value=""/>
</target>

Concatenate your JavaScript and CSS files

Concatenating JavaScript and CSS files contributes to making your site faster according to Yahoo!’s Exceptional Performance team. File concatenation using Ant is trivial using the concat task. However, it is often important to concatenate JavaScript and CSS files in a very specific order (If you are using YUI for example, you want yahoo.js to appear first, and then dom.js/event.js, and then animation.js, etc. in order to respect module dependencies) This can be accomplished using a filelist, or a combination of filesets. Here is an example:

<target name="js.concatenate">
    <concat destfile="${build.dir}/concatenated/foo.js">
        <filelist dir="${src.dir}/js"
            files="a.js, b.js"/>
        <fileset dir="${src.dir}/js"
            includes="*.js"
            excludes="a.js, b.js"/>
    </concat>
</target>

It is also possible to prepend some comments to the destination file, which is often used for license and copyright information, using a nested header element (see the Apache Ant manual) Just keep in mind that minifying your code will make these comments go away, so you may want to do this later in the build process.

Preprocess your JavaScript files using CPP

Preprocessing JavaScript code is very useful to facilitate the development process. It allows you to define your own macros, and conditionally compile blocks of code. One easy way to preprocess JavaScript code is to use the C preprocessor, which is usually installed by default on UNIX machines. It is available on Windows machines as well using cygwin. Here is a snippet of an Ant build.xml file illustrating the use of cpp to preprocess JavaScript files:

<target name="js.preprocess" depends="js.concatenate">
    <apply executable="cpp" dest="${build.dir}/preprocessed">
        <fileset dir="${build.dir}/concatenated"
            includes="foo.js"/>
        <arg line="${js.preprocess.switches}"/>
        <srcfile/>
        <targetfile/>
        <mapper type="identity"/>
    </apply>
</target>

You can now write JavaScript code that looks like the following:

include.js:

#if DEBUG_VERSION
function assert(condition, message) {
    ...
}
#define ASSERT(x, ...) assert(x, ## __VA_ARGS__)
#else
#define ASSERT(x, ...)
#endif

foobar.js:

#include "include.js"
function myFunction(arg) {
    ASSERT(YAHOO.lang.isString(argvar), "arg should be a string");
    ...
#if DEBUG_VERSION
    YAHOO.log("Log this in debug mode only");
#endif
    ...
}

Just a word of caution here: make sure you use the UNIX EOL character before preprocessing your files. Otherwise, you’ll have some issues with multi-line macros. You may use the fixcrlf Ant task to automatically fix that for you.

Minify your JavaScript and CSS files

Minifying your JavaScript and CSS files will help make your application faster according to Yahoo!’s Exceptional Performance team. I warmly recommend you use the YUI Compressor, available for download on this site. There are two ways to call the YUI Compressor from Ant. Here is the first one using the java task:

<target name="js.minify" depends="js.preprocess">
    <java jar="yuicompressor.jar" fork="true">
        <arg value="foo.js"/>
    </java>
</target>

Here is another way using the apply task that allows you to pass several files to the compressor using a fileset:

<target name="js.minify" depends="js.preprocess">
    <apply executable="java" parallel="false">
        <fileset dir="." includes="foo.js, bar.js"/>
        <arg line="-jar"/>
        <arg path="yuicompressor.jar"/>
        <mapper type="glob" from="*.js" to="*-min.js"/>
    </apply>
</target>

Update: Starting with version 2.2.x of the YUI Compressor, the ant target described above needs to be slightly modified:

<target name="js.minify" depends="js.preprocess">
    <apply executable="java" parallel="false">
        <fileset dir="." includes="foo.js, bar.js"/>
        <arg line="-jar"/>
        <arg path="yuicompressor.jar"/>
        <srcfile/>
        <arg line="-o"/>
        <mapper type="glob" from="*.js" to="*-min.js"/>
        <targetfile/>
    </apply>
</target>

Also consider the following ant target to minify CSS files:

<target name="js.minify" depends="js.preprocess">
    <apply executable="java" parallel="false">
        <fileset dir="." includes="*.css"/>
        <arg line="-jar"/>
        <arg path="yuicompressor.jar"/>
        <arg line="--line-break 0"/>
        <srcfile/>
        <arg line="-o"/>
        <mapper type="glob" from="*.css" to="*-min.css"/>
        <targetfile/>
    </apply>
</target>

Work around caching issues

Adequate cache control can really enhance your users’ experience by not having them re-download static content (usually JavaScript, CSS, HTML and images) when coming back to your site. This has a downside however: when rev’ing your application, you want to make sure your users get the latest static content. The only way to do that is to change the name of these files (for example by adding a time stamp to their file name, or using their checksum as their file name) You must also propagate that change to all the files that refer to them.

The copy and file name replacement will be handled by a custom Ant task named FileTransform (see FileTransform.java) The first step is to build the task from the source and define the custom task:

<target name="-setup.build.tools" depends="-load.properties">
    <mkdir dir="${build.dir}/tools/classes"/>
    <javac srcdir="tools/src"
           destdir="${build.dir}/tools/classes"
           includes="**/*.java">
        <classpath>
            <pathelement />
        </classpath>
    </javac>
    <taskdef name="FileTransform"
        classname="com.yahoo.platform.build.ant.FileTransform"
        classpath="${build.dir}/tools/classes"/>
</target>

(Note the use of a “-” in front of the name of the target. This is used to make the target “private” i.e. not invokable directly) The second part is the copy of the static content that is going to be cached using the newly defined FileTransform task:

<target name="-copy.js.files" depends="-setup.build.tools">
    <mkdir dir="${build.dir}/js"/>
    <FileTransform todir="${build.dir}/js"
        changefilenames="true"
        propertiesfile="${build.dir}/js.files.mapping.properties">
        <fileset dir="site/js" includes="*.js"/>
    </FileTransform>
</target>

The mapping between the old names and the new names is stored in a properties file. The final step is the copy of the files that refer to files which name was changed in the previous step. For this, we use the copy task:

<target name="-copy.php.files" depends="-copy.js.files">
    <mkdir dir="${build.dir}/php"/>
    <copy todir="${build.dir}/php">
        <fileset dir="site/php" includes="*.php"/>
        <filterset>
            <filtersfile file="${build.dir}/js.files.mapping.properties"/>
        </filterset>
    </copy>
</target>

You can download an archive containing a very simple Ant project illustrating this advanced technique.

Deploy your application

Deploying a web application is usually done by copying a set of files over to the production servers and running a list of commands on those servers (stop the services, run a few shell commands, and finally restart the services)

You have many options to copy files to remote servers with Apache Ant. You could use the copy task to copy files to a locally mounted file system, or you could use the optional FTP task. My personal preference is to use scp and rsync (both utilities are available on all major platforms) Here is a very simple example demonstrating the use of scp with Apache Ant:

<apply executable="scp" failonerror="true" parallel="true">
    <fileset dir="${build.dir}" includes="**/*"/>
    <srcfile/>
    <arg line="${live.server}:/var/www/html/"/>
</apply>

And here is an example showing how you can run remote commands:

<exec executable="ssh" failonerror="true">
    <arg line="${live.server}"/>
    <arg line="sudo webctl restart"/>
</exec>

Conclusion

I hope this article has given you some ideas of what’s possible to automate with modern build tools such as Apache Ant (other cool things include automatic generation of documentation, source control integration, packaging using rpm and such, etc.) My professional experience has taught me that the build process cannot be an afterthought. Just like performance, it should be part of your project from the very beginning. The second lesson is that all web developers need to be active participants in creating and maintaining the build process (with maybe one build engineer to centralize all the information) instead of relying on somebody else to build their component. Finally, keep your build process alive and maintain it actively as your needs will change during your project life cycle.

25 thoughts on “Building Web Applications With Apache Ant

  1. Pingback: Federico Feroldi’s blog » Blog Archive » links for 09-12

  2. Pingback: stephenwoods » Ant & Phing

  3. Demetrios Kyriakis

    This article about ANT is just plain cool :).
    I think it should be posted on the ANT site too.

    Please, please write more articles like this about automations with ANT :).

    Thank you,

    Demetrios.

  4. Jim Priest

    Great article!! Especially the bit about compressing JS code. I’m using jQuery more and more - and being able to compress this (and it’s many plugins) will be helpful!

    FWIW - I have a Ant wiki page setup with a lot of info:

    A lot of it is centered around using Ant with ColdFusion but still good info.

  5. Pingback: Compressing JS and CSS with Ant

  6. Learn PHP

    wow extensive tutorial. thanks for the article. definitely something new to learn here.

  7. Rob Wilkerson

    Great article. I’ve been using Ant to build web apps for several years now, but I’ve never processed my JavaScript files as you’ve discussed. I’ll be trying that next time around.

  8. Pingback: Julien Lecomte’s Blog » Trimming comments in HTML documents using Apache Ant

  9. Pingback: Web Application Construction Tools » Chris Norton

  10. Pingback: JIRA: FleetWatch3

  11. Magnus Naslund

    I’m trying to minify javascripts with ant and yuicompressor-2.2.4 but the apply script (the 2nd) doesn’t work since the result is sent to stdout. What to change in the script to make it work?

  12. Nick Aliwell

    I tried to post a sample before but it didnt work.
    heres an ant target that worked for me.

  13. Philippe Mouawad

    If you are interested, I developed an ANT Task to do minification on a set of JS/CSS using ANT fileset.

    For download and configuration:

    Thanks Julien for your great tool.
    Philippe.

  14. Pingback: IN MY OPINION » YUI Compression tool as Ant Task

  15. Mike Henke

    Could you update the ant example for yuicompressor.jar to the latest release 2.3.5? Maybe that could be included in the documentaion (Readme file) so it is update for each release. Thanks

  16. Jean-Daniel

    Thanks for this great article.

    Just a note about the CPP invocation, I suggest to had the -CC flag to avoid comments stripping. The comment stripping will be performed by yuicompressor and it will preserve conditional comments.

  17. Mitchell Cohen

    Greate article…really. Any chance you post your ant script?

  18. hiren panchal

    indeed great article.

    would it be possible for you to post sample ant project which minifies and concatenates js and css files?

    thanks in advance

  19. Jens Elkner

    Yeep, that’s a good one. However pretty inefficient ant “support” is pretty annoying. Thus I wrote a Css and Js Ant filter, which might be used e.g. in a copy task. BTW: Also improved performance by ~ factor 10 ;-).

  20. DJDaveMark

    Any reason why we can’t use the ANT tasks ‘replace’ et ‘replaceregexp’ for preprocessing the JavaScript code instead of ‘cpp’?

    This would save installing Cygwin

  21. Rimian

    Nice one! I’ve been using Apache Ant to deploy and manage tasks for a while but this takes me to a new level. Thank you.

  22. Armindo Cachada

    Julien,

    I really found your blog post useful. I used your FileTransform, tweaked it a bit and added extra features. If you are interested you can find more at:

    and

Comments are closed.