Building

Compiling
Resources
More On Building
Cleaning
Continuous Compilation

To remove any confusion, Buildr’s build task is actually called build. It’s also the default task that executes when you run buildr without any task name.

The build task runs two other tasks: compile and its associated tasks (that would be, resources) and test and its associated tasks (test:compile, test:setup and friends). We’ll talk about compile more in this section, and test later on. We’ll also show you how to run build without testing, not something we recommend, but a necessary feature.

Why build and not compile? Some projects do more than just compiling. Other projects don’t compile at all, but perform other build tasks, for example, creating a database schema or command line scripts. So we want you to get in the practice of running the build task, and help you by making it the default task.

Compiling

Each project has its own compile task you can invoke directly, by running buildr compile or as part of another build task. (Yes, that build).

The compile task looks for source files in well known directories, determines which compiler to use, and sets the target directory accordingly. For example, if it finds any Java source files in the src/main/java directory, it selects the Javac compiler and generates bytecode in the target/classes directories. If it finds Scala source files in the src/main/scala directory it selects the Scalac compiler, and so forth.

A single project cannot use multiple compilers at the same time, hence you may prefer creating subprojects by programming language. Some compilers like Groovy’s are joint-compilers, this means they can handle several languages. When the Groovy compiler is selected for a project, .groovy and .java files are compiled by groovyc.

Most often, that’s just good enough and the only change you need to make is adding compile dependencies. You can use compile.dependencies to get the array of dependency file tasks. For Java, each of these tasks points to a JAR or a directory containing Java classes, and the entire set of dependencies is passed to Javac as the classpath.

Buildr uses file tasks to handle dependencies, but here we’re talking about the Rake dependency mechanism. It’s a double entendre. It invokes these tasks before running the compiler. Some of these tasks will download JARs from remote repositories, others will create them by compiling and packaging from a different project. Using file task ensures all the dependencies exist before the compiler can use them.

An easier way to specify dependencies is by calling the compile.with method. It takes a list of arguments and adds them to the dependency list. The compile.with method is easier to use, it accepts several type of dependencies. You can use file names, file tasks, projects, artifacts specifications and even pass arrays of dependencies.

Most dependencies fall into the last three categories. When you pass a project to compile.with, it picks up all the packages created by that project. In doing so, it establishes an order of dependency between the two projects (see Defining the Project). For example, if you make a change in project teh-api and build teh-impl, Buildr will detect that change, recompile and package teh-api before compiling teh-impl. You can also select a specific package using the project’s package or packages methods (see Packaging).

When you pass an artifact specification to compile.with, it creates an Artifact task that will download that artifact from one of the remote repositories, install it in the local repository, and use it in your project. Rake’s dependency mechanism is used here to make sure the artifact is downloaded once, when needed. Check the Artifacts section for more information about artifact specification and repositories.

For now let’s just show a simple example:

compile.with 'org.apache.axis2:axis2:jar:1.2',
  'org.apache.derby:derby:jar:10.1.2.1', projects('teh-api', 'teh-impl')

Passing arrays to compile.with is just a convenient for handling multiple dependencies, we’ll show more examples of that when we talk about Artifacts.

Likewise, the compile task has an array of file tasks that point at the source directories you want to compile from. You can access that array by calling compile.sources. You can use compile.from to add new source directories by passing a file name or a file task.

For example, let’s run the APT tool on our annotated source code before compiling it:

compile.from apt

When you call apt on a project, it returns a file task that points to the target/generated/apt directory. This file task executes by running APT, using the same list of source directories, dependencies and compiler options. It then generates new source files in the target directory. Calling compile.from with that file task includes those additional source files in the list of compiled sources.

Here’s another example:

jjtree = jjtree(_('src/main/jjtree'), :in_package=>'com.acme')
compile.from javacc(jjtree, :in_package=>'com.acme'), jjtree

This time, the variable jjtree is a file task that reads a JJTree source file from the src/main/jjtree directory, and generates additional source files in the target/generated/jjtree directory. The second line creates another file task that takes those source files, runs JavaCC on them, and generates yet more source files in target/generated/javacc. Finally, we include both sets of source files in addition to those already in src/main/java, and compile the lot.

The interesting thing about these two examples is how you’re wiring file tasks together to create more complicated tasks, piping the output of one task into the inputs of another. Wiring tasks this way is the most common way to handle complex builds, and uses Rake’s dependency mechanism to only run tasks when it detects a change to one of the source files.

You can also control the target directory. Use compile.target to get the target directory file task. If you need to change the target directory, call the compile.into method with the new path.

We use method pairs to give you finer control over the compiler, but also a way to easily configure it. Methods like dependencies and sources give you a live array you can manipulate, or iterate over. On the other hand, methods like with and from accept a wider set of arguments and clean them up for you. They also all return the same task you’re calling, so you can chain methods together.

For example:

compile.from('srcs').with('org.apache.axis2:axis2:jar:1.2').
  into('classes').using(:target=>'1.4')

Buildr uses the method pair and method chaining idiom in many places to make your life easier without sacrificing flexibility.

Occasionally, you’ll need to post-process the generated bytecode. Since you only want to do that after compiling, and let the compiler decide when to do that – only when changes require re-compiling – you’ll want to extend the compile task. You can do that by calling compile with a block.

For example, to run the OpenJPA bytecode enhancer after compiling the source files:

compile { open_jpa_enhance }

You can change various compile options by calling, you guessed, compile.options. For example, to set the compiler to VM compatibility with Java 1.5 and turn on all Lint messages:

compile.options.target = '1.5'
compile.options.lint = 'all'

Or, if you want to chain methods together:

compile.using :target=>'1.5', :lint=>'all'

Sub-projects inherit compile options from their parent project, so you only need to change these settings once in the top project. You can do so, even if the top project itself doesn’t compile anything.

The options available to you depend on which compiler you are using for this particular project, obviously the options are not the same for Java and Flash. Two options are designed to work consistently across compilers.

Buildr turns the warning option on by default, but turns it off when you run buildr --silent. It also sets the debug option on, but turns it off when making a release. You can also control the debug option from the command line, for example:

# When calling buildr
$ buildr compile debug=off

# Once until we change the variable
$ export DEBUG=off
$ buildr compile

The default source and target directories, compiler settings and other options you can use depend on the specific language. You can find more information in the Languages section.

Resources

The compile task comes bundled with a resources task. It copies files from the src/main/resources directory into target/resources. Best used for copying files that you want to include in the generated code, like configuration files, i18n messages, images, etc.

The resources task uses a filter that can change files as it copies them from source to destination. The most common use is by mapping values using a hash. For example, to substitute “${version}” for the project’s version number and “${copyright}” for “Acme Inc © 2007” :

resources.filter.using 'version'=>version,
  'copyright'=>'Acme Inc (C) 2007'

You can also use profiles to supply a name/value map that all resources task should default to, by adding a filter element to each of the profiles. The following examples shows a profiles.yaml file that applies the same filter in development and test environments:

filter: &alpha1
  version: experimental
  copyright: Acme Inc (C) 2007

development:
  filter: *alpha1
test:
  filter: *alpha1

You can specify a different format by passing it as the first argument. Supported formats include:

Format	Usage
`:ant`	Map from `@key@` to value.
`:maven`	Map from `${key}` to value (default).
`:ruby`	Map from `#{key}` to value.
`:erb`	Map from `<%=key%>` to value.
`Regexp`	Map using the matched value of the regular expression (e.g. `/=(.*?)=/`).

For example, using the :ruby format instead of the default :maven format:

resources.filter.using :ruby, 'version'=>version,
  'copyright'=>'Acme Inc (C) 2007'

For more complicated mapping you can also pass a method or a proc. The filter will call it once for each file with the file name and content.

If you need to copy resource files from other directories, add these source directories by calling the from method, for example:

resources.from _('src/etc')

You can select to copy only specific files using common file matching patterns. For example, to include only HTML files:

resources.include '*.html'

To include all files, except for files in the scratch directory:

resources.exclude 'scratch/*'

The filter always excludes the CVS and .svn directories, and all files ending with .bak or ~, so no need to worry about these.

A file pattern can match any file name or part of a file name using an asterisk (*). Double asterisk (**) matches directories recursively, for example, 'src/main/java/**/*.java'. You can match any character using a question mark (?), or a set of characters using square brackets ([]), similar to regular expressions, for example, '[Rr]eadme'. You can also match from a set of names using curly braces ({}), for example, '*.{html,css}'.

You can use filters elsewhere. The filter method creates a filter, the into method sets the target directory, and using specifies the mapping. Last, you call run on the filter to activate it.

For example:

filter('src/specs').into('target/specs').
  using('version'=>version, 'created'=>Time.now).run

The resources task is, in fact, just a wrapper around such a filter that automatically adds the src/main/resources directory as one of the source directories.

More On Building

The build task runs the compile (and resources) tasks as prerequisites, followed by any actions you add to it, and completes by running the test task. The build task itself is a prerequisite to other tasks, for example, package and upload.

You can extend the build task in two ways. You can add more prerequisites that will execute before the task itself, or you can add actions that will execute as part of the task. Which one you choose is up to you, we’ll show you how they differ in a second. If you call build with a list of tasks, it adds these tasks as prerequisites. Call build with a block, and it adds that block as an action. Again, a common idiom you’ll see elsewhere in Buildr and Rake.

Let’s look at a simple example. Say we want to generate a Derby database from an SQL file and include it in the ZIP package:

db = Derby.create(_('target/derby/db')=>_('src/main/sql/derby.sql'))
package(:zip).include db

There’s nothing fundamentally wrong with this code, if that’s what you intend to do. But in practice, you don’t always run the package task during development, so you won’t notice if something is wrong with this task when you build. For example, if it fails to generate the SQL file. In addition, the package task runs after build, so you can’t use the database in your test cases.

So let’s refactor it. We’re going to use the variable db to reference the file task that creates the database, and make it a prerequisite of the build task. And use that same variable again to include the database in the ZIP package:

db = Derby.create(_('target/derby/db')=>_('src/main/sql/derby.sql'))
build db
package(:zip).include db

Much better. We’re using the same task twice, but since we’re using Rake here, it will only execute once. In fact, it will only execute if we don’t already have a Derby database, or if it detects a change to the SQL file and needs to recreate the database.

Derby.create is not part of Buildr, you can get derby.rake here.

Here’s another example. We want to copy some files over as part of the build, and apply a filter to them. This time, we’re going to extend the build task:

build do
  filter('src/specs').into('target/specs').
    using('version'=>version, 'created'=>Time.now).run
end

The build task is recursive, so running buildr build picks the current project and runs its build task, which in turn runs the build task on each of its sub-projects. One build task to rule them all.

Cleaning

The build task has an evil twin, the clean task. It’s the task you use to remove all the files created during the build, especially when you mess things up and want to start all over.

It basically erases the target directories, the one called target, and if you get creative and change the target directory for tasks like compile, it will also erase those. If you decide to generate files outside the target directory and want to cleanup after yourself, just extend the clean task.

For example:

clean { rm_rf _('staged') }

The rm_rf method deletes the directory and all files in it. It’s named after UNIX’s infamous rm -rf. Use it wisely. This is also a good time to introduce you to FileUtils, a standard Ruby library that contains convenient methods for creating and deleting directories, copying and moving files, even comparing two files. They’re all free of charge when you use Buildr.

Continuous Compilation

And if all that weren’t enough, Buildr also offers a time-saving feature called continuous compilation. This feature, implemented by the cc task, instructs Buildr to loop eternally, polling your project’s source directories for changes. Whenever a change is detected, Buildr immediately triggers the appropriate compilation step and goes right back to polling. This allows you to reap many of the benefits of an incrementally compiling IDE like Eclipse without sacrificing your favorite build tool.

To get started, simply invoke the cc task at the command prompt:

$ buildr cc

This task will immediately invoke the compile and test:compile tasks on your project if necessary. This ensures that your build is completely up to the minute before polling is initiated. After this initial build (if required), Buildr will print a notification indicating which directories are being monitored. By default, these directories will include any source folders (e.g. src/main/java/), any test directories (e.g. src/spec/scala/) as well as any resources (e.g. @src/main/resources/). The Buildr process will remain running during this time, meaning that in order to test this functionality, we will need to open a new shell:

$ touch src/main/java/Test.java

The moment we run this command, Buildr will detect the change and invoke the compile task. It will not invoke the test:compile task, since none of the test files were actually changed. This ensures that potentially time-consuming tasks are avoided if possible. Note that, unlike the build task, the continuous compilation also does not actually run any of your tests. Continuous compilation is designed to be a simple daemon which runs forever, quickly recompiling your project as soon as you save or delete a file. We can terminate the continuous compilation task by pressing Ctrl-C. Left to its own devices, the cc task really will loop forever.

There are several advantages to continuous compilation. Number one is convenience. Once you invoke the cc task, you can focus exclusively on the code, editing and saving your files in an unbroken workflow. There is no need to break your concentration to invoke Buildr manually unless you need to run the test suite, deploy the application or anything beyond compilation. The second advantage is speed. By using the continuous compilation process, you avoid repeatedly incurring Buildr’s startup overhead. While this startup time is kept to a minimum, it is still perceptable, particularly when running on JRuby. Since the cc task runs within a Buildr instance which has already been started, there is no need for repeated, inefficient load times. Again, this allows you to focus more completely on what’s really important: the code.

By default, the cc task will poll your sources once every 200 milliseconds. We have found that this frequency strikes a nice balance between CPU load (which is insignificant) and nearly-instant detection. However, you may wish to tune this value based on your own needs. To do so, simply use the cc.frequency property in your project definition:

project 'foo' do
  cc.frequency 1.5      # poll every one-and-a-half seconds
end

If you find that the 200 ms default imposes too much overhead, try changing cc.frequency to a higher value. On the flip side, if you find that you’re waiting too long for changes to be caught by the poll, tune the frequency lower.

Now let’s talk about the artifacts we mentioned before.