Optimizing Developer Builds Performance

Software developers are always looking for a faster way to build their code. The answer to faster builds is surprisingly simple, but it does depend on the type of build.

Two very different types of builds are documented here.

Building the GlassFish V3 application server

The GlassFish V3 application server is an open-source Java EE enterprise server consisting of a large number of modules. Its build system is complex, based on the popular maven 2.0 environment. Yet it is more or less brain-dead in terms of exploiting system resources: it runs single-threaded for most of its invocation, though some internal threading does occur, making dual-core systems worthwhile.

I tested the GlassFish V3 build as of mid-December 2008. I did an initial build, which downloads all dependencies from internet maven repositories. I then repeated the following command five or six times, to verify consistency in results: "mvn clean; mvn -o install". The "-o" flat tells maven not to access the network; it builds locally.

The systems were clones of each other, and both used the same model Hitachi 320GB hard drive, so the runtime environment was nearly identical.

Results

The results are striking: the fast Hitachi 320GB hard drive was just as fast as a RAM disk, and faster than any external eSATA solution. This is not too surprising, given Mac OS X caching, and the fact that builds do lots of small I/Os, which are not efficient for eSATA on the MacBook Pro, and do not benefit from the striped reads or writes from a RAID (32K stripe size).

Bottom line

(a) So long as the internal hard drive remains fast (not too full), builds will run at top speed without any special effort; a RAM disk (risky in the event of a crash) offers no meaningful help.

(b) External eSATA does not help and a striped RAID actually degrades performance.

(c) Build speed scale closely with CPU clock speed.

* Single 1TB Western Digital RE3 via Sonnet SATA Pro, FirmTek SeriTek/2EN2 Details.

** Striped dual 1TB Western Digital RE3 via Sonnet SATA Pro, FirmTek SeriTek/2EN2 Details.

The MacBook Pro 2.8GHz (2008 unibody) showed a 22% performance advantage over the previous generation 2.4GHz model. But this is not very impressive: its clock speed is 17% faster, accounting for most of the difference; its faster memory and larger CPU cache must account for the rest. It means that a previous generation MBP at the same clock speed would perform at effectively the same speed, so get a refurbished at a huge discount while they last (late 2008/early 2009).

Building with Apple’s XCode

XCode is Apple’s integrated development environment. It is highly efficient, using all available CPU cores and fully exploits a quad-core Mac Pro at 400% CPU usage (an 8 core system would likely be fully exploited as well).

For this test, I built a universal binary DiskTester (32/64 bit for PowerPC and Intel), which means the entire application is built 4 times for the four combinations.

Bottom line: XCode scales linearly with the number of cores! Fast builds means as many CPU cores as possible. Build speed also scales ~linearly with the clock speed, but that is of minor importance relative to adding even a single extra core.

* MacPro is 2006-vintage quad-core 3.0GHz

Conclusions

For the GlassFish V3 build, the dominant factor is CPU speed, multiple cores are of little relevance due to the inefficiencies of its maven-based build.

For XCode, the dominant factor is the number of CPU cores, with CPU speed a secondary factor.