Sunday, June 26, 2011

Ant, Gradle, Maven, Build Software and Processes

Still pondering all my options for a build solution for a particular project but these are my thoughts going into further testing and proof of concept for a new build and deploy solution.

What build solution you choose really depends on the requirements of your particular project, security concerns and the staff you have available to support the solution.

I find Ant simple to use and manage. I like the control of what is getting into my code when and how. Because I put it there.

For all these dependency management issues addressed by Maven and some other build solutions: I think it is not difficult for experienced programmers to find and download a jar file or source code, though I think some companies and organizations providing software libraries and open source software could make this more straight forward and manage their repository of code and libraries better.

Tangent: And when getting source from the Internet not require SVN to download source (dislike). Make it easy for me to download a zip. It's a lot faster and I want a stable version that's not going to change the next time I download it. I also don't want those .svn files in my source if I don't want the software changing on me or to upload changes to your repo.

That being said large companies managing what software their developers include in production software have other concerns. These companies should have a process for requesting and downloading new libraries off the Internet to a central repository so they know what is being deployed to production systems, regardless of what build software they are using. This process should include making sure the code is from a trusted source, checking downloads with provided security keys and possibly even compiling open source libraries to make sure no rogue code has slipped in, and/or (crazy thought I know) removing questionable code in these libraries which is not needed and may introduce security problems (but let your dev's know so they don't try to use features that are not available).

You may not have these next problems with Maven, if working on a project that is small and does not have a lot of dependencies or using an internal repo at your company where Maven is not allowed to access the Internet (but you might in any case). My experience with Maven on complex projects has in general, been a lot of overhead managing things even though Maven is supposed to make things simpler. Maven's plus is that it handles dependency management by going out and grabbing all the libraries you need off the Internet or internal repo for you. Yes I know it does other stuff too. In my case, it seemed like I ended up with a bunch of compile errors to fix every time I turned on my computer. On a recent project, Maven kept updating things when it seemed like dependencies should be stable because I wasn't changing versions. This bothered me. As did having to re-fix my code every time I turned on my computer for no apparent reason. I think when you know what you're doing with the dependencies, Maven can actually make things more complicated, time consuming, and less secure. I stopped using Maven for that project and have been much more productive ever since.

Also with Maven, there are security concerns with pulling random code from the Internet (unless your company has a well managed internal repository which is well managed and secured). If you don't believe that last point, please don't work on any type of financial application or anything that stores personal private data or store anything important on the machine you dev with or let it on a network where valuable data is stored. Thanks.

I know a lot of system administrators use various scripting languages to manage various things and may not prefer an XML configuration file for managing builds. However when it comes to a build system I would prefer separation of execution from configuration. Like MVC separates the model from the view, the steps to run and files to move are in XML while the programmatic execution is in Java. Personal preference, but separation of duties can make things cleaner, easier to manage, less error prone, and more secure as we have learned from MVC.

I'd also rather work with languages I and a lot of other dev's know: Java and XML, vs. learning another new scripting language specific to a particular build software solution. That makes it hard to find employees to support that system, and less people in the organization knowing it who can help fix problems. I also don't want to have to extend (rewrite) the build system to support Java when other solutions alrady exist that meets my needs.

PS as you can see from my resume I have learned a lot of new languages besides Java over the years. This is why consideration for when a new language is really needed and the pros and cons of each is important to me: WebDatabaseProgrammer.com

As for the need to customize, if you're looking for something Ant can't do, I don't think it's that difficult for an experienced Java programmer to figure out how to add new classes that can perform tasks under the Ant framework, or customize Ant itself. I have customized open source code that didn't do exactly what I wanted. It's not that tough. Download the source, grab the dependencies and edit the Java as needed. Just like any other project. If you intend to get updates however, keep your custom code separate from the open source library you downloaded, using and appropriate design pattern.

It seems to me that using a universal language that many developers know should cover whatever the build system needs to do. Java + XML can do pretty much anything you can imagine. I think rare would be the project where Java couldn't get the job done well enough. Of course adding new Java tasks requires compiling code but it also adds some control to what your build system is allowed to do and universal logging can ensure you can audit what your build system is doing, regardless of who edits the XML file. You can add a layer of security also based on who can edit the Java classes and who can edit the XML.

I actually like that variables are immutable once the Ant build process starts. This ensures your variables aren't changing mid stream and avoids tricky errors related to variables changing in random places during the build. XSL transformation works the same way and it took a bit of getting used to, but now I like it because once you get to the point of transforming, or building, certain things should be set.

In general, your build system is probably one of the most important places you don't want to take security risks. That's where you can audit everything and make sure nothing unintended gets into your production software. Whatever build solution you choose, I recommend making sure your processes for updating your builds are secure and audited, including database updates, compilation, and deploying all types of files. It doesn't matter how much development and testing you do on the source code if someone can alter it when it gets deployed and after everyone's done scrutinizing it. Make sure this process is secure.