Building an Enterprise Repository with Artifactory

*The content of this blog is a translation of a blog posted in Portuguese by Diego Pacheco.*

I clearly recall experiencing DLL hell while working predominantly with Microsoft products. We suffered from dependency issues back then and we still suffer from them today.

When I started working with Java I suffered from a similar development concern. The specifics are a little different, but the essence of the problem is the same – dependencies. Java developers know this experience as JAR hell.

It’s amazing to me that companies still struggle with these issues in 2010. There are good solutions for solving dependency problems. Two such solutions are Apache Maven 2 and Apache Ivy. Both these Apache solutions provide dependency management.

It’s common to see a folder called *lib* in the classpath of Java applications containing more than 100 jars. Often, 70% or more of these jars are completely useless, never used, never have been touched by the system ClassLoader, or will seldom ever be accessed by any system or user-defined ClassLoader. Still, they remain in the build. But why? Why leave so many many jars to bloat the application? For two reasons:

1. Dependency Problems: Everyone has probably seen the famous ClassNotFoundExceptionat one time or another. Often, this problem is solved by copying all the jars from the application server and framework distributions we use to the good old *lib* folder. Of course, this is not an elegant solution, and it is not a good solution, but is widely used because it is easy to implement and it saves time. The problem you get to live with is that you’re hoarding a bunch of jars you don’t need and the size of your application will swell.Note: The more robust application servers like Websphere for instance, let you create shared libraries. You don’t have to pack all the jars with your distribution (i.e. ear, war, jar) because you can let the server put them in the classpath.
2. Ignorance: Many companies still don’t use any dependency management solutions. As a consultant I‘ve seen this a lot, in Brazil and abroad. Maybe some companies believe their developers are doing great without some fancy management solution. The rest of us will be investing in dependency management solutions. Personally, I like either Maven or Ivy for dependency resolution.

• Transitive dependencies: making it easier to find those elusive dependencies
• Versioning: Using the “dump it in the *lib* folder” approach there is no control over which version is needed or which version is in production or development. A dependency management framework provides you with fine granularity control without going mad.
• Automatic downloading: With a dependency management solution, you don’t have to search for dependencies, download them, and put them into the classpath of your application. This is a huge advantage.
• Versions update: Here’s another big headache handled for you, especially if you have several applications that make use of a jar that you develop internally. Good dependency management keeps you from going over every application to perform version changes by hand. This will save you some real hard work and a huge amount of time.

I hope that you are already convinced that having a dependency management solution like Ivy or Maven is a good idea. Maybe you use one or the other already.

Often, companies that use Maven or Ivy forget to set up a good corporate repository solution behind it all.


Building a corporate repository with Artifactory

I’ve used Java corporate repository solutions such as Nexus and Archiva in the past. Recently I use only Artifactory and I’ve never had a reason to look back. Artifactory is a very elegant solution to dependency management. The main issue I had with Archiva is that it downloads and resolves dependencies rather slowly.

Firstly, I recommend Artifactory because it does a much better and faster job of downloading and resolving dependencies.

Secondly, I heartily recommend using Artifactory because it provides a comfortable web UI for the management of your third party jars (solutions and plugins) like Spring, Hibernate, JBoss, as well as managing those that are produced in-home by your company. This separation can be done using Artifactory’s repositories.

Thirdly, I strongly recommend Artifactory because it provides separation and control of the many versions of development and production material. Most of the time people use Artifactory only as a proxy solution that helps you save some traffic from your company’s bandwith, but it’s actually much more.


Dependency Control

Artifactory allows you to control the permissions of users and groups. You decide which applications can consume or use certain repositories and jars. For example, you can restrict the use of Spring only for some projects. I wouldn’t do that 🙂 but it can be done. Artifactory also provides you with a much better way to manage your internal dependencies and the solutions you write by letting you limit and centralize access to those dependencies. You may never miss a dependency again.


M2 and Ivy Consolidated Repository

The figure on top shows that by using *virtual* Artifactory repositories you can serve both Ant-Ivy and Maven clients. The coolest thing is that you can serve both clients from the same repository.

How is this possible? Because Artifactory is flexible.

Artifactory works under the Maven 2 dependencies standard. This standard includes:

• GroupId: the group of the solution, macro grouping of a module, or a providing company (e.g. org.springframework)
• ArtifactId: represents or identifies the available / consumed artifact – could be the name of the jar like in our old *lib* folder (e.g. spring)
• Version: the version of the jar (In Maven when we see something like 1.0, normally it indicates that the solution is a production release and if available, you should use GA. When a solution is under development, a SNAPSHOT version is used; sometimes followed by the date and time of the last build.)

With Ivy things work pretty much the same in the attribute organisation maintaining a respective equivalence to Maven 2:

• Organisation: similar to Maven GroupId
• Module: similar to Maven ArtifactId
• Revision: similar to Maven Version

The big difference between the two Apache Dependency solutions is that Ivy uses a regex-style pattern for dependency resolution, and Maven uses a predefined pattern for dependency resolution. It’s a fact that Ivy gives you more flexibility, but also demands more work. What I suggest is to use the same Maven pattern for Ivy, so deployments to Artifactory will be consistent. This way you’ll be able to use the same repository for Ivy and Maven.


Configuring Ivy to download and deploy to Artifactory

To accomplish this, all you have to do is use this XML file I’ve named ivy-artifactory-settings.xml:

1<ivysettings>
2    <settings defaultResolver="public"/>
3    <credentials realm="Artifactory Realm"
4                 host="seu_host_do_artifactory"
5                 username="admin_user" passwd="admin_password"/>
6    <resolvers>
7        <ibiblio name="public" m2compatible="true"
8                 root="http://seu_server:8080/artifactory/
9                 seu_proxy_repository"/>
10        <url name="publish_artifactory" m2compatible="true">
11            <artifactpattern="http://seu_server:8080/artifactory/
12            seu_repositorio_release_repository/[organisation]/
13            [module]/[revision]/[artifact]-[revision].[ext]"/>
14        </url>
15    </resolvers>
16</ivysettings>

Do not forget to use the pattern [organisation]/[module]/[revision]/[type]s/[artifact]-[revision].[ext] to resolve the dependencies. By the time you are publishing your jar via Ivy, you can use a task to make the Maven POM based on Ivy’s dependencies. It should be something like this:

1<ivy-makepom ivyfile="${ivy.xml.file}"
2             pomfile="${basedir}/dist/${ivy.organisation}/
3             ${ivy.module}/${ivy.revision}/
4             ${ivy.module}-${ivy.revision}.pom">
5    <mapping conf="default" scope="compile"/>
6    <mapping conf="runtime" scope="runtime"/>
7</ivy-makepom>

Taking into account that you created your jar in the dist folder and that it follows the Maven 2 pattern, you will notice that it’s located within the hierarchy of organisation, module and version folders and that the jar is tagged with the module name and the version.

Consuming and publishing your artifacts using Maven shouldn’t put you to any trouble, since Ivy now follows the same pattern as Maven.

So no matter if the project uses Maven 2 or Ant with Ivy, by using Artifactory and following the patterns and tips from this post, you will be able to use the same repository for everything.

Best regards until next time.