Use Case – Astrid: Artifactory-Sourced Dependency Insight at Netflix

Talk Transcription:

[John] All right. My name is John Schneider.

[Nadav] I’m Nadav Cohen.

[John] And we are from the

[Nadav] developer

[John] productivity

[Nadav] team

[John] at

[Nadav] Net

[John] Flix.

[Nadav] Cool.

[John] You have no idea how much we practiced that. It’s like, this is about as organized as our day gets at Netflix. And it’s all because Netflix is a freedom and responsibility culture. What does that mean, Nadav?

[Nadav] Well, in our context freedom and responsibility means that every team can pick their own tooling, their own languages, and their own release cycles.

[John] And although each team can kind of do their own thing, here we are as a central developer productivity team that’s trying to provide good information and insight to every team at Netflix. And help them ensure, you know, maximum productivity and minimize their time to delivery.

[John] So, as far as we see it, there’s two ways to really integrate your code base. You can either do source level integration. Source level integration sometimes called build from head or mono repo. And there’s some really successful organizations doing that. You know, Google and Facebook and Twitter are all doing build from head on some level or another. But they do so with large engineering organizations and – and often a huge engineering effort.

[John] I think Google has hacked, you know, the filesystem to make their build from head system perform well and Facebook has hacked Git itself to make their build system perform well enough.

[John] The other alternative is binary integration where every team publishes binaries to a central artifact repository and we integrate that way. And Netflix does binary integration. So we’re a relatively large organization but there’s only a few of us in developer productivity so we’ve chosen binary integration.

[John] What we’re here to tell you today, is how we deal with some of these binary dependency issues. The most common ones we see. We’re going to give you some solutions for how to solve those. Fair warning. This is a JVM based talk so we’re going to give you solutions there. We’re going to give you solutions in Gradle which is our chosen build framework for the most part. But, you know, similar solutions can be can be written in other tool stacks if you chose to do so. And with that we’ll start with the first problem. Nadav.

[Nadav] All right. Just to add on top of what John said, the only relevant piece of information here in this slide is that we have all the code that we’re about to demo on GitHub. So you can access it there and follow along if you want to.

[Nadav] All right. And with that, we’re going to go to the first example. Basically we’re going to demonstrate what happens when you put duplicate classes in the same class path in Java. So for this example I’m gonna basically build a command line application that prints out the power set of a given set. All right, so first thing I want to do is I want to add some libraries. I don’t wanna reinvent the wheel, right, and I’m also kind of lazy. So I’m going to go add Guava which is a Google library. And I’m also gonna add Google collections.

[Nadav] All right. Now that I have my dependencies in place, I can actually write the code that prints out the power set. I’ll go ahead and do that.

[Nadav] All right. So assuming I didn’t get anything wrong, this should build in Gradle. All right. I can also check my dependencies. I see that I have both Guava and Google collections. And then I can also run it. If I run it, you can see I get the power set of the sets successfully.

[Nadav] Now here is the problem. The particular thing about Java class loading is that order is not guaranteed. In other words if you have two classes in the class path, Tomcat, for example, might choose to load A and then B whereas Jetty might choose to load B and then A. All right. And this becomes a problem when you have duplicate classes, or multiple definitions of the same class on the class path. Eventually you have Java just picking one arbitrarily and tossing out the rest. And you, eventually, you know, you just don’t know what you’ll end up getting at runtime. Which is of course a problem. Right? Everything passes compilation, it’s great, we ship it out to deployment, and then everything, of course, breaks.

[Nadav] So this is not a problem that’s specific to Gradle but basically I’m going to use Gradle to show you how a very simple change in the ordering can cause this problem. Right?

[Nadav] So I’m gonna go to — to the Gradle file. I’m gonna change the order and then try to build again. Build is successful, as you’d expect. And then I’m going to check the dependencies. Oops. All right and again the same dependencies. Gradle didn’t actually recompile because as far as Gradle knows it’s exactly the same code, it’s actually the same dependencies so everything stay the same. But when I go ahead and run, we get a runtime exception. Right. We’re getting a no such method error. So what actually happened here was that our code dependent on a very specific method on the set’s class. Right? From Guava. But because we used Google collections first in Gradle, that one was loaded first and because this one has the same set class with different functionality but the same set class. It basically masked away the class we wanted to use and cause this runtime exception. Right. Which is, of course, a problem.

[Nadav] So what we did internally at Netflix to address these kind of things was to basically see where these, kind of, duplicate classes problem can happen. In this case it happened because Google collections in Guava, as I’m sure some of you already know, is in fact the same library. Right? And that is why they share the same code. Gradle, unfortunately, doesn’t know this because different naming, different dependency means they both can live happily together. So we built a plugin, internally at Netflix, called Resolution Rules which helps Gradle understand when dependencies are, in fact, the same and then can do conflict resolution accordantly.

[Nadav] Right. Let me go ahead and add the plugin just to show you how easy it is.

[Nadav] Nebula. And then I’m going to go and add the rule itself. In here I’m saying to the plugin, every time you see Google collections, I want you to replace it with Guava. Because it has been renamed to Guava and also keep track of who made the rule and what day, in case I ever forget. We know that might happen. Right. Going back to the build, I just wanna include that rule. And hopefully get everything in correctly. This will build and run. All right, build passes successfully. I run dependencies. And now you’ll see something has changed. Now instead of having Google collections and Guava, I basically set up Gradle manage Google collections to Guava. And once it saw that it’s actually the same library it conflict resolved it to the latest 19 which is exactly what I wanted in this case. All right. And when I run the same program, you see that the power set works as expected despite the fact that I have multiple declarations of conflicting libraries. All right.

[John] So this was a somewhat contrived example. We introduced two first order dependencies on things that are known to be the same thing. But in reality this has really hurt us in the past year. This kind of problem, the duplicate class thing. Specifically when we moved from Java 7 to Java 8 as an organization, one of the libraries that caused the most pain was ASM. The version of ASM, the last version of ASM that supported Java 7 by code had a group named ASM, an artifact named ASM. The first one that supported Java 8 by code had a group named org dot OW2 dot ASM and then an artifact named ASM. And so as we’re trying to make this transition from 7 to 8 we’re having to try to push out the old ASM, ASM version, and not be able — we couldn’t rely on our build tools conflict resolution mechanism to do that for us. So these things can really, really be painful in practice and it’s helpful. This is a helpful rule as a result. On to the next one?

[Nadav] Right. Does anyone have any questions? Specifically now before we continue?

[John] Any questions on this specific problem?

[Nadav] I think there’s someone there.

[John] Okay. Is there?

[Nadav] I’m not sure.

[John] Speak up if you are. Or otherwise. We’ll go on. All right. Go on.

[Nadav] Okay so the next problem we’re going to show has to do with bundled dependencies. Basically what I’m going to do is I’m going to build another command line utility but this time I’m going to use slf4j to log things onto the console. All right.

[Nadav] All right. So first thing I want to do is I want to add the dependencies for slf4j. Just have it ready. Let’s check what this actually brings to the table. So I can run Gradle dependencies. And I see that this basically brings three dependencies. The first one, as you’d expect, the logging implementation. The second is the API for slf4j. And then the bridge between them two. Okay.

[Nadav] So I’ll now go ahead and write the code to actually write to the console. So.

[Nadav] Sure everyone has to do this on a daily basis. Okay. All right. If all goes well, this should run. Perfect. So we do get the message from the logger. Now I want to make this a bit more interesting. I want to use my own library that I’ve been building in the past couple of weeks, which basically returns you a random quote from a Netflix original. I don’t know how – how useful that is to everyone else but it’s super useful for this example. So I’m going to use it anyway.

[Nadav] First of all I want to add the dependency. Okay. All right. And checking the dependencies that came as a result we see that we get the quote service. Great. Can go ahead now and use the service. Okay. And then just run it and see what happens.

[Nadav] All right. So I’m getting, again, a runtime exception because of dependencies and this time, I hope it’s clear to see, it’s complaining that we have two conflicting jars in the class path. One which I know is there because I put it there because I want to use logging, but the other one, I have no idea where it came from. But it’s stopping me from running the app. So because this is Gradle I know I can, you know, exclude the dependencies and I can remove all sort of transitive things. So I’ll go ahead and try to do that. All right. And then I run and I get the exact same exception. Nothing has changed, even though if I look at the dependencies I see that it’s still the same. Let me try to even remove all the transitives. See what happens. Same thing. So what actually happened here and again maybe some of you have noticed this, this is what we call a bundle dependency.

[Nadav] A bundle dependency is basically a simplified delivery mechanism for jars. Essentially the library producer just embeds all of the dependencies within the same jar, making it super easy to actually produce and consume, right. You just get one jar. But then you also lose all the information around dependency. Right. So in our case because we actually wanted to remove some of the transitive dependencies we just couldn’t because that information is lost.

[Nadav] So now that we know what the problem is, the next step would be to make sure these kind of bundled dependencies don’t appear in our build. Again, we can use the Resolution Rules plugin to do exactly that. You just fail the build if that – if that specific bundle ever appears in the class path. All right. So again I’m going to add the plugin. Okay. And you’ll notice by the way that I’m adding the Resolution Rule inside of the dependencies block. That allows for a very interesting thing that we actually do internally at Netflix which is to externalize the rules for everyone else. Everyone else who feels like this is a rule that could be useful for them can use that dependency internally in their build and benefit from, you know, our collective knowledge and our collective experience of all the type of dependency problems that might occur.

[John] To add to that we have roughly five thousand builds or so at Netflix so there’s about four of us that work on, that respond to problems related to those. So you can pretty much guarantee we’ve seen like a widespread common dependency problem. So this collection is catalogue of resolution rules should benefit many organizations.

[Nadav] All right. Thank you. Okay, so in this case we’re using the deny rule. Basically says every time you see the quote service bundle, just fail the build straight away saying that this is a bad bundle that we know wraps this bad library that you don’t want to use. And then we can also point them to maybe a wiki entry that explains further why this is a problem. Right. We’ve seen this again and again. As John mentioned, you know, we support everyone’s build environment and this is a problem that just resurfaces time and time again. And so this could be super helpful from a self-service kind of perspective.

[Nadav] Again we have the author and the date. In there just to keep track of who’s built that rule and why and when. And then let’s try to run it. All right, now we get an exception but now it’s a very explicit exception. Right. The build is failing because we have the wrong, we have a really bad bundle that we just want to get rid of. So now that we know what the problem is we can fix it. Let’s remove this. And use the non-bundle version. Okay. And now we can check the dependencies. Now that we’re no longer using the bundle package and the dependency information is actually there, we can see what’s actually been brought in. Right. And this is the problematic package that we’ve saw before.

[Nadav] But now that we have this information, we can also say transitive equals false, and just get rid of it all together. Right. And now we get only the service that we need. And once we have that in place, we can just do Gradle run and get the quote. I think this is from Jessica Jones specifically. All right. That’s it.

[John] So great. This is a — these are the two demos we wanted to do of Resolution Rules today. We’ve got more to follow. Are there any questions on Resolution Rules and what it could potentially do for you before we move on to the next thing? Yes.

[Audience] So, could someone just pack all the Resolution Rules together on their project and hold the NLP libraries […] they are using are conflicting pretty much, right. Because you would know initially that which package can […] with the other packages in their project. So.

[John] Yeah.

[Audience] After you have […] rules, […] can combine them together as a super set of all the rules?

[John] We will — we’ll show you something a little later. We have a Gradle Lint demo coming up and we’ll show you how we collect information on what potential problematic packages may exist and then how — we can build rules accordantly.

[John] But I did want to give you a concrete example of the bundle dependency problem. The one that we’ve encountered the most often is the Jersey bundle. It’s actually kind of a demonstration of both problems we’ve talked about so far: the duplicate classes and the bundle dependencies. Jersey bundle just packages up all the available Jersey libraries into one and, you know, in cases where somebody’s tried to depend on a specific Jersey core specific version but also have Jersey bundle. It results in like version mismatches between those two packages. And depending on which order they get loaded in the class path they can blow up in really unexpected ways. So that’s — that’s the most pernicious one. We’ll show you the scope of impact that has on Netflix here in a moment. But.

[Nadav] Sure. I think maybe another part of the question was do we actually have, you know, those kind of rules available for everyone else to use, right. So yes, the open source version of Resolution Rules actually has quite a bunch of rules that are, we think universally useful for everyone else. So I mean if you have, if you want, please feel free to check out the GitHub project. You’ll see all the rules in there.

[John] Yeah. Great. Any other questions on Resolution Rules? Yes. There’s one back there.

[Audience]

[John] Right now this is a — this is a Gradle only solution because it uses a Gradle resolution strategy and so forth but again I think the kinds of solutions we have here could be reimplemented in other build tools if that were helpful.

[Audience]

[John] We do have the freedom and responsibility culture, so of course any team could choose a different build framework if they choose to. And there are some that are on Maven or elsewhere. SVT?

[Nadav] Some were using Node ones.

[John] Yeah. But we try to provide, you know, a paved path like the, you know, if you accept this paved path, you’re going to get the most support and the most — in the easiest way. And so the vast majority are Gradle builds at this point.

[Nadav] Just because it’s much easier to use something that has been, you know, prepared in advanced and really works well with the Netflix environment.

[John] Yeah.

[Nadav] Now, one other thing to add if you’re using Maven, there is the enforcer plugin that does something — does work very similarly to the Resolution Rules plugin so you can try and use that. If you’re using Ant, well good luck.

[John] Great.

[Nadav] All right.

[John] Well. The next thing we would like to show you is –

[Nadav] […]

[John] — Yeah, you can just go to the browser, I guess. Great. So here’s an illustration. We have a tool called Astrid which is an internal dependency insight tool at Netflix.

[John] I think you’ve heard in the keynote, a demonstration from JFrog, of a tool that’s very similar to this, actually. This — one of Astrid’s chief functions is, if I’m a library producer and I want to know what the scope of impact is on the organization. That is who’s depending on me, both directly and transitively? I can use Astrid to learn about that.

[John] So specifically we just mentioned that Jersey bundle’s a big problem. We’ve known about this for a while. We’ve introduced Resolution Rules very recently but in spite of the fact that we’ve — you’ve kind of shopped around this idea that you don’t want to use Jersey bundle, it affects people downstream to you and so forth. Today if you looked at the scope of impact of Jersey bundle in our organization, we just search for any use of Jersey bundle here, you’ll see that it exists in, you know, over a quarter million library versions produced, you know, deployed internally and this is affecting — Jersey bundle exists in close to seven hundred Amazon server groups, actual deployments out there in EC2. So this is just one of those illustrations of how widespread, if this – if the Jersey bundle was introduced at a relatively low level, it’s gonna percolate throughout the graph and affect, you know, a large number of people. We expect with things like Resolution Rules this number will drop significantly here in the short term.

[John] Other things that Astrid is capable of doing. It’s kind of a, you know, motley crew of a lot of different features. We use it to scrape metadata from Artifactory, commit hashes and things like that, and generate a GitHub style release notes page for a particular module. So if, you know, the producer of platform-ipc, which is one of our internal jars, is not being diligent about providing change logs to its consumers, people can come here and kind of see what’s changed in the most recent versions. Specifically if they have a problem they kind of look here first.

[John] And that’s Astrid, so the — the other things we’re using Astrid for or going to soon, we find the Artifactory cleanup story to be particularly hard at Netflix. If we try to create a rule, like, let’s just delete snapshots that are older than 90 days, that’s really hard on us because all the teams are on different release cycles. If we delete a snapshot that’s 90 days old, it may be included in a candidate that’s 30 days old, than a release that’s been released this week. So we don’t want to delete things that are actually in production. Using Astrid since we’re tying together, piecing together information all the way from binary artifacts up to and including the ASGs they are actually deployed in AC2. We know exactly what’s being used in any given time and we can purge the old stuff after that. It’s a good use there. Really interested to see what JFrog comes up with. I think it’s very similar.

[John] Any questions on Astrid or what’s it capabilities are? What your use cases might be cause we were considering open sourcing this so this is kind of an interesting — interesting position that we find ourselves in. No?

[Audience] A question.

[John] Yeah.

[Audience]

[Nadav] That’s a good question.

[John] Yeah. The question was how do you figure out what projects are dependent on a particular dependency. And so what Astrid’s doing is actually ingesting the dependency descriptor like a ID file or a Pom. And so we just have just a massive amount of data in memory about what all the first order dependencies are and then we can walk that the other direction, doing a simulation of latest version conflict resolution in the process and working that way. We rip open Debian files much like I hear JFrog is doing. Look at what dependencies are inside there. And go all the way up to the deploy […] that way. So, yeah.

[John] Any other questions on Astrid. Yes, sir.

[Audience]

[John] Yes. The question was is this Java only or NPM related. Right now it’s Java only. Historically many of Netflix’s services were written in a JVM based language. Although that shifted recently. I think we’re about, like, 60 percent Java, 30 something percent NPM and then a smattering of other things. And so yes, we do have like a relatively short term feature that’s going after the same sort of insight for Node because it’s becoming an increasing importance internally. So.

[John] All right. Well. The next major feature we want to show you is Gradle Lint and — so resolution strategy. The resolution strategy probably what we showed you already is a little bit difficult for us because as a central build team, we’re imposing, you know, an opinion on all of our teams but we have that freedom and responsibility culture so it doesn’t always feel exactly right. What does feel a little better is if we write a rule, we look at your build file, we write a rule, we surface a violation saying hey this can be a problem and then we provide you with some sort of auto fix rule that if you run a certain command that gonna fix your code right there were you can see it and then you can choose whether to accept that — that violation or not. And this is what we’ve done recently with the Gradle Limp plugin. We’re going to demonstrate some of its capabilities. It’s not open?

[Nadav] Oh. There it is.

[John] Okay. You just have to be up there. There you go. Oh my. […] make the presentation.

[Nadav] Reset.

[John] Yeah.

[Nadav] […]

[John] What’s that? Where are we? All right.

[John] So we’re going to start with a very simple app. And this is again, a little bit of a contrived example. We’re going to start with a very simple app that just contains nothing but a few dependencies references. Here the main method refers to the Amazon S3 Client. It refers to, you know, another type the default AWSCredentials provider. And there’s this nice handy function called something with Jackson that’s never referred to — that refers to the Jackson object mapper. That’s one class that we’ve got. The other one is a test cause, you know, we do TDD, right. And this test is absolutely nothing but referred to junit. And then on the build side we just have three dependencies. We have the AWS Java SDK, the Jackson databind core, and junit itself.

[John] And, so. You’ll notice a few things that are potentially wrong with this. One, I’m referring to junit in the compile scope when really I should of referred it in the test compile scope. As it stands right now, if I’m a library producer and I push this out as a binary dependency everybody downstream of me is going to get junit in their classpath when they probably don’t want it. That’s something we find that happens with an unfortunate degree of irregularity. Also I’m using this AWS Java SDK which is a family jar. It’s actually a zero byte jar. It has nothing in it. It has a Pom file that just contains all the other components of the AWS Java SDK, many of which I’m not using. So this is kind of a convenience dependency I’m adding it but much of what it brings in I’m not using. And then finally I’m referring to Jackson databind here, version two-five-five. But I’m actually, because of latest version conflict resolution, I’m actually getting two-six or above. So we’re going to see how Lint will help us surface these issues and fix them for us. So that we know exactly what’s going on.

[John] First of all we’ll apply the plugin. And similarly to the Resolution Rules plugin we got, you know, basically a way of packing rules in jars, including those jars, those unused dependencies rules when it’s available with Gradle Lint itself. And you can declare either directly in your build script or in some organization wide in a script which ones you want to use. So here we’re saying we’re going to use the unused dependency rule. And when we run Lint, we’ll see that we have — Lint has reported a number of violations. First it’s saying, hey you’re using junit but you’re not actually using junit in your main source. You’re only using it in your test. So we should ship this one over to the test compile configuration so you don’t affect your downstream consumers. Secondly it’s saying the AWS Java SDK is really just an empty jar. It contains nothing so we can strike that because otherwise you’re going to be bringing in dependencies you don’t really need. And then, you know, another thing it’s doing is it’s saying, hey I see you have a first order dependency on the S3 SDK specifically so we want to add that. We want to be declarative about the fact that we require that, and add that to our build script. Similarly the AWS Java SDK core is required directly so we want to add that as well.

[John] And these rules are great but what’s even better is we can just, that at this point, because each of these rules pack with an auto fix rule, we can just run fix Gradle Lint and we’ll see that it actually transforms our build script itself, adding the two dependencies that it discovered that we actually needed from the Amazon Java SDK. It relocated the junit dependency to the appropriate configuration so we don’t affect people downstream of us. And now our dependencies are a lot cleaner. We’ve gone from probably, you know, 30 or 40 or 50 dependencies down to a much smaller set now.

[John] So we’ll add another rule, just one more we’ll — we’ll add the overwritten dependency versions. This is what I was referencing to with the Jackson version here where right here we have a declaration on two-five-five but in fact the Amazon Java SDK is using Jackson as well and it’s using a higher version. We hear a lot of complaints from developers. They’ll put a specific version in their build file and then they’ll wonder why they aren’t getting it. They’re not getting it because it’s being conflict resolved away for some reason or other. And we want to surface that as well. So here if we run fix Gradle Lint you’ll see that it just went ahead and said, okay you’re just actually using two-six-six, we might as well surface that in your build so that you’re not confused about which version you’re actually going to get at runtime. Any questions about what you’ve seen so far with the Gradle Lint plugin. Okay.

[John] So this — this Lint capability is really kind of emergent for us. It’s leading to what we think is going to be essentially an eventually consistent mono repo with binary integration. What we mean is, you know, things like this we can, you know, binary dependency management is hard. We’re not always specifying the right version. We want to make sure people use as few dependencies as possible but are explicit about the ones they do use, they have the right versions. But Lint, we also imagine is going to be used manipulate source code as well.

[John] A great example of this right now, internally we have a — before slf4j existed we were ahead of the curve and wrote we our library called […]. Which did essentially the same thing then slf4j came about. Got to be the ubiquitous answer and yet still, to this day, you know, we have a hundred thousand references to […] floating around. And, you know, it’s relatively easy whenever we see this block of code, we could replace it with this other block of code. And yet, you know, it’s really not a responsible use of our time to go out there and just hunt down every one of these references.

[John] So what we’re allowing, we’re going to very shortly allow our library producers to do is pack rules into their own jars that will manipulate source code, changing API calls, things like that. That’ll just get surfaced in the build over and over again and whenever a developer runs fix Gradle Lint, they accept that change and it gets, it’s worked into the code. So this is like an alternative to the mono repo where, you know, when you — if we decide to go from […] to slf4j we make that refactoring operation across the whole org. Here we can provide a rule and then it kind of works through organically as people run fix Gradle Lint.

[John] All right. I think we’re basically open for questions now. Anything on the Netflix developer productivity team, or anything you’ve seen thus far?

[Audience] So are those available for anybody to use?

[John] Yes. These are all on GitHub. The ones that we’ve demonstrated today, the Gradle Lint plugin, the Resolution Rules plugin, are both out there.

[Nadav] Yeah. Astrid is the only one that’s still internal.

[John] Yeah.

[Nadav] But we’re actually looking for feedback. If this is something that could be interesting to others, we might as well open source it.

[John] That’s right. Yeah. We kind of want to hear what your use cases are there. So.

[Audience] So what impact does this have on the project […] When you put rules, Resolution Rules […]. Does that reflect any of the output of the dependency report?

[John] The question is: Is the Resolution Rules plugin reflected in the output of the dependency report? And this is a really insightful question cause it — there’s two possibilities, either it affected a first order dependency or it affected a transitive dependency. In the case where it affected a first order dependency, let’s say you had mis-matched, you know, AWS versions for whatever reason. That will be represented in the Pom or the IBXM, ultimately.

[John] In the case where it affected the transitive version, it will not. And that’s kind of a stick-in issue for us again cause, you know, potentially if you apply resolution rules and it fixes a transitive, your code could compile and test and everything looks great, you release it, if your downstream consumer does not also apply Resolution Rules, it could appear that your project is dysfunctional.

[John] And so right now we’re just kind of, like, saying well, you know, everybody’s applying it so I should be okay. Ultimately it does affect for a shorter version so it should kind of percolate throughout. But where ever we have a Resolution Rule, we’re probably also going to have a correlatory Lint rule. So that we get to the source of the problem where ever that first order dependency is and fix it there. That’s ultimately the more correct solution.

[Nadav] And just to add on top of what he just said, we also have integration with our metrics plugin which we haven’t mentioned at all. Basically the Lint plugin can talk to another plugin that we have called info-broker which talks to the metrics plugin which eventually sends out metrics to an elastic search instance.

[John] That’s right.

[Nadav] And then we can do all sorts of interesting aggregations and to see, you know, how many violations we have throughout your organization. Is there like a specific team that’s, you know, behind and we need to talk to and our plan is to actually do the same for Resolution Rules as well. So we’ll actually have a centralized place with all the aggregating information about the organization and based on that information we can act accordantly. Right. Maybe write more rules, maybe talk to the teams, it really depends on what we find out.

[John] And a great example of this the way metric helps us is this. The problem that we address with the adaptation earlier, the Guava Google collections problem. Oh it’s unused I need to. Shoot. Let me get rid of unused because it’s trying to be too smart for me here. So this is a rule that we have it’s just a duplicate dependency class rule that looks at your dependencies and says hey you’ve actually got two things here that contain the same class, or in this case, there’s an overlap of over 298 classes between these two libraries. Unfortunately we don’t have an auto fix rule for this, cause I don’t know which one you really want, but it surfaces it anyway so developer’s not going to be surprised by a runtime order, you know, problem later on. And also since we’re shipping all these violations up to our metric collector, we know as a team which, we see right here, okay this was an example where there’s a duplicate dependency class between these two libraries, we’re gonna add a resolution rule for it, we’re going to add another Lint rule for it to take care of the problem. Yes sir.

[Audience]

[John] The question was: Are the rules only managed by our team?

[Nadav] Yeah. So the answer is we have a few centralized rules that are available to everyone. But everything is free to add their own rules if they feel like they can be more strict about the way their builds are — are built essentially. Right. So you just add another dependency with your own rules, your own internal rules and then, you know, you can apply those.

[Audience]

[Nadav] No. They’re not hard coded at all. You just add the dependency with your own resolution rules and you’re good to go.

[John] Yeah. That’s right.

[Audience] So you have this on the Java side, are you going down the same route for NPM also? […] very much a Wild West […].

[John] I think what we’ve noticed — the question is: Are we going down the same route for NPM? And there is a, for the record, there’s a bit in there about NPM being the Wild West too so you want to say your name too, sir. Just kidding.

[Audience]

[John] Okay. There you go. So far, our understanding is that, you know, since NPM has that, like, hierarchal packaging mechanism it’s really not as ugly there as it is in Java where we got that flat class path, and you know, one class is going to win and so forth. We had a lot of hope for Project Jigsaw to solve — to allow us to layer, basically, you know, layer dependencies in hierarchical way in Java. And while they’re providing some compile time […] for that, unfortunately at runtime, the construction of that layer — that layer mechanism is left up to the runtime. So we’re not getting a solution for that with Java 9. And, you know, it’s going to continue to be a pain point for us, and so we had to do something in the immediate — in the present time. Yes sir.

[Audience] So that you’re now talking about Java and NPM, is there a metrics database or […]. Is that going to be a technology that […] or is it pretty much Java specific?

[Nadav] That’s an interesting question. So the question was: If our elastic search database would be language agnostic, right, once we move on to different technologies. And I don’t think — I don’t — we’ve actually talked about this internally yet but I believe the answer is going to be yes.

[Audience] […] open source it. Just got to roll with whatever.

[John] The question is: Will we open source that? Eventually if it’s multi-platform. Right now the metrics collector is open source actually for the Java side — for the JVM side. It’s the Gradle metrics plugin. Which these things tie into. Yeah. Anything we do for NPM that’s a general solution like this, I don’t see any reason why we wouldn’t open source that as well. And we’re just getting spun up on it. So.

[John] […]. I hear […]. Pretty wild guy. Any other questions for us? […]. All right. Thank you so much coming and we’ll be around. Thank you.