Log4j Log4Shell Vulnerability Explained: All You Need To Know

Shachar Menashe: Thanks Courtney. So just a quick introduction, I’m Shachar Menashe, Senior Director of security research at JFrog I basically manage all the security research teams. So obviously this low official vulnerability has been a huge problem and you’d stay on the internet right now. Our security research team has been looking at it since it was reported last couple of days and basically trying to understand all the attack vectors, the fixes, the mitigations, the false positives. There are a lot, there’s a lot of noise and a lot of information about this, more about it. So basically in this webinar, we wanted to aggravate everything in one place so we can give out a concise explanation to everybody and just have vendors understand what is the real impact. Are they actually impacted? What are the available fixes and what are our recommendations for mitigating this issue? So let me jump to the slide, as well as we will have a visual cue as well. Great. So first of all, we wanted to explain about the vulnerability and why it’s so critical, what’s the root cause of the issue, etcetera? So what is the law commercial vulnerability? Well, first of all, it’s a critical remote code execution vulnerability. It affected a lot of for data logging framework for Java. One of the most popular logging frameworks it got assigned to the CVE 2021 for today. It has the highest score of accidental death then, which is always the problematic factor. So what actually happened with this vulnerability that the vulnerability is triggered when an attacker can control a string that gets log into with this logging framework? So the logging framework has a lot of APIs for logging, logging on info like a debug, etc., etc.. So here we’re seeing some example of a log on info, and the idea is if an attacker or some malicious factor can get. A special token inside any of the strings that get get logged, then the attack takes place. This is problematic because usually log strings are not passed in any way and they’re considered to be very trustworthy resources so they don’t need to be validated or filtered or anything. It’s just strings that are logged out. So in this case, the strings are passed and can be doing malicious activity just from having malicious text logged out. So that’s why it’s a very problematic issue. So again, this is critical because of exploiting it is extremely trivial and one hundred percent persistent. It’s not a statistical exploit or anything. There are public exploits available on data on the internet, so basically any attacker can just download it and exploit it, host the server on his machine, his or hers, and then just brute force. Send out these malicious tokens to every web application and every website, every API that they attack, receive and just wait and hope for exploit, exploit exploitation. So it’s extremely trivial to exploit for even novice attackers. The second problem is that this logic framework is massively popular. It’s being used by third party vendors. It’s being used directly by developers. Basically, anyone that is using a job application might use this login framework. There are other frameworks like Log for J-1, which is a previous branch of this, which isn’t susceptible to this exact issue. There’s a log back, which is another popular one, but not for data is one of the top one. So it’s used very it’s very, very ubiquitous. So pages everywhere. And because it also got such a wide media coverage, there’s a ton of attacks happening right now. The third issue is, again, it’s not enough that you have the package installed. You also need that in that some remote input would actually get law like remote input from an attacker. The problem is that it’s extremely likely that untrusted input will reach a logging function because usually when you log something, you want to include parameters from whatever remote location or you want to include what happened. So it’s very, very likely that untrusted input will make its way to one of these logging functions. Of course, we also have a companion blog post which would be sending the collateral material, and we specify an exhaustive list of all the logging functions that you need to look out for. But it’s it makes sense that in some way, untrusted input will reach a logging function. What we saw online is that the most common attack vector is actually including this attack string in the user agent. When doing it should be a request because usually on the server side, the logger will just say, what was the user agent that did the request? And then once that gets logged in, the attack happens immediately. So that explains a bit about the vulnerability itself, but not what caused it. So we’re trying to keep it a bit technical, but still high level. So actually, this vulnerability is extremely interesting because I would say that it’s not a bug in the programing. It’s actually just a feature that was added quite a while ago, and nobody thought of the security implications of this feature until now. So actually, there’s no bug to exploit here, but it’s just a feature that in this case allows the more remote code execution. So the type of the bug is the unsafe class, the certainly the serialization, a very well known type of Java bug. This is caused due to a Log4J feature that’s called message lookup. So basically, the when you log out a message usually in every logging framework, when you log from the Gougeon doesn’t get passed, they’re not just logging it out verbatim, but in a lot more data, though there is a feature that’s enabled by default in the older versions that just looks for specific tokens in the logging message that replaces them. So example if I would log this token like a dollar sign in the curly brace Java Kotlin runtime, if I send this out to be logged, what would actually be logged with the Java version one seven zero 067, so it gets replaced automatically? And this is by default, so, so usually this is harmless. It can actually be harmful because of information leakage. But the bigger issue is that there’s actually a specific type of lookup called the Java. I look up and what this lookup does it load the remote class using Jamii, which is a directory service for specified classes, which can be either local Java classes or a remote one. So this look up allows loading a class and then outputting a specific variable from the class, outputting the value of a specific variable from the class. So the problem with this, this is, first of all, that the class can be hosted remotely. So you can have a stranger that accesses a different machine, like a different computer to retrieve the class. And it’s not just classes that are stored locally on the machine in your class file. And the second problem is that in order to evaluate the value of the variable, the class needs to be the serialized, which means the class is actually executed, so it’s running code from that class. So combining these things together, it means that if I specify a string like I showed the last line and this is the most naive type of attack string, but there are other ones which will share as well, which are obfuscated. But for example, this render it means the Log4J will go to the attacker server and fetch an arbitrary class, whatever the attacker hosts on that server and then execute that class, which basically means that arbitrary Java code is running on the vulnerable machine. So this is classic. I think that this is a visualization which leads to a remote code execution. We also highlighted in our blog some problematic lock ups besides the Janelia, which are worth looking at because, for example, your web application firewall might not be looking out for them. So we’ll share that in the collateral material as well. I won’t dove too much into this, but this is the way that our attack attack looks like. This is also available on our blog, so you can check it out from there. Basically, the tax ring gets sent in some way. It must be logged out, so this depends on each specific application. But again, it’s a very common scenario that it’s logged out. This triggers an LDAP query, which is a protocol that basically helps. Helps reference the the Java object through Jane Art. Yeah. So the attacker returns here, whatever he or she wants, but basically returns of the same server and references a name of a malicious class. It could be anything. It’s just going to be the class that’s going to be shared afterwards. The vulnerable server sees this and then retrieves goes to whatever server the attacker sent him to here. It’s the same server that initiated the attack here. It’s tragedy period can be through the setpiece or whatever other supported protocol, and then the attacker can supply whatever arbitrary class and the class would be run on the server. And here there’s just an example of something that delete all the files on the machine. But of course, it could be something more at this time and basically anything. So besides the attack diagram, we wanted to start talking about some of the questions that we got as a DevOps vendor says this vulnerability was disclosed. I would say the first thing that we got to ask the most is regarding, of course, whether we’re effectively armed. So there’s a few things that I wanted to elaborate on that. So first of all, we also set out a formal response and we did right away that day for our products are not affected. And that’s because we just don’t use this vulnerable package for a dash core. So none of our products do it. So actually, it was a pretty easy check on our side. But some vendors that actually can scan our packages with other dependencies, scanners like not JFrog x ray, but other ones, for example, flagged the lock for J API package and that got flagged as vulnerable to this CB. So no matter how we tried to push out this message, we did get asked. But basically the language API package by itself is not vulnerable. So we have we checked that ourselves and the security research team before this detection data was uploaded to JFrog Extra. So we did a static check, a dynamic check. It’s also actually mentioned in the official thread of this issue in the Apache Log 42 forums. So we feel very, very confident that if you have some package and you only have lock for J API, the MAVEN package and you don’t have the lock for your dash or package, then you are not affected. It does make sense in some scenarios that you have up for your API and don’t have Lock or Jacob. For example, you may use the lock for your API and your backhand might be log back or any other logging framework, actually. So this is a scenario that happens, but you are not liable if that happens. It was very important for us to go besides this because we hear this is maybe the questions that we we got to ask the most. The problem is that some security vendors flagged this package as vulnerable as well and not just lock for decor. And that’s what caused the confusion. So a lot of like open source tools, even commercial vulnerability scanners treated this package as vulnerable, and it’s not. Another thing that we got to ask is whether if I just have like for Jake or, let’s say, installed on my machine, but I don’t call any of the API or I call the Lightning API only with constant data like nothing that comes from remote input or any variable at all, just literal constant stream. In that case, you are also my marvel. So that’s also very important to say. So in a blog post, we actually talked a lot about recommended fixes. I wanted to tell you about that because that’s obviously another point that we got asked a lot. So first of all, the best fix is to upgrade like 4G. We’re aware that not all lenders can do that, so we’ll talk about mitigations as well. But that’s the best fix. And for the paranoid, we also have another suggestion, but let’s talk for a second about upgrading your 4G. So. So there are two versions. There are two stable releases that you can upgrade to right now. There’s two zero, which we reckon that’s what most of the vendors upgraded to by now, and there is a 260 zero which came out today or yesterday. It depends on your time zone. So 215 zero actually takes care of the issue pretty well. What 250 zero does. First of all, it disables message lookups by default, so there’s no funky strings that can be logged that will cause any lookup behavior in any case, unless by default, unless you purposely enabled this behavior. So this takes this change by itself takes care of pretty much most of the issue unless you’ve tinkered with the configuration. And there was another important change that basically, even if you’re using Jane Dye in any way, not even not the methods look up, although right now it’s uncertain, like for easy exploitation vector for that. But even if you even if you are using it in some way, then it’s constrained only to a specific protocol, which is LDAP and the Pass and Java, which is a local only protocol. And for the remote protocols, the default remote protocol, it’s also constrained to local IP addresses as well. So, for example, the string that we saw before like LDAP with remote hostname that would not be allowed even if methods lookups were enabled, which is not the default N250 zero. So basically with 215 zero, we feel pretty confident to say if you’re upgrading to this version. And you don’t tinker with the configuration like the default configuration, then you should be safe. There’s also another upgrade available, which is up to 60 zero, and that takes it a step further because because this became such a big deal and actually, you know, these are like both GDI and message look ups also have performance considerations for the logging framework. So in Du 16 zero Jane the high, any support for Jingye was the same old by default. You can still opt in and enable it, but it isn’t using Java properties, but it’s disabled by default and the message lookup mechanism has been completely removed. So now you can’t even opt in to that mechanism. It’s just been removed. So if you want to be perhaps extra paranoid and maybe gain some performance, then you can also upgrade to two six zero one two 15 on the default config should also be sufficient to remediate this issue, and it’s one of the best fixes. So I would say if you have like this is the best fix and this is what you should do. But why is it not perfect, for example, to upgrade to 250 zero? We are still afraid of human error. For example, let’s say you have locked 4G as a direct dependency and also has a dependency of dependency. Like, let’s say you’re using some third party package which uses block 4G. And that third party package may not be updated it. And you don’t even know if if it’s vulnerable to this issue, which everybody right now is scrambling to understand, like what are all the packages that are vulnerable and not just if you’re using like 4G directly? So the human error part here is that you might by mistake, not upgrade all of your dependency to 250 zero. If if you do upgrade all of them, then you’re fine, but it’s problematic. So an additional in addition to upgrading to 5.0, we also recommend. Doing one of the mitigations again, it’s explained very thoroughly in a rollercoaster, but there’s an environment variable that you can set globally on your machine and then it’s a great factor because then this machine will be completely safe from the methods hook ups issue like you disable the method lookups feature. So it doesn’t matter. And because you define it globally, it doesn’t matter if you have even five different instances of a lock for running. All of them just get disabled because of the environment variable. So I suggest looking at our blog post for that and we suggest that in this environment, variable globally, even if you plan to upgrade to two, 15 or 16 just because of the possibility of human error in upgrading. So they said the best mitigation that we believe. There’s also other mitigations that that are available. For example, you can pass a specific job option, do the job. And while we advocate your job application form and methods, you know, lookups. But as not as an environment variable, but as a JVM option, we still think the environment variable is better because you can have it very easily globally on the entire machine. Also, there’s older a lot for gay versions, which don’t support these properties. In this case, you’ll have to realize it’s a black man. It was also supplied in the original media advisory that you can search for all of the vulnerable love for gay core packages like the jar packages and just remove the functionality directly from the jar files. It’s a simple command that should really be a last resort because this is very troubling to human error. Like again, there could be jars inside jars. There could be a specific path that isn’t exactly what you would expect it to be. So this is really the last resort. And if you would be in this situation, I would really suggest upgrading a lock for DJ and not doing this medication, but it is possible. There’s also another mitigation, which comes from Java itself. So for specific versions of Java, that actually they’re not even that new anymore. So even for two year old versions, Java, actually you are proactive and denied any visualization of remote classes through JDA. It’s just a mitigation that it’s always configured in your versions of Java. We have the exact version on our block was, of course. So some have been also saying, oh yeah, if you’re using your Java version, then you’re actually not explainable. So this is true to an extent. There might be some factors on your specific machine that make the issue even more exploitable if you’re using a newer Java version. And also even though the remote class loading is unavailable in the newer Java version, there are some other vectors that can be exploited as well, such as information disclosure even in the newer version. So actually, in our blog, we have a dedicated appendix for that. It’s a bit technical, but we feel it’s very exhaustive. So if you’re stuck in this situation that you can’t do any mitigation, you can’t upgrade. But at least you have a newer Java version, or you can upgrade the Java version, but not the Log4J version. We highly recommend checking that part out. It’s Appendix B in our blog, and it has exhaustive list of what we know, at least how to exploit this issue, even in newer Java version. So we recommend checking this out. The last, but not least, we want to talk about our own detection. So last but not least, we also wanted to clarify to our own customers of JFrog, actually that of course, this issue is detectable through JFrog Xray from day one with the specific research data that’s coming from our security research team. So, you know, as always, we’re getting private research data with all the specific mitigations. For example, from day one, we didn’t include the lock for J API package because we tested it and saw that it’s actually not exploitable. So you wouldn’t get this false positive trough Xray. And just letting go to our customers that of course, you can detect this issue both through our UI, CI/CD, the command line interface like JFrog CLI and also our JFrog ID plugin, which is available for visual studio code and other IDE’s as well. Like this would be detected. This was detected from day one on all of data on all of these. So for all of our customers, you can feel safe to search your code, your third party code and detect this issue and upgrade as needed.

Courtney Gold: Now, this is great. I think that was some amazing conversation, I think there’s some great information that you provide because I really appreciate it. I hope everybody on the call appreciates that. As a reminder, we will be sending out the recording of this webinar. So not to worry, we’ll also be sending you some additional resources. He. He did mention the blog. We will provide that in the email that we provide to you. Post event if there’s a question that we couldn’t get to. Please email webinars at Rom-Com. That’s webinars at Broadcom and we will get those questions answered to you within 48 to 72 hours. But we are open for questions and answers, so please put anything you need, anything you love your cares about in the Q&A chat box, but anything else? I think there was some great information. Shekhar, thank you for your time and effort today. We appreciate it and everybody else coming on for the call. We truly appreciate it and thank you again. Taggart, if you want, just give a quick goodbye and then we’ll send everyone on their way.

Shachar Menashe: Sure. So thanks again, I hope it was helpful for everybody and will really love to take Q&A on this since we’ve been researching it since it came out, and it doesn’t help anyone with, you know, upgrading mitigations, detections, etc.. Thank you.

Courtney Gold: Yes. Thank you. Bye bye.