Manage Security and Compliance with JFrog Xray [swampUP 2021]

thank you all for joining um and thank
you nero for the introduction my name is
roy
i’m a solution engineer at jfrog in my
previous roles
i was a software engineer both android
and end and today we will discuss about
jfrog x-ray devsecops
and more interesting topics so let’s get
started
we’re going to cover um we’re going to
describe the problem
and talk about an executive order that
was issued in last may
and then cover devsecops and from there
we will deep dive into x-ray
and of course we will finish with a demo
and afterwards you’re more than welcome
to ask questions
drop them in the q a box and i will try
to cover
all of them so let’s get started
we know that today’s software on average
is comprised roughly from
between 60 to 90 percent of open source
software
there are a lot of open source
dependencies that we use in our code
and it has a lot a lot of advantages so
first we never
have to reinvent the wheel usually when
we want to implement a new functionality
there will be a library that we can use
that someone already wrote and
implemented this functionality
for us so it’s very easy and convenient
to use
with different package managers
consuming dependencies and consuming
open source software
it’s very easy and of course it allows
us to develop software a lot
faster since we don’t have to reinvent
the wheel we don’t have to write
everything ourselves
we can be a lot faster when we write our
code
but of course with every
uh with every advantage comes a
disadvantage so
there’s risk introduced by open source
software
in the form of security vulnerabilities
um
license compliance issues if we are not
compliant with a specific license that
our software contains
and of course open source software could
affect on performance
now i want to cover two cases
in which this uh um that are examples of
why open source software
uh should be handled uh carefully
so first is equifax so equifax is a
consumer credit
reporting agency that is based in the us
and they use the java dependency called
apache struts
which caused leaking of data of
more than 140 million people
they used a version that had a
vulnerability
which again created this massive leak
another example um is a case from 2008
where cisco was sued by the free
software foundation
because they were using um
a gpl the gpl license
and some of their components
which was not allowed and of course
they suffered um a lot of damage from
this lawsuit
so with these two examples we understand
that
open source software is great but we
have to make sure that
we cover the security aspect and the
licensing aspect
before we dive into devsecops
i want to briefly cover an executive
order that was issued from the white
house
on may 12th this year which basically
equates cyber security with national
security
what this executive order actually um
says is that it
mandates also after vendors of the
government
to provide a softer bill of materials
and now usually these type of of orders
have a trickle-down effect
to the private sector so we should see
this
happening in the private sector as well
and basically it’s going to force
software providers to figure out all the
dependencies
and transitive dependencies that they’re
using in their software
it also means that regulation is
probably coming in the next few months
now a note on this we also have a
recorded webinar on this executive order
that we can share
so we will share the link after this
webinar
okay so we understand that
there is a problem we understand the
problem um
and now what i want us to focus on is
what exactly
uh how are we going to tackle this and
what do we need to understand
so first we need to understand in our
software
what are the infected components are we
using them
and how are we using them for example if
there is an
infected maven dependency is it used
only in a specific java application
is there maybe a docker image that uses
this java application and so on
and second how are we going to fix all
these issues
without harming our development process
without decreasing
our development speed
okay let’s start with talking about
devsecops
i assume that you’ve seen this slide
sometime in the past but
if not we all know that devops is a
never-ending cycle
of automatically building and deploying
software
and we can see the different phases here
of planning coding
building testing releasing deploying
operating
and monitoring in each iteration
and again it’s so this is a never ending
cycle
that is done automatically as part of
the devops vision
and we want to enforce security
in every step of this cycle
and this is exactly what devsecops aims
to do
it aims to embed security in every part
of the application lifecycle
but it also means that we are going to
develop applications that are more
secure
without harming uh speed so we are going
to embed security
in every phase of this cycle
okay now devs and cops
does bring with it some constraints
so first is the team size we need to
address this
the security team oftentimes is way
smaller
than the development teams so we cannot
rely
only on the security team for this
purpose
and we have to involve everyone in the
security process
second if this procedure is going to be
manual
for example if we have an excel sheet
with the different licenses of
of the different packages that we use
this procedure
could be uh could decrease the speed of
development
so again if it is manual it is a problem
and we need to think about how we’re
going to automate it
third is when do we validate our
software
so think about um
you know you have there’s an application
that we are writing
and maybe only in the very last phases
or after
um the software has already been built
we are going to test it for security
purposes and scan it
and only then we can find if there are
issues or not
this could be too late in the process
and again
it could slow us down and this brings me
to the
to my fourth point that efficiency could
be affected
um if devsecops is not done correctly
so we need to address all of these um
bullets
however we do know the devsecops brings
with it
advantages so first
it keeps our um our software more stable
and it increases the quality of course
we know that if we do it properly
this could save us time and could save
us
money because if we validate our
software at an early stage
we will not have to roll back at a later
stage which is going to be
more efficient and it’s also going to
help us stay competitive
because the agility the devsecops could
bring with it
could help us um increase
um our cadence of releases and decrease
our um the time it takes for us to
develop
new versions for our application
and it’s going to help us gain
consistency when it comes to compliance
again making sure that we are not using
any licenses that we are not compliant
with
and that we only have packages that are
permitted for us to use in terms of
licensing
now let’s talk about jfrog x-ray jfrog
x-ray is
our software composition analysis
solution
it is part of the jfrog platform and
let’s break it down
so x-ray knows how to scan
binaries um and to give us insight from
two aspects
the security aspect and the compliance
aspect
now it integrates natively with
artifactory and we’re going to see a
demo
of x-ray so you will see all of that in
action
and x-ray could be used by all the
different teams
as part of our development process the
security team
the legal teams the development teams
the operations
and so on
okay x-ray offers
a variety of features and i want to
focus on the core features that x-ray
has
so x-ray uses a variety of databases or
a variety of sources
to detect vulnerabilities or to obtain
data about vulnerabilities
one of those databases is fallen defeat
which is a private vulnerabilities
database that is considered to be one of
the best databases
in the world for vulnerabilities in
addition to other databases
that x-ray uses x-ray
continuously scans your artifacts
so it’s not x-ray doesn’t scan
a binary only when you trigger a scan
or only on demand it’s going to scan it
whenever there is a need
if there is new data for example if
there is a new binary x-ray will scan it
x-ray knows how to recursively scan
binaries for example let’s think about a
docker image being scanned by x-ray
x-will will know how to peel the darker
image layer by layer
and look as deep as it can to detect
dependencies
so for example if we have a docker image
that in one of its layers um there is
a maven dependency which is dependent on
another maven dependency that is
infected
x will be able to detect that which
leads us to
the fourth uh feature which is impact
analysis
we need to understand if we have an
infected uh
package who is using this package
how did it um how is it included in our
code
is it in one of the layers of the docker
image is it
is it a dependency of our npm
application and so on
and lastly reports we will see that in
the demo how we can produce reports
from x-ray
two very important topics or two very
important concepts that we need to
understand
are policies and watches before i jump
into a demo
i want to cover what policies and
watches are
so watches are the
enforcement of policies on top of
resources
let’s break this down so a resource and
artifactory
could be a repository it could be a
release bundle and it could be a build
process
let’s leave our release bundle aside for
a minute let’s focus on
builds and repositories i assume that
you’re familiar with repositories
for example docker repositories or npm
repositories
we will see that in the demo and builds
as part of build integration as part of
um
ci the ci support that artifactory
offers
we can publish our builds into
artifactory providing
not only the artifacts were produced as
part of this build
but also the context that created them
and watches are enforced on resources
again like builds and repositories
now every watch has a policy attached to
it
so with a watch we connect a policy
to a resource now policies
define what is the logic of our
violations
so for example if we need to forbid the
use of gpl
we’ll create a policy of type license
that forbids
this this uh specific license the gpl or
if we want to
enforce a very strict security policy
or create a policy that triggers a
violation
whenever a new security issue is
detected
from severity medium and above we will
see this as part of the demo
it will make more sense
okay so without further ado let’s jump
into a demo and let me just share my
screen
one second please
okay
okay let’s just give it one minute to
make sure that the screen was loaded
okay great
so what you’re seeing here is the
administration part
of the jfr platform and i’ve navigated
to x-ray
watches and policies and i’m going to
create a new policy
um and we will break down the different
options that we have when we’re creating
policies
so again i’m going back i clicked on new
policy
let’s call this policy whether or not
x-ray weather policy a policy has two
types
it could be of type security and it
could be a type license
we will give this a meaningful
description
and every policy has a set of rules
let’s create a new rule
so first is the rule name
and then what is the criteria for this
rule and we have two options
you can either use severity which
varies between low medium high and
critical let’s choose medium for example
or we can use a cvss score which is
a way to measure the severity of
a vulnerability there is of course
correlation between
these two methods of measuring
so let’s stick with the variable one
minimum severity is medium
and on top of our criteria or on top of
our
policy we can define automatic actions
so those vary from very restrictive
from less restrictive actions um to
sending an email whenever a new
violation is detected or to notifying
a deployer or even triggering a web book
if we want to integrate x-ray with
another tool
to more restrictive measures like for
example blocking the download of an
artifact
if it is infected so
let’s talk let’s for example just notify
a deployer
and in the more restrictive options we
have the option to fail the build
this is done as part of build
integration we will see an example of
that
in a few minutes it is also possible to
plug the download of an infected
artifact
or an of or of an artifact that was not
even scanned by x-ray
so let’s leave it at this let’s save
and as you can see we can add multiple
rules
to a policy um and the first rule
that actually matches um that’s great
that
its criteria matches the the
vulnerability will be triggered
so let’s create this policy okay
now let’s create a watch so just a quick
reminder
a watch helps us enforce a policy
on top of a resource
so let’s call this we’re gonna watch and
watch
for the x-ray webinar
now we have different types of resources
in our factory
repositories builds and release bundles
let’s choose for example a repository
let’s take a conan repository
we can also use patterns
to fill to match repositories but let’s
keep it simple and just save this
repository so again this watch is
applied on one repository
and now we can attach policies to this
watch
for example let’s take our webinar
policy we can attach multiple policies
which will trigger multiple violations
so you can trigger more than one
violation
on the same issue from different
policies
now the idea is that watches help us um
control the enforcement of policies on
specific resources
so that for example a specific team
could have
its own watch and there could be maybe
global policies
that we define for the entire
organization and every team
could choose to use different policies
from the global policies
and to assign um different policies on
their resources
it’s more it gives us granular control
of the enforcement in x-ray
okay let’s create the watch great so
we’ve seen how watches and and policies
are created
and this is part of setting up
setting up x-ray i just want to quickly
remind you
that policies could be also created
the type license
and if we create this rule we can see
that
the criteria is a bit different we can
choose
allowed licenses or banned licenses so
for example let’s span
a specific license like gpl
one of them
and we can disallow unknown licenses
and we can use multiple license
permissive approach which basically
means
if more than one license is found let’s
take the most permissible one
and the actions are uh identical to the
actions on security issues
so again an artifact could be downloaded
if an a forbidden license is found or
the build could be failed
if a forbidden license was found and so
on so
let’s just cancel on this and
return to our demo okay
let’s take a look at what an infected
package looks like
i’ve navigated to the packages viewer
factory and i’m filtering
by package type let’s use package type
docker
let’s take a look at this apache struts
docker image
and let’s examine the latest version
navigate
to x-ray data and here we can see that
we have
113 security issues 80 licenses
and 222 violations
let’s examine one of them for example
this violation so we can see that this
is a security violation
and the package type as an event this
means that the infected component
is a maven component
we can see what is the summary of this
issue
what is the severity
when it was created a reference to the
issue itself
which policy triggered this violation
because remember
a policy triggers violations or creates
violations
so every violation has a policy that
triggered it
and of course what is the watch
that this policy was enforced with now
if we take a look on the graphical
representation
what we see here is a docker image that
in one of its
layers there is a maven dependency which
is dependent on another maven package
that is infected this gives us the
impact path
as part of impact analysis we can see
exactly where the infected component
uh is and what it is what it affected
and we can also take a look at the
infected package itself
if i go to security and actually take a
look at the issue
and see what is the fixed version for
this issue
okay we also have the option to ignore
some of the issues that we found that we
find
so if i go back to my
docker image you can see that there is
an option to ignore violation
now there are several options when we
are ignoring a violation
it could be based on on the
vulnerability on the component
on the artifact but in anyhow
as best practice we should add a note to
why we are ignoring this violation
and we should also set an expiration
date
because we don’t want to ignore
violations indefinitely
because they are there for a reason okay
let’s leave it at this i also want to
show you
[Music]
for example
the ancestors of this docker image
so you can see in the descendants tab
within a docker image the different
layers
and with every layer that is infected
you can actually see
which packages are infected so for
example if i open more
more layers you can see that in some of
the layers it’s a maven
dependency that is infected in another
layer it’s a debian
dependency that is infected and we can
also check the ancestor ancestors
which means where is this docker image
actually used
so again it’s going to help us control
and monitor our instance whenever we
find it whenever we find an issue
and understand where it was actually
used
another topic that i want to cover is
build integration
so as part of build integration that
artifactory has was
with um ci servers we can also integrate
x-ray now what you’re seeing here
are the list of builds that were pushed
into artifactory
let’s examine one of them npm sample
with violations
the name uh implies that this is a demo
build process that is that was created
for this purpose exactly
but it is a real jenkins build you can
see that we have three executions here
that were pushed
into uh artifactory and let’s examine
one
in particular the fourth execution and
i’m going now to
quickly share
my other screen because i want to show
you
jenkins as well so bear with me just one
second
okay
okay let me share my screen
so you should now be able to see
my champions instance just a minute
guys
great okay so here are the different
builds that i have
in my jenkins instance and i’m going to
examine npm sample with violations
now again this build what you’re seeing
on the screen right now
is the equivalent of this build and pm
sample with violations
and in particular execution number four
let’s examine execution number four and
take a look at it
and we can see as part of build
integration what were the published
modules of this build
we can see it’s just one artifact that
was published and we can see for example
all the different dependencies
if we can if we’re talking about um the
software build materials this is exactly
what it is
we can examine the build and compare it
to another execution
which artifacts were produced were they
changed or not which dependencies were
used and were they changed or not and so
on
but let’s focus on the x-ray data
because this is what we want to uh
this is the purpose of our webinar today
so
again it’s part of build integration one
of the things that we can do
after publishing our build process to
artifactory
is to trigger an x-ray scan so i’m
quickly jumping back to jenkins
you can see that the last step in this
build process is an x-ray scan
in execution number four it failed
now this basically means that all the
different
dependencies that were used in this
build process and the artifacts that
were published in this build process
are scanned by x-ray and if i take a
look for example at
one uh one issue you can see that in
this build process
i’m even an npm dependency was used
that has um a high vulnerability a high
security issue
um and we can see a reference to it here
now this gives us another quality gate
in the form of x-ray so again
as part of your ci process and it’s part
of the shift left
philosophy we want to fail as fast as
possible
so we don’t want to wait for the release
of our software
for x-ray to scan it but we want to
incorporate it integrated
in our ci process so that whenever a new
version is built
it is scanned by x-ray so x-ray is not
only a quality gate
but it also does it also helps us shift
left
and deliver faster so again
if i for example examine the descendants
view we can see
within the build process the different
dependencies we can see which were
infected and which are not
okay now failing the build
is an advanced feature
that should be uh should be adopted
carefully
and the first phase is when onboarding
with x-ray it probably doesn’t make
sense
to fail the build because
at the first phase we just want to um
understand what is the current status of
our dependencies of our artifacts
fix the critical issues and only then
start enforcing more restrictive
measures
okay sorry one second
okay let’s continue
and let’s take a look at x-ray reports
x-ray can produce reports
of different types and reports represent
a moment in time
so for example let’s produce a webinar
report
we can choose the type it could be ideal
violations licenses filed
or vulnerabilities let’s choose
violations for example
the scope could be any type of uh of
resource so let’s take repositories
let’s take for example the conan
repository
and of course we can um
there are more filters here and more
settings that we can use
but let’s keep this report as basic as
possible
and generate it so this is actually
going to create a report with all the
different violations in a specific
repository and if i take a look at
another report that i created a while
ago
we can see that we can export the report
in several formats and this is important
because if we want to integrate our
reporting with another tool
if we want to analyze the data it’s
going to be easier probably to do it in
a csv or a json format
okay now another thing that you can do
with x-ray
and it’s again as part of shift left
methodology
is the fact that you can scan
uh from your ide um your your
components so i’m going to share
my ide instead of
okay
so x3 has id integration with several
ids
one of them is vs code we also support
intellij
and eclipse and more ids and what you
can see here
is that even before pushing my code
even before committing my code after
adding a dependency and even before
having this dependency available on
artifactory we can scan it with x-ray
we can get for example this express
package that you’re seeing here in the
package.json file you can see that it is
infected
and we can see a summary of the problem
and a summary of the different issues
that were found as part of this
dependency so we can see more
dependencies here
and for example take a look on the
bottom part
we can see exactly what the issue is so
this is an important feature because
even before committing our code even
before
building our code we can detect issues
which will make
our developer process faster because
when we fail fast
we will be able to release faster we
will not have any blockers
um when as we as we progress
um and we want to fail as fast as
possible like i said earlier
okay in addition to this we are also
working on git repository scanning
this is a feature that is uh coming in
the next uh
in the next quarter so that
actually will be able to scan github
repository
get repositories um even before
uh those dependencies that are found in
the git repositories
are present on artifactory again as part
of the shift left philosophy
okay i believe that our demonstration
is done and we can now move on to
q a
going to skip these slides
great okay
so let’s take a look at the q a section
so um one of the questions was about
the jenkins pipeline and if we had to
use a rest api
for the integration so again as part of
build integration we have a native
plugin for jenkins
so the invoking of the scan was no more
than just another step
in the jenkins pipeline using our plugin
we could have used the rest api for this
purpose but it’s easier to use the
plugin of course
and if not we could have also used a j4
command line interface
to invoke a scan so it’s as easy
as adding two lines of code in the case
of of many ci servers
jenkins being one of them and if not
again the commander
interface could be used for this purpose
and the rest api
okay
so another question was regarding
vulnerability matching
so i’m going to read the question if a
vulnerability matches a policy rule the
first matching policy rule is used
but does the system stop there or does
it continue eventually using all
matching policy rules
so what happens is when it comes to
rules
the first matching rule is the one that
uh that is used
so the other checks will not be
performed but
with policies all the policies will be
checked
so you could have again on one package
more than one violation
from different policies but it’s always
from one rule
jonas asked a question thank you um for
the compliment jonas
he said that it was a super cool
presentation um so
you mentioned uh the shift left
philosophy and i just want to quickly go
over that again
um the shift left philosophy
basically means that we want to do
um we want to fail as fast as possible
and again i want to emphasize why this
is important
the worst thing that can happen is that
you want to release conversion to
production
you scan it with x-ray and only then you
detect issues
or you’ve already done your architecture
design
based on a specific component that you
wanted to use
and only then you understand that there
is an issue so
as part of shift left philosophy these
issues are going to be resolved
very early on in the in the development
life cycle
okay let’s see if we have any more
questions
okay what happens if my image was
already used in production but there is
an issue
okay so that’s a great question
one of the key features of x-ray is the
fact that it continuously
keeps scanning your artifacts so even if
a docker image was already deployed to
production
and a new vulnerability was detected
x2 will still notify us about that so
even if we have an image deployed in
production
and again a new vulnerability is
detected new data is found
without the need of of of us triggering
another scan
x-ray will notify us about that and we
can fix the issue
can we use x-ray to scan proprietary
binary artifacts
i’m not convinced that i completely
understood the question but
you can scan jars for example docker
images
and binaries that were not built by you
with x-ray i hope this covers the
question
any other questions
okay so i think that we are good to go
um and i think
that we can wrap up so thank you so much
for joining this webinar it was a
pleasure
um on my side i hope that you gained a
lot from this
this uh this slide will be shared with
you um
and other than that if uh if uh
any of this is of interest to you feel
free to uh reach out to us
and schedule our technical sessions uh a
technical session with us
or to use uh um our free uh
uh tier subscription just to uh uh get a
better impression of
the j4 platform thank you very much
great thank you so much roy for this
presentation uh thank everyone for
joining us today we hope
you found it useful i see that we got a
few more questions in and we’ll make
sure to follow up with everyone
via email uh you’ll also see the
recording of the webinar tomorrow and if
you have any other questions please
feel free to reach out to us thank you
everyone
and have a great day goodbye
you