DevSecOps for Kubernetes-Based Applications@ SKILup Day DevSecOps

hello and welcome to devsecops for
kubernetes based applications
my name is van rupert and i’m developer
advocate for jfrog
specialized for devsecops
what do we want to do today we want to
talk a little bit about what are the key
features or the key points for
cloud native development or cloud native
in general
because this will lead us to some
special things
called devops or devsecops here we want
to
see why devserkops is good for business
and what are the key points here
and this will lead us to the developer
view
we want to have if you are implementing
security
from scratch so the main focus will be
how will be the view for you as a
developer
if you have to deal with dev sag ops and
if you’re developing for kubernetes
based applications
one point in the end will be what is a
good thing or what you can do
to speed up your production line or your
ci environment
if you are dealing with long running or
with complex
um ci pipelines what is true
immutability and why you should go for
it
one thing for everybody is if you’re
dealing with new toolings or new
technologies
and you have already an existing
infrastructure how to integrate
jfrog tooling in your existing
environment yeah that’s it so
let’s start better view
cloud native if you’re looking for the
words cloud native or what it’s
mean in the internet you will find that
it’s a
broad topic it will go from service or
oriented architecture of api um oriented
communication
through how to use container and all
this stuff and everything is managed by
devsecops
so the meaning of this is not only some
single topic about how to do a or b
it’s a broad topic about software
development in everything
around reading about it will show you
that you have a lot of different
meanings opinions and all this stuff but
in general you have
via cloud native foundation the
possibility to
get the condensed and the official
meaning of this
finally so it took some years but it’s
available so
cloud native what does it mean it means
that you have solar service oriented
architecture
service oriented architecture means that
you’re breaking up your
big big application into smaller pieces
microservice is one word or the other
thing is that you’re just going to
serverless functions on
which means that at this point you will
have
very tiny part projects or from the big
project you have
tiny sub-projects and the sub-projects
can freely decide
what technology they’re using and
how they want to deliver that inside the
container that
could be managed by a docker or via
biocommunities in the end
so this is a container-based
infrastructure that
everything is running inside docker
managed by kubernetes
the service oriented architecture is a
way how to split all this stuff
and then there is is api oriented
communication
api oriented communication is just
to define how the parts are
sharing information and
all together is the possibility to
choose freely the technology
it’s good and bad at the same time so
it’s good because you can
freely choose it’s good because you can
get rid of legacy stuff
but it’s bad because you’re dealing
always with all these tiny details of a
new technology
if you’re a senior in one technology
doesn’t mean that you’re a senior in the
other
technology as well your ramp up time
will be maybe faster compared to a
really
youngster but in the end if you’re
senior here you’re a youngster here
again
and this means all this stuff like
security issues
best practices and uh what are the good
and stable subcomponents or dependencies
you have to learn again and this is a
bad thing
because with new technologies you have
new attack vectors that can be used
new ways of breaking into your system
and all this stuff you have to learn so
you need tools to help you
to identify what kind of security
breaches are there
and then the other thing is how to deal
with license
comparing java with javascript for
example
the java ecosystem is a way more stable
one
and the javascript one is very
active it’s it’s developing a lot of
stuff in short time
but the dependency tree between
components and how this is versioned
and the security issue there and the
compliance issues
this is a completely different thing
okay this is the technology how to run
it uh
on a technology how to uh split it up
how to communicate between all the
styles
and the whole thing that is managing
this one is called
devsecops so the cloud native foundation
is explicitly talking about
cloud dev segues not about devops
so the main thing is what is
the difference between devops
and devsecops this is one thing but
first
i want to have a few more um ideas what
does it mean
or what what part of this cloud native
full stick we want to
look at i want to have a detailed look
at
how to implement stuff so if you’re
looking at this part from
coding software until let it run in
kubernetes
we have through the application this
piece of code you’re writing as a
developer and this is the first thing
you’re providing this one must be
wrapped inside linux or must run
inside linux and this one will be
packaged in docker images
that will be delivered
via registry uh to your kubernetes
universe
so these are the four steps i want to
have a look at
and if you’re checking here so if you
have a security breach
inside your application during the time
you’re coding it will be available in
all other stacks
maybe it can be masked or hidden by some
stuff like a firewall or reverse proxy
but in general their security hole
is available the same thing here with
license so you can have license
or transitive dependencies here during
the time you’re coding the application
you have the same thing
in the linux distribution and for sure
in docker and kubernetes as well so
everything will be managed or the whole
life cycle of this will be managed by
the
term called devsecops okay this is
so have in mind if you’re
pushing security or if you have security
issues during the
development time then they are available
until the end and this is where this
term shift
left is coming so if you’re rotating
this one
at 90 degree or 490 degree then you see
that you start coding application linux
docker kubernetes and
shift left means that you’re going in
this direction that even during the
coding times the earliest possible time
you are starting with the term security
so and inside the application
development again
coding part and then testing and so on
shift left is the same orientation
speaking about dependencies in the java
world you have a bunch of dependencies
because you don’t want to reinvent the
wheel that makes sense
because if you want to sort something or
want to create a pdf library just use a
dependency for it
but you have no control about it you
have to trust this guy not only that
he’s creating good quality that he’s
maintaining this stuff
but as well you have to check for
example for compliance issues
so is all every dependency checked
that he is using so not only you are
checking the
compliance issues for this one for this
dependency but you have to check it for
the next
level again so all transitive
dependencies must be checked for
security
and as well as for compliance issues so
it’s easy to analyze and so stuff but a
lot of stuff is based on trust and you
have to check
security and compliance issues this is
one thing
so um yeah to achieve this in depth
cycle ops
well it’s the same like devops you have
to optimize everything you have to speed
up your production it means
get rid of boring work from the
developer let the ci
environment do all this things say
security to integrate is just
extending a little bit the tooling but
you’re going through the same stuff
you’re using cr environment you’re
adding some checks
you um train your people that they have
um
security that they are security-minded
or securities skilled
the same thing you have done earlier
with operations
so here one thing that’s very important
is be reproducible
everywhere so every tiny step must be
somehow reproducible otherwise you can’t
analyze later if you have a security
breach
so in general devsecours means on the
box as early as possible and make sure
that there is no security and no
compliance issue
and make sure that this is as soon as
possible
eliminated inside your production line
the next step will be security from
scratch and how this world looks like
for you as a developer
security from scratch
now we saw what what are the basic ideas
about it
and we’re going back to one picture i
showed in the beginning it was this
application linux docker coordinated
universe stack
that is more or less basic for for cloud
native
he i explicit exclude
the concept phase it’s true if if you’re
talking about security
even during the concept phase you should
have an eye on security
you can’t have security per definition
or not per definition per
architecture so this will have some
um influence on it so
even there you have this security id so
this concept this way of isolation this
way of reporting
it will include changes in processes how
to deal with all this stuff
but it’s not part of this talk here i’m
just focusing from the application
down to the kubernetes deck and just as
a reminder
if we have some security hole in the
application
during the time we’re coding it will be
in the linux
in the docker and in the kubernetes
layer as well so if you’re doing all
this stuff
as early as possible then it will help
us to minimize the risk
but talking about the different layers
if you have different application or
different
microservices and the freedom to choose
what is the right technology you want to
use
immediately you will have corresponding
infrastructure things
for example if i’m talking about java
then i have
for my application a maven repository
the main repository is a single source
of trees
if i have to check what is used in my
application
and based on this information i have the
possibility to analyze a whole graph
but then talking about linux i have the
same
linux if i’m using debian i have a
debian repository
in the background and this is source
where i’m grabbing all my stuff
and in this repository i have my
binaries the license information and all
this stuff
after i’ve done this one going to the
docker image layer
i have my docker registry and again
this is a repository where all this
stuff is stored
and if i have access to this one i have
access to the whole binary stack
and can analyze every single layer
inside my docker container and the
docker files and so on
and after this one how it’s mounted
together
then i have my kubernetes layer and
kubernetes
then we can talk for example or about
hem repositories what we see here is
that we have more or less a common part
of this deck for example this
uh docker helm part and then based on
the linux distribution a debian
repository or an alpine repository
whatever
but then the biggest fluctuation will be
in this application layer if you have
different languages different
technologies we have npm or maven
or paybob nougat or whatever
so we have different levels here
different repositories
the good thing with artifactory is that
artifactory can handle all these types
last time i checked it was i don’t know
24 26 different kind of packet managers
so you can have all these repositories
managed
inside artifactory so everything that’s
coming out from the internet or in from
the internet
will be stored in artifactory and hold
there
the good thing is you’re more or less
independent if you’re grabbing it once
and storing it
you can say okay now even if the
internet connection is not available we
can produce
internally caching for sure but
talking about security x-ray
is a component that will scan everything
that is in artifactory how to do this
and how to use it i will show it a
little bit later but
think about artifactory and x-ray as a
combination
that x-ray will have a detailed view on
every tiny binary that’s going in
that information you can consume via web
ui
or vian rest api so a machine can deal
with a combination artifactory and x-ray
or the human can directly interact with
with it via a web ui i will show you
both
things how it looks like but the main
thing is you have
one single point all binaries
in all configurations are in and even if
you have to
for example think about the security
payload injection even if you want to
manage this one this could be part of
artifactory inside the generic artifact
repository so all these layers
will have different behavior but you
have different update cycles you
have to know the knowledge of this
repository so to deal with
transitive dependencies and get the
license information
out of it
how to define this stuff artifactory
will give you the repositories x-rays
connecting to it and now i have to
define
how to search for security and
compliance issues we have three levels
for this definition inside x-ray one
thing is a rule
a rule is a stateless definition what
should happen
so you can say if you find something
within c
v s score from a to b please
write an email too or please start a web
hook or
whatever break a build so this one
is stateless and independent from the
repository itself and this is an atomic
thing
and then you can combine this to a more
domain specific policy a policy is a
logical name
and a combination or a
aggregation or a composite of rules so
the rule itself will describe what
should happen
a policy will have a logical name and a
bunch of rules so here you’re more
domain specific so for example the
policy for web apps a policy for
whatever and then to connect this one to
the repositories you need
watches a watch will connect build
information or a repository
with policies and then you can see the
reports
so the maintenance here is quite easy
you have a very fine-grained way to
describe what’s important for you and
how to react to this one in a generic
way and then you can just combine it to
different repositories the good thing
here is that you can just
create a watch to for this combination
i will explain a little bit later if
you’re talking about the repository
structure and what you can do here to
kill rebuilds and to be really mutable
so have this your mind rules are
aggregated to policies and policies
via watches are combined with the
resource you want to
check
you as a developer you have now the task
to implement a use case or proof of
concept or whatever
so what you’re doing you’re starting
your ide you have a tiny fresh
side project you just start from scratch
assuming that you’re including java and
using maven
but you have different other yeah
package managers as well so
it’s not it’s only an example because
i’m a java developer
so you start writing your pom xml file
and you add the first dependency because
you don’t want to reinvent the wheel you
don’t want to implement this sorting
algorithm
or whatever and then immediately you
will see information
inside your ide so we are providing for
intellij for eclipse vs code and all
this
on an open source base plugins for the
ides
so if you have for example netbeans yes
so far i
don’t know it no plugin for this one you
could create for example for netbeans
here and plugin based on the
implementations
we are providing on github but the main
thing is how to use this one
so you are adding this one and then you
have this ide plug-in
and you will see immediately this
dependency
will have this security information it’s
red
green and then you’re checking all
transitive dependencies
and you can see the license information
as well
how this looks like and how to how this
handling is i will show you for
intellij and this is the next thing
okay now we have a short view to the
x-ray ide plug-in
i just added in my palm dependency so
this is about inversion it’s a little
bit older one so that we
can guarantee see some compliance issues
so
what you have you will have in the ide
plugin here
some some kind of window i’m using
intellij so it
could look a little bit different on
eclipse or whatever
then you will see the dependency we
added here already here this uh 4120
4120 and then you have the whole
dependency tree and you see
there is a security issue with jackson
data binding
you what you get is a ranking if there’s
a high medium
or low security risk you will see the
license information or type version and
so on
the good thing is you can see here all
detailed information for example
if there is already a fixed version
available
and what time is summary of the issue
itself
one cool thing is if you are somewhere
then you can just go here and say
sharing project descriptor
then you’re jumping here back to this
one and the good thing is if you decide
to exclude you can just go here and say
exclude dependency what you will get is
the dependency exclusion that’s it the
only thing i want to show is how to
install it so in intellij is quite easy
again just plug in you’re searching for
jfrog plugin
updating it or just installing it in
intellij you have to do a restart
in eclipse could be different and then
the only thing you have to do
is you’re going to the settings of this
x-ray plug-in
adding your coordinates and then you can
just
connect your intellij to the x-ray
installation
that’s it have fun you saw now how this
is realized in this ide plugin and
this was just an example with intellij
but it’s available for other ids as well
so this is a way a developer would see
it immediately
and now i want to just
show you how how it’s available on on
the web ui and for this i’m taking an
example
for example uh the next step so we coded
the application now we want to wrap it
uh on linux docker so that we can
provide it later on
um kubernetes so what what you’re doing
if you want to wrap your application
your fed jar for example that’s running
uh inside the dockerfile
you you start with the dockerfile then
the first line
from and bang here we have it
we are using a base image and this base
image
you have to analyze so this docker image
will be based on debian ubuntu
alpine whatever and immediately you
should have view
on what is going on on this operation
level operating system level
if you have a library somewhere in your
linux distribution
that is critical and then you can decide
if you want to have this package or
you’re explicit de-installing it or
or or so the tip will show you
how to get this information about doc
images and again over the
java site but this time with the web ui
so
what we see now is how to create a rule
how to combine
different rules of policies how to
create a watch and how this report will
be accessible inside your web ui
and then you will see that from your
application
your tiny jar file up to the operation
system and the docker image you will see
everywhere in this tree
what’s going on because x-ray know
the whole dependency tree that means
after we combined
the different technologies x-ray knows
okay this jar is inside this layer
inside this docker image and this docker
image is based on this linux and this
linux will have this
glibc whatever version
inside every time we are
scanning or we are updating the security
database immediately if we are pushing
this one to our installations
if you are not connected to the internet
because you are on prem you can just
download on a regular basis security
database and provide it internally
immediately if you have this one the
whole graph will be rescanned
and it means even if you provided it
yesterday you binary your docker image
and it’s
running on production and if you know
today
something new it will show you
immediately
that this image is infected how this
looks like how this
report looks like i will show you on the
web ui
now how to create the report
vulnerability report for this first of
all you have to
log in into your platform
by the way this is a free tier it means
if you want to activate a free tier for
you to try all this stuff out
follow my how to about 3t activation
or just go to jfrog.com start minus
free
so after login what you need
to create reports is
you need re repositories where something
is already in
so i have here a docker virtual
repository that is combining a local one
and a remit one and
inside this docker repository i have
here
the build pack depth docker image
but you can do it with everything else
so it could be a maven repository
whatever but you need something to
with content so to create the report
go to security and compliance go to
reports
then you can create a report you should
choose a
name in the free tier you have only the
possibility to scan for vulnerabilities
if you have a paid plan then you can
scan for compliance
as well so license checking the next
thing is you have to choose scope
i’m just scanning repositories right now
but you can do the same stuff with
builds and release bundles
so it’s just the same i’m focusing here
on repositories
the next step is just the selection of
the repository
i will just scan here for my docker
repositories
a and b selecting then
if you see it here this is what will be
scanned
save you can add some more detailed
informations what you want to have
inside this report or not so for example
you can
choose if you want just a range of
zvss scores i’m just collecting
everything you can
select the date and scan data i’m just
scanning for everything
with every cvs escort on all components
after this generate report
as you can see it will take some time to
do the job in the background
it depends on the amount of repositories
and the amount of content inside of
repositories
if this is created once you can
go here and say i want to have a view on
it i want to see detailed or i want to
export or delete it
we want to have a view this is a report
what you will get is a cve number
a short summary the severity
and then detailed information about the
cvss score
and why and the component
you can see it’s uh the lip that is
affected
the artifact let’s affect it infected
and then the path where you can find
this stuff
and very important if you have a fixed
version for this
already if you want to export it go here
to export
then you can choose between different
formats i’m just using pdf
for example will take some time and
download
you will get this download i will open
this one right now
and i will show this pdf
here is a pdf with all the detailed
information
so with this you are able to create on
the fly reports
about the content that is inside your
artifactory
so we have one thing that is not
easy if we are now leaving the
application docker linux layer
because we want to talk about kubernetes
and how to do the last steps the last
step is
if you have docker images you must make
sure that your kubernetes stick is
explicit using
your repository to grab this
dock image even if you’re selecting
official ones
so how to make sure how to deal with
this for this
we have a tiny open source project you
can see it’s actively maintained so
uh grab it it’s called qnap and that’s
available on the jfrog
qnap on github so it’s open source check
it
and it’s a kubernetes dynamic administ
at mission webwork complicated work for
me
so what it what it is doing is
it will make sure that first of all all
images that are requested are pulled
from some
special repository that is under your
control and scanned and only what say
inside and free it could be instantiated
and the next thing is if you have a new
version of this docker image
you must make sure that as soon as
possible
active instance are killed and
redeployed within your version so that
you have more control about this one and
how to manage
where is the source where grabbing all
the stuff so that kubernetes is just
grabbing
um green images this is for example
done with qnap so have a look at this
one and
now we have from application development
over how to update the linux stack how
to deal with docker images and make sure
that you have clean layers
um up to how to manage this docker
images in
kubernetes and it’s under control how
long it will
live and how fast it will be updated in
which way will
it will be updated and what docker
images are used
if you want to try all this stuff by
yourself it is easy because you are just
going to this platform
well it will take approximately 10 or 15
minutes and then you have the whole
ecosystem ramped up for you in your
environment
of your choice amazon google and i don’t
know
i will say thank you at this point if
you want to reach me because you have
more questions or you want to
just stay in touch the best way to reach
me is twitter
and i’m more than happy to have feedback
and start a discussion with you about
the topics devsecops java cortland
mutation testing tdd whatever
just call me and so far i would say
thank you very much for attending and
[Music]
see
[Music]
[Music]
you