Demystifying the SBOM’s impact on Secure Software Deployment at OWASP NoVA Meetup

hi everybody we are back in 2022 and i
want to thank all of you ohas nova
participants for
joining us again in the new year i want
to try and keep things as quick as
possible but i’ll do a little bit of
housekeeping at the end of the
presentation
so for right now we have bill manning
who is a uh he works with jfrog and he’s
here to talk all about s-bombs so
without any further ado oh would you
mind taking it away
yeah no worries at all how you doing
everybody uh my name is bill manning i
am one of the solution architects and
also uh the manager of our solution
engineering department um uh for north
america new business uh for jfrog um
i’ve been with my jfrog about five years
uh just a little background on myself
i’ve been i started off long ago
obviously you could tell from the white
beard i have here um you know i start
off as a software developer i’ve been
everything from software developer to
cto to venture capitalist uh you name it
i’m a mentor of techstars 500 startups
you name it i give back to the community
where i can
but today i’m here to talk to you uh
about uh software build materials and
when i before i talk about this you know
one of the things that i’m going to talk
about when we are going through this is
how do we get here right because
software build materials is technically
you know nothing new like just really
here at jfrog we’ve actually had this
this concept for the past eight years uh
in the form of thing called build info
but really what it comes down to is you
know why is it important in this day and
age specifically and you know what was
the catalyst
for things like the us government to go
and make this a mandate and we’ll talk
about that too but also the importance
of why it’s important for you as an
organization
do let me go ahead and i’m going to
share my screen
give me one second and i’m going to
share my lovely little presentation here
so
there we go i hope you guys can see it
so today’s talking course is
defensifying you know s-bomb impact on
software you know secure you know
basically secure software development
for a wasp it’s a mouthful to say but as
stated my name is william manning um i’m
at jfrog if you want to follow me on
twitter my my handle is just simply
william manning you can call me bill
i’ve been called a lot worse um over
time
um but first of all i just want to let
you guys know um that uh actually we’ll
put it into the chat um but for the
first 25 people who go to the little url
that you see there below um we’re
actually gonna be giving away a t-shirt
and we’re gonna be giving away our book
called liquid software it was written by
a couple of individuals in the company
uh we all had a bunch of us this is from
a little while ago a bunch of us had a
lot of contributions to this um but the
thing is is that uh just so you know go
ahead get yourself a uh a free t-shirt
and uh and a book for the first 25
people
so let’s talk about why so of course we
all see these tremendous headlines if
you follow things like hacker news or
you just follow twitter and you go in
and and you search for all these various
you know attacks that are happening in
this day and age uh one of the main
features that we’re going to be talking
about here is basically securing your
software supply chain and this leads up
to the software build materials being
one of the end results because now in
this day and age you know you have
things like you know there’s a hacker
exploit that might have happened or you
know uh
one of the libraries one of the
third-party transit dependencies uh that
you’re you know is being utilized
somewhere has some sort of you know
nefarious means like this one here from
uh mpm one of the things about it what
it did is it installed basically a
bitcoin miner into any sort of web
services model that you’re using
um you know we always see these things
you know corporate data secrets are
leaked and you know from repositories
because last year alone software supply
chain attacks were up 650 percent let
that set in for a minute right that’s
insane you know the thing is is as as
developers
we have this inherent trust um you know
if you’re building your software and you
know you find the libraries that you
need not depending on type you know
whether you’re doing it in python you’re
doing nougat mpm uh go you know you’re
doing it in dart or or whatever even
those you know you’re doing docker
containers to host your application or
you’re using it for your development how
do you know uh that the pieces that
you’re inherently bringing in are safe
and secure because you know for i’ve
been doing this for decades now and
there’s always this trust model you know
i need to get something done um i look i
find a library that you know does
something that parses a string the way
that i happen to need it or you know
power you know looks through a hash map
and finds the values i need more
expeditiously than something that i
could write right this is something we
do as developers and the thing is is
that this trust has been eroded um and
it’s been eroded because of various
attack strategies that are done by the
industry
you know we we see this all the time you
know supply chain attacks have been on
the rise for for years right i mean this
is nothing new this has been around for
a while it’s just that it’s just ramped
up in this volume uh over the last
couple of years i mean and the big thing
that really pushed it over the edge and
really was like i said the the spark
that launched this whole idea of better
software supply chain management and the
ideas of actually having things like
s-bomb was of course solarwinds i mean
that was the one that really brought
everything to prominence there because
you know over 18 000 customers globally
were affected by this and i’m going to
talk about some of the aspects behind it
on why this was such a bad thing and all
the various also attack methods that are
being done
in terms of you know supply chain
attacks currently right now
this alone is a multi-billion dollar
remediation that is still in process to
this day there are still customers that
we deal with that are still dealing with
the out you know the basically the
fallout from actually the solar ones
attack
but i mean even recently we’ve even seen
other ones i mean log for j was just uh
you know two weeks ago you know lock4j
one of the most prominently used actual
libraries on the market
was so susceptible it was incredible you
know the simple idea that you can inject
a query into a logging structure that
could cause detrimental harm to anything
you’re doing was terrible but why are
supply chain attacks so prominent why is
it so you know why does it seem to be
happening more and more and why is it on
the rise and then there’s a lot a couple
of reasons you know there’s a bunch of
reasons on why so if you look at what
i’m showing you here you know you can
get the idea that it’s first of all it’s
super low effort it really doesn’t take
a lot to actually get something
nefarious into the stream of of software
supply chain right it doesn’t take a lot
of technical effort itself and once it’s
in it’s in and the spreads can spread
like wildfire remember you know if any
of you have ever worked with something
like mpm npm is the most notorious i’m
an npm developer i’ve built two
companies using npm before i joined
jfrog as as our base foundation for all
the things that we did because it was
easy it was swift it was super you know
it was just it was robust enough to do
what we needed to do but you know you
type in a package that you want to use
for say something you you’re a problem
you’re trying to solve you put it into
your package.json you do an npm install
and it brings along 120 of its friends
right i mean that’s just the way it
works i mean the thing is is that you
have direct dependencies and you have
indirect dependencies you have the
implicit you know dependencies that you
state and then all the indirect
dependencies that come along with it
that help you build your software
the other thing too is is though it does
have an abusive relationship in terms of
trust right because the thing is when
something comes in there’s a lot of the
times you’ll hear these whole approaches
of like you know throw the baby out with
the bathwater or the analogy of that is
is that in some cases some of these
supply chain attacks have harnessed the
reputation of some of the really some of
the best libraries out there and a lot
of this is not really you know that
apparent to most and the thing is though
is the biggest factor is it’s so easy
for an attacker to blend their way into
the community we see this all the time
whenever we do some of these root cause
analysis on on a lot of this we you know
my company we actually have our own
security organization we have our own
research team that goes and looks at
this and starts to get back into the
information we actually are a cve
provider and the thing is is that
as we’re doing this we realize as you
trace it back the thing is is that these
attackers come in they infiltrated some
of these attackers that we’ve they’ve
actually started to you know it’s really
difficult to hunt them down we can get a
generalized idea of where it came in and
the thing is is that most of the time
some of these people what they do is
they come in and they start contributing
um as regular contributors and so
they’re not even considered a suspect in
this and that’s the problem that we see
these days
and the thing is is the reason why this
attack you know this occurs is you know
you as a developer are inherently
trusting of the things that you utilize
to do your job it’s like having the
proper tool set you know you’re a
mechanic you want to make sure you have
the right tool set to work on it you’re
a carpenter you want to make sure you
have the right hammer you don’t want to
use a mallet for something that you
might want to use on say a finishing
hammer um you know these kind of things
you want to make sure you have and
there’s like i said this trust model
that we have you know when we go on we
define our transitive dependencies uh we
do our builds uh you know we produce
what we’re going to do as our output and
you know we go ahead and we give it to
our customers but the problem is is that
what happens if you introduce something
potentially threatening
you know suddenly it’s in your code and
then it ends up at your customer or it
ends up in your installation or your web
service or your devices doesn’t matter
the endpoint is not the you know not the
case the programming language is not the
case it’s an inherent problem that is
dynamic across the industry
and what happens is a lot of the times
it’s not the direct dependencies that
you use that actually cause these
potential you know you know nefarious
things to happen to your software and
potentially expose your your company
because let’s face it man you know
whether you’re doing this on your own or
you’re doing this as part of your
organization no one wants to be a
headline or no company wants to be
associated to a headline where suddenly
you know blah blah blah company
accidentally leaks all this data due to
a software glitch right they’ll always
turn into a software glitch or something
because in this case even if you have a
straight dependency model where you have
your direct dependencies a lot of the
times these attacks occur in the
indirect method you know it like gives
you an example if you looked at the
hierarchy of what happened in titus
solar winds the actual component that
came along with it that caused the issue
itself was like a third level dependency
or a fourth level dependency it was
buried deep inside the software itself
that was used to create it and the best
part about that is is that that was
actually it was actually pretty um
pretty beautiful in its execution and
the way it actually did it was the fact
that it didn’t activate itself until the
actual software was activated then it
set off a timer for 14 days and 14 days
after it started that’s when it starts
to do the nasty ds that it did so when
you bring this in you’re unintentionally
bringing along all these other
components with it and you get the same
effect you can actually you know you can
actually trace a lot of this there’s
there’s plenty of sites out there where
you can go ahead and you can trace these
kind of movements of these binaries and
how they are over time of course there’s
your products like vulnerdb and others
that are you know large repositories of
this and you can always check out all
the standard cves you can all you know
with oasp and all that too you guys know
this
so the thing is is that whenever i talk
to you know anybody about this whether
i’m doing the public forum or i’m doing
it with our company uh the biggest thing
is remember you know always having to
tell you know people is 85 to 90 of the
software you produce is on somebody
else’s code base right we depend on
these vulnerabilities we depend on
developers uh we depend on these
libraries to do our job and 99
of those hey you know at least or you
know our container at least some you
know is open source and at least 75 of
that has at least one open source
component with a vulnerability
associated to it
and the thing is is that 49 of those has
at least one high risk vulnerability and
when i say higher risk i’m talking about
if you were looking at cve we’re talking
you know you know up to the critical
range here and the best part is is that
ninety percent of the components that we
use to build our applications are
usually outdated by four more years or
they were completely abandoned
one of the initiatives that’s on right
now is the idea of applicability and
also to reputation how reputable are
these binaries how applicable are these
binaries and these potential threats to
the actual things i’m doing these are
things that people are trying to go
ahead and build uh you know processes
and also build other components around
it uh to see if they can actually flush
some of this out
and some of the things that we always
talk about too though is is the fact
that you know oops let me go back 74 of
these application vulnerabilities can be
fixed by just a simple update you know
like for me personally when we work with
our customer base here you know we
actually say hey the library you’re
using is potentially nefarious here’s a
potential remediation that can help you
but of course that’s always that’s
always subjective to the usage of the
actual components itself in the code
so when software attacks there’s many
different attacks that happen and when
these attacks happen was really like i
said this is really the driving force of
where we’re going in terms of what is
software built materials and you know
how it actually affects
you know of course we have is you have
different you know different things like
um you know typo squatting uh you know
you fat finger in the name of the
library um by accident and somebody out
there has either you know written a you
know a a library with you know multiple
versions they’ve branched it off they’ve
added in the garbage they want to add in
and you accidentally go ahead and you do
some typo and next thing you know you’re
bringing in a library that can be
potentially threatening or malicious
you know you have things like dependency
confusion dependency confusion is always
a real interesting thing you know you do
a search um you know and and you’re
searching for a library and you’re you
know maybe it’s something like uh one of
the examples i give is that we found is
like paypal right there’s certain paypal
um things they go out there you know
paypal is a trusted name in terms of
finance um you know the fintech industry
they have real high standards actually a
good friend of mine is one of the uh
directors over there for security um
actually is my neighbor um and the thing
is is that you know there’s this level
of trust that people have or you know
apple hey they know what they’re doing
or you know or yeah this is one of the
biggest names in the industry netflix
you know netflix you know produces a lot
of open source components to the
community uh microsoft who years ago
everybody knows this right they were the
ones they’re like oh no open source and
now they’re they’re actually one of them
actually one of the largest contributors
to the open source community itself but
the thing is is that these you know
these public dependencies like paypal um
we actually found this one uh in a uh a
issue we’re researching and when we did
we found out that you know one of the
things that somebody brought in was pp
logger off paypal um and a couple other
things right and everybody’s assuming
had the name paypal in it and these were
terrible
um and what they were was actually they
were exploits these are complete
exploits that were actually being
brought in into certain things and
actually this is part of a project i put
together just to see how bad it was and
when i tested it um it basically exposed
everything it started opening a bunch of
remote calls
it was terrible it was just a horrible
thing but you know being aware of this
because when you start looking at
software build materials one of the
things about it is is understanding the
component analysis component graph of
everything that you produce
so we the thing is is that you know the
best defense is a good offense right you
know that’s always the big as everybody
always says um so when you start looking
at this you know all those things that
we talked about led last year in may to
the u.s government saying we need to do
something right so the binding
administration enacted the executive
order on improving the nation’s cyber
security um as soon as that came out i
had to go download it and i had to read
it uh it was part of my job and let me
tell you it took a lot of coffee and it
took a lot of time to read it um i do
have some legal background in my
experience so it wasn’t too bad but the
problem the thing is is that when we
look at this section four is the part
that really stands out right this is the
enhancing software security part but
then when we start going into uh section
seven the thing says is that providers
uh you know provider a provider and
purchaser a software build materials for
each product directly or or by
publishing it on a public website and
what this really came down to was as us
government says we’re not going to
accept any software that is actually
going to be something uh you know we’re
not going to take it on trust you need
to provide us with basically a bunch of
information about the software you
produce so we can have it on record so
our security teams can go ahead and look
at it
lots of multitude of different reasons
but basically what it came down to is
this became a mandate and actually the
thing is is that
actually oh i thought there was another
slide in there this whole thing um you
know was actually enacted fully by the
house by the way back in november just
so you know this is now a thing but also
too the software build materials mandate
is now bled into the private sector
right so now financial institutions like
i work with about you know some of the
top banks in the world they’ve now
adopted this we work i know there’s a
lot of fintech a lot of avionics um you
know space industry food industry you
name it have adopted this this strategy
and we we’re getting more and my company
is actually getting more and more calls
on this all the time everybody wants to
know what the software build on
materials is
and the thing is is it’s not new this
has been around for a while right as i
stated before and basically what a
software build materials is it’s like
when you get a box in this case of like
german chocolate delight i have no idea
why i picked that but i did and really
what it comes down to it’s the list of
ingredients and it’s simple as fact it’s
a list of ingredients and also caveats
too you know like in this case you know
the whole idea contains wheat contains
milk you know these kind of things but
this is a way for you to have you know
if somebody gets your software and they
request this they can go ahead and do
searches and we have a lot of customers
that i deal with personally they’re
saying here’s the list of binaries that
we you know these third-party transit of
direct and independent dependencies that
we have approvals on or we don’t have
approvals on uh you know they’re usually
based on some sort of analysis that was
done using like oaas or you know using
other tools to say here’s the list of
the latest nefarious binaries does it
contain this let’s parse through it and
in the past it would’ve been really
super difficult for you to go in there
and supply this if a cousin of a company
came to you and said hey we want to buy
your software but we want to know every
little piece of ingredients inside of it
a lot of companies will go ah i don’t
that’s going to take some time and trust
me i’m a dude who actually you know went
ahead we had a product that we built we
did a lot of work for it as a startup
and we actually went up selling this to
uh google and motorola back in 2010 but
part of the deal was during the
acquisition of course and i’ve had a
couple other acquisitions you need to go
through and do a breakdown of all the
components and all the licensing that
you use in your software and supply this
as part of the m a procedure so they can
go through it and make sure that you’re
not using someone else’s software that
is either purchased or what not that
could actually uh potentially even
interrupt a deal by the way in m a to
say your software is not what it is
actually cracked up to be you’ve
actually just written something on top
of somebody else’s because we can see
everything that you’ve used
but this all comes down to is is that
when you look at something i said you
know when i decided to put this talk
together i was hungry um i decided that
you know what i didn’t really want to
talk about directly about software when
it came to this but what it comes down
to is cake i like cake um i thought this
was a funny cake but anyway what it
comes down to is how do you know what’s
in that cake well here’s all the
ingredients that are in that cake you
know the thing is though it’s two cups
of you know a flour you know two you
know you know one and a half teaspoons
of baking soda whatever but this lets
you know everything that is in this cake
when you’re doing it but if you notice
there’s certain things that aren’t there
there’s no instructions so that’s the
thing is when you think of software
build materials you know one of the
sections i have when we talk about this
is you know some of the uh some of the
things that are misinformation um around
what software build material should
really entail
but this also this standard just so you
know actually started back in 2018
by the food and drug association and in
conjunction with the national
telecommunications information
administration
why medical devices medical devices run
on software uh there was a couple of
incidents where medical devices the
software updates were could potentially
kill people um i actually gave another
talk on this it was a fun scary talk
about how software can kill and the idea
here is the same idea is that the
american food and drug administration
said hey we do this for food we’re
notifying our customers of the things
that are involved in this at the same
time we should also be aware of the
software which is like a bunch of
ingredients that people are using in
their medical devices um actually it was
you know it came down to pacemakers and
and insulin pumps right and then other
things too such as uh you know
ventilators i mean come on it’s been two
years of hell here right and uh you know
we all know about ventilators are now
more than ever and the thing is is that
it was a way for these manufacturers to
provide accountability and provide this
to the fda when trying to get their
devices approved because it wasn’t just
the devices it’s the software on those
devices that they wanted accountability
for
so a software build materials is simply
just a list of ingredients that makes up
the software inside right it’s
everything from the libraries and
modules that you have both proprietary
and open source free and paid you know
the old joke of uh you know everybody
knows this one right you know your free
isn’t speech not as in beer um you know
the thing is also too
can are they widely available and are
they restricted in access
other additional information which is
just as important and this is super
important it’s not just the libraries
you use both transitive indirect
dependency and direct dependency
injections and and all that it also
comes down to you know what tooling was
used is the tooling you use to actually
create the safe when i talk about that
i’m talking about ci servers right like
jenkins or team city or bamboo or azure
devops or whatever you know other
information such as what environment was
this constructed on you know being able
to actually have the information on how
it was constructed maybe some various
settings and some of the stuff you want
to keep to yourself but others you want
to be able to provide you know what
settings and what version of these of
these components were also used to build
this because sometimes
some of the products that are being
utilized in this case can be
questionable uh by certain people in the
industry i’ve actually seen this before
where there are certain times where
certain types of processes are frowned
upon by certain agencies because of of
regulatory standards that they have to
have
so what is it used for right when it
comes down to it so when it comes down
to what it’s used for it’s for anybody
who produces software right i mean what
it comes down to is that it’s simple as
that it’s a way to also have it so that
anybody who’s purchasing software or
somebody is choosing can get the
information up front once again it’s
like going to a supermarket and grabbing
a piece of food off the shelf looking at
the you know the ingredients on and
being like ah crap pine nuts i’m
allergic to pine nuts i can’t have this
right it’s the same kind of idea it also
allows people who are purchasing your
software to go ahead and even from a
legal aspect there’s also a section on
this that also includes licensing the
thing is is that license compliance and
governance is also a huge strategy
behind this that when people look at
this they can go through and say okay
they used all these open source
components oh that’s pretty normal okay
um but also too you know what there’s a
couple of licenses in here we don’t like
to adhere to you know maybe it’s you
know like it’s the idea of like uh you
know having like a gpl and an lpl right
it’s like it’s you know they don’t
really coincide well together and being
able to look at it and say well we don’t
accept these types of licenses uh
because our organization has mandated it
from our legal team i i’ve heard the
slew of different excuses from various
companies over my years on you know
certain components and i’ve had to do
this in the past with some of my other
software products um some open source
stuff that i attributed to um on and
also stuff that i’ve done in the
corporate world and the thing is is that
this is really a way of accountability
right for asset management um you know
license compliance management uh being
able to also too if something goes wrong
look at solarwinds look at log4j when
log4j came out we actually released an
open source tool tool for people to go
ahead and search the software that
they’re utilizing to find all the
instances of log4j right because it’s
been used by some by so many people or
the thing is they want to make sure they
can go hey bring up that s4 you know
this s-bomb for all the software
reproduce let’s do it you know we’ll use
either cycle and tx or spdx we’ll talk
about those formats in a little bit but
you know let’s go ahead and let’s let’s
write a parser or let’s put it into a
software build materials you know
management system and let’s do a search
to make sure that none of the software
that we built or is purchased actually
has this potential threatened side
because identifying this is actually a
very very very important thing
so the thing is is that you know this
helps identify uh you know all the
potential threats that might be in the
software you produce and provide to your
customers both for yourselves as a
corporation or as a developer who might
be delivering you know building out open
source projects uh for your organization
that you work at so this way you can go
in and say hey by the way once again you
know what we discovered that four of the
products that we’re currently using what
right now contain this potential threat
we should really contact the actual
vendor and ask them it’s also a way for
you to go in and manage the licenses
that you have like i said do never never
never discount the power of legal right
we and most of you guys you know if you
haven’t been directly affected by this
um you you you know i have
a couple of times um you know having to
go in front of in front of legal and
explain uh the reasonings why and then
having to go back with my tail between
my legs and say all right guys we need
to go back and we need to go ahead and
refactor uh based on what we’ve learned
uh from a legal standpoint um we’re not
allowed to use this or we’re not allowed
to do this whatever um it’s also a way
for you to go ahead and like i said and
also uh you’ll get a comprehensive
information on you know on how something
is constructed now this helps internally
more than external right you can exclude
the actual environmental and system
information uh that you provide to your
customers because most of the cases you
don’t have to but one of the things that
gets exported as part of that is this
and this actually can help you also
track over time um you know
how you’re building your software have
you changed anything uh did the version
of python change that when i built this
thing you know we we upgraded the
version between this and this and since
then we’ve been having issues uh with
you know with velocity in terms of speed
of where we’re doing our uh our
transactions uh you name it right
there’s a lot of information in here and
at the same time too one of the greatest
features of this is it actually reduced
as a bunch of actual cost internally in
terms of having to do this i just had a
customer recently who talked to me and
said hey with the stuff you showed us
our root cause analysis that we were
doing on some of our particular builds
used to take us days or weeks to compile
together and bring it to the team and
get an assessment now we can go in and
do a search pull together and it’s
minutes or hours right so you know we’ve
actually cut down on the amount of time
it also allows you to go in and make
sure that when you’re doing your build
processes you’re not now wasting time
trying to figure out why you can
actually enact it quicker because faster
responses
actually means that you can get back to
work better and also provide faster
features and and hopefully you know or
fixes to you or some sort of remediation
to your customers and when you’re doing
this one of the big benefits behind that
is is of course you know making sure
that you ensure that trust level with
your customer base right so it allows
you to act faster lack quicker remediate
more effectively and then you know do it
in such a way that you’re not wasting
time um trying to find this stuff in
this in the meantime you’re you know
your your customers have gone to talk to
your competitors who have said hey we’ve
already addressed this because we did it
you know we did this like a week ago and
you’re like here five weeks later here’s
the patch
that you need to do sorry let me take a
little drink of uh coffee here
now uh you know
i’m sorry to interrupt there’s a
question here i didn’t know if we wanted
to address it now or um sure let me go
ahead i can i see i don’t know if i can
see there i can post it on the oh i’m
sorry i i actually no i actually have it
well i have it on one browser window and
i have another so i didn’t see it over
here to the side oh cool
um i can try and rephrase it a little
bit too but it’s basically just log4j
had three critical cbes that go back to
2017 so
yes
four or five years ago now which is
scary um well i didn’t you know why i’m
here fba and back then i know that you
can’t speak for the industry but what
are some of the reasons why
now is the big problem
well it’s very simple is that i think
what it was is that all this stuff has
been here like i said for actually for
much longer than even four or five years
ago with even some of these components
right some of the stuff has actually
been going on far longer than that and
this has actually been something
inherent in the industry for for forever
um
it it wasn’t until it became major i
mean like i said the one that really
that really sparked everything was of
course solar ones it became an i mean it
was on the news more than anything i’d
ever seen i was like when i read about
the initial solarwinds thing i was like
oh this is going to be terrible but i
didn’t realize how bad it was going to
be until i turned on the news and it was
on every freaking news station that was
out there and this suddenly became the
thing and and it’s interesting because i
think it raised awareness like i said to
something inherently terrible uh in the
industry sometimes that’s behind the
scenes that we just deal with right
we’ve just dealt with it over time in a
lot of cases you know most companies
i’ll tell you right now almost
i mean so many companies i know
do not check the third party libraries
they use so the even the log for j even
with those high cves so like back in
2017 and 2019 and i think there was one
actually in 2014
and that’s just no 2015 maybe it was i
can’t remember exactly off top my head
most companies were completely unaware
because they never checked
never checked
right nobody ever really sat down and
there was a lot you know like they’re
like oh we might run a you know uh an
analysis to say you know is there some
things but this has been around for a
while but you’d be actually surprised
how many major
companies out there in the world are
only recently over the past couple years
really starting to adopt the idea of
this kind of level of security even
though there’s other products out there
like snake which is relatively new but
you have like prisma that was you know
part you know that is now part of
synopsis or you know aqua sec right you
know like that or even like for jfrog
and us right with x-ray or you know the
thing is is that you know you know all
these tools that are on the market the
adoption of them are there but the
actual implementation was actually
sparse in a lot of cases
um i hope that may i hope that makes
sense
all right
did i answer your question
yeah yeah i think so
yeah let’s see
we can have no vulnerabilities i see
here it says but it takes it takes a
known exploit to make it dangerous as
yes yeah that was the biggest threat
yeah absolutely dude seriously like it’s
it like i said it take you know like the
you know the solar woods thing was just
a spark you know people all of a sudden
people started questioning right i mean
like i said this stuff has been around
forever i mean how many exploits have
been there out there for like the iphone
where they’re like oh you go to this url
and suddenly your phone goes into it you
know an infinite loop right because it
was exporting a bug somewhere in a
software library or you know or you know
somebody you know all these records got
leaked and the reason why they got
leaked was is that people like all the
hackers got in right but how did they
get in in a lot of cases it’s just
exploiting something that’s been there
right you know they just happen to go in
and say hey you know what i’m going to
try all the stuff i have to get in and
hey look you know what these guys
actually had an rpc thing that was open
uh i took a look and i you know i it was
a web page i right clicked i’ve used
source i traced all the things back into
it because i didn’t do a very good job
of hiding um anything so i clicked in
and i said hey look i happen to know
there’s an exploit here and i know how
to do it right i mean there’s there’s
you know there’s these things that just
happen over time it’s it’s it’s insane
to me sometimes but now it’s like it’s a
thing right and then now the government
said now that the government’s actually
recognized this is the thing now it’s
even bigger right and it’s become you
know more parents and more into more in
the industry and now everybody’s trying
to fire fight your you know we’re not
the only ones talking about software
supply chain security now right i mean
the number of talks out there is is
incredible you know when i when i
started giving these talks like
throughout four years ago people were
like oh that’s cool you know it’s a nice
to have right it was originally it was
like oh this is nice to have you know
what we have you know we’ll probably use
it and i’m like all right and now people
were like how do i get my hands on
something that’s gonna let me know right
it’s become it’s now become a mandate no
there’s companies that have budgets now
they’re like we need to address software
supply chain and the problem is though
we also want to avoid things like i
don’t know if you guys are familiar with
the term like cargo cold thing right you
know this always happens if you work for
the big industry your cio your cto
you know a vp of something goes to a
conference sits down some guy gets on
stage does a big scary speech about
something and says this is where you
need to go and then suddenly the cio
comes back and says hey managers get
together we need to do this we need to
enact this as a kpi in 2022 get it done
without understanding the context and we
see this all the time i actually give a
song actually i gave a talk with this
guy named baruch who’s also a guy i work
with give us we actually give a talk
called songs of ice and tire fires and
the whole idea behind it is
that
you know there’s all this talk but now
there’s and there’s little action in a
lot of cases and a lot of it comes back
to uh not having the context behind
doing something right not having the
context to make the the proper decision
and this was one of those cases
initially right we saw a lot of we saw a
lot of firefighting coming from
companies where we suddenly just saw
this ramp up of com customers contacting
us going i’ve got to stop this you know
and we’re like okay
what give me your tell me why
and a lot of companies could never put
into words to be honest we want to be
safer
that’s great everybody wants to be safe
you know what i mean we want to protect
our customers of course that’s your job
you know your job is this isn’t a nice
to have this is your job you know to
protect your customer base um but the
thing is is that and nowadays this has
now become a thing it’s it’s really
amazing like i said solar winds and the
cost associated scared the crap out of
tons of companies um they’re worried
about liability oh my god we did
something terrible you know we use
something and then you know there became
that whole let’s throw it all away and
start from scratch and that’s not the
way to do it right it’s you know logical
approaches and calm and cool minds will
actually prevail in this case but having
the right tool set to make those
incredible decisions is also a big
factor too and having knowledge and maps
about what you do help you actually plot
a better course right and that’s what
software build materials really does
it’s actually another enablement feature
for you to have a better course of
action in times when things go terrible
makes sense so of course what’s that i’m
sorry go ahead i’m sorry
oh no i just said it makes sense i’m
sorry i’m getting out of here it’s all
like
no no no it’s cool it’s all cool um so
actually what i’m going to do is let me
uh let me move my other window to the
side so i can see if you guys are asking
questions i think that’s a better idea
um oops give me a second here can you
guys still see my screen still
cool
all right so software contains a lot of
stuff right i mean that’s what we’re
just basically coming down to there’s a
lot of components inside so i’m going to
talk about cake because i like cake like
i said i did this i actually put this
together when i was hungry uh my bad
sorry if there’s any diabetics out there
i apologize now don’t cancel me please
um
but you know at the same time you know
this is really it’s a really good
analogy to actually you know go ahead
and do you know look at the cake right
it’s a nice beautiful cake it looks very
tasty if you don’t like chocolate i’m
sorry um but you know at the same time
right it cake is made of many many
different parts right and your software
is too you know we know somebody made it
right so this is we know that we also
know that you know somebody had to mix
this together and do something we know
that it was put into an oven unless
you’re doing of course one of those like
cakes that you put in the sun and it
bakes and stuff like that whatever um of
course you know it might be decorated or
not whatever it could be tasty it’s
probably as tasty like i always make the
joker saying and we know it used
ingredients we know that there’s stuff
in here you know that the end product is
the cake but there’s accumulation of
parts that made up the cake
so when we look at this this comes back
to it again right so once again here’s
the cake and here’s the ingredients but
there’s also so much more involved with
it and this is what i was talking about
before which is also too you know we
know there’s instructions how to
assemble these actual pieces that made
this cake up and this comes down to your
environmental system information your
your you know what you know what ci
server was it produced on your desktop
what version of the actual software did
you use did you use like i said you know
are you using python 2 7 or using python
34 whatever you know i mean it doesn’t
matter what you know what was the
execution that you actually used to
compose your software and it could be
everything from compiled software to
real runtime software you know we start
getting into like mpm and web services
it can be docker images whatever
whatever is used to assemble the things
that you do has not only components but
it also has instructions
but what if we change the instructions
too and this is where i was talking
about before by having a software build
materials what if you know instead of
using baking soda right that we’re going
to say here um what if somebody actually
used baking powder and you’re talking to
somebody here by the way um who’s done
this by accident um i you know when i
was less skilled um you know in the
actual kitchen that kind of thing um you
know you get different outcomes but you
know understanding what went wrong is
really
the case here you know baking soda on
the left big power on the right yes
there is a difference
so when we look at this you know let’s
go back to our software example again
right here’s my software you know here’s
those transit dependencies i used to
build my software and then what i used
to do it gets put into a software bill
of materials right so the software bill
of materials is the list of components
and ingredients that i use to bake my
software and i can supply this
information to my customers as a level
of assurance basically telling them that
i’m willing to show them or i have to
show them because i’m going to tell you
right now when a lot of these regulated
industries are going to start demanding
this level of information from your
company or from your your contributions
to software because they want to make
sure they can protect themselves
and if we go in and say we changed it
say we did change out you know baking
soda for baking powder you know this is
going to change the way the ingredient
you know the way the actual software is
cooked or in this case you know our cake
is cooked you know we can go in and we
can adjust the software build of
materials so this way we can actually go
in and then somebody can compare these
two software build materials for two
different versions and say okay there is
a deviation change between these two
levels of software
and if there’s something terrible inside
this also means that i can go in and see
in my software buildable materials is
this part of it i read something like i
said in hacker news that said hey by the
way oh sorry i made you hungry dude um
but you know um software-built materials
you know the whole thing is is that you
know i can go in and say this component
is actually it does contain it and the
other thing too is that when i start
collecting all these different software
releases and software builder materials
i can go in and say oh by the way this
nefarious component or you know this
this one you know unregulated piece or
this piece with a faulty license it
doesn’t matter here’s the versions of
software that i have it affected this
means like i call it blast radius when
anything is actually found in terms of
third-party direct or indirect
dependencies in software you want to
know the about you know the blast radius
because it also by the way you know the
thing is is a lot of these components
that you might be utilizing are also
used at other pieces of software not
just a single piece of software
especially if you have multiple
different types if you’re running like i
said like a multi-tier web service or or
whatnot the thing is is that
understanding the effect of something
that could affect your software and
understanding how far it went back and
how long it’s been introduced this
actually goes back uh to what john said
before right you know cve is going back
to 2007 and 2018 at 2017 and 2019 right
and the thing is is that you know being
able to understand how far back this all
went is is very essential and this
allows for companies to have risk
mitigation uh remediation you know
releases and press releases to say yes
we’ve actually known this let your
customers know your customers would
rather be on you’d be honest with them
to say look we addressed the issue and
just so you know
we realized nothing affected but you
know what this issue went from you know
this release today that we just gave you
without the issue all the way back to
2015 we didn’t know here’s a report on
the issue was only discovered recently
or it was only exposed recently but just
let you know it affected other
components too
so what about cakes with a lot of layers
and i see there’s a question here that
says let’s say i create an s-bomb in an
app i deliver to you if i add version
numbers does that over share my phones
to you
no actually um so you actually need to
include the actual version numbers of
the software uh software build materials
one of the components of it if you look
at the spdx or cycling the x format is
there is versioning because you are
going to want you do need to make sure
that you give the actual version number
of the actual software component that
you have
but
let’s talk about you know like what
about a cake with lots of layers right i
always love talking about this right you
know each component was created
differently right each has its own set
of ingredients uh each could have been
made by somebody else right you don’t
know it might have been assembled by a
team of people it might be different
teams of people it could be different
companies right um you know what
together they do something in this case
uh once again i’m sorry for making you
hungry but you know once again they come
together because like in this case you
know i might have made the cake but i
bought the frosting you know and i’m
putting a trust in the company that
produced the frosting you know i’m
saying oh you know what this frosting is
good i’m gonna probably eat what’s left
over with a spoon afterwards you know
when i haven’t used everything but how
do i know that that that’s safe i’m
putting trust in the manufacturer of the
actual uh in this case you know the
frosting itself or the cream layers in
between might be another thing right and
the thing is is that
you know it might come from different
locations even you know and
and depending on its purpose the thing
is is that you don’t know um you know
what each piece will actually do in some
cases you don’t care right you might
just depend on it but at the same time
you want to know everything about it
so you know if you were to look at it
like a web service right i always say i
love my web services layer cake right um
each one of these is independent entity
that’s part of what you’re trying to do
right so like i said you know the
frosting is helm
helm frosting um you know then we have
layers you know you know zero one and
two and then the base cake right these
could be independent each one of these
would have its own separate software
bill of materials
so when you look at it i could say a
docker image even even a docker image
will have multiple layers inside of the
software build materials because you can
still release your software as a docker
image right everybody knows this you
know some customers are like we actually
virtualize everything here’s our helm
chart download and install our software
and run it right once again there’s an
inherent trust here’s the instruction
set i gave you to go run the software
it’s going to pull it from somewhere and
bring it into my system and run it well
the software of the materials will also
encapsulate the docker image it will it
should include the os the runtime and
the application layer each one of these
is independent this also includes system
environmental settings inside the
container itself what were the paths you
know what version of the of the runtime
was used to actually run the application
this is also good for you as a developer
because then you can also say if this
one was running worse than this one what
changed between these and by having a
software build materials basically build
information on what you’re producing you
actually have the ability to go in and
make better choices and also go and say
quickly oh my something did change
between this you know what we actually
changed the path on where the x the uh
the execution of the runtime is and oh
god bob left on uh oh thanks a lot bob
bob left on the debug flag again right
um so everything’s slow because it’s
filling up my logs and uh we have to go
back in and we have to do another build
and we have to do another deployment
without debug flag left on
but let’s talk about web services right
so each one of those containers would
have its own web service even the helm
chart right can even have information
around it and at the end of the day when
you are building this if you are if you
do find something nefarious you can go
ahead and look through all those build
materials and find out what it affected
so the thing is is that you know when we
start talking about you know s-bomb uh
you know stuff right you know things you
need to keep in mind right of course
it’s all about devsecops right i mean i
by the way i’m just so you know and you
guys will probably get mad um but like
i’ve stopped even using the term devops
i i have um i i don’t think it’s it’s
really appropriate anymore i think
devsecops is should be the term from now
on i think every level of security
aspects should be part of our uh you
know part of our vocabulary should be
part of our terminology um
i think that you know i i see uh the
world as a better place you know like
world peace you know so i feel like
right now but you know the whole idea of
you know there’s a level of
responsibility we have as developers as
you know whether we’re release managers
whether we were um you know whether
we’re release managers whether we were
uh you know software developers or
security engineers or qa whatever
there’s a level of responsibility we
have and understanding you know like
what’s ci tooling you know when was the
software built you know my ability am i
using something that’s five years old
right you know do they you know
internally what stages of the of the
sdlc right software development life
cycle did it go through i want to know i
want to know qa’s data on what they
tested when we released this product and
how did they get through you know how do
they slip through their hands right i
could be that guy um you know what foss
software was used right once again it
comes down to the whole idea you know
freezing speech not as a beer right um
the idea of you know what frost pro you
know what was used when actually
compiling this what environment was this
done right you know in this case what
oven was my cake baked in right i might
want to know these things you know
because maybe in some cases where i’ve
seen i’ve helped uh some you know some
of our customer base they’ll go through
and some of the issues they run into
actually might be in the process that
you’re actually using it may not be in
the software tools they’re doing it
might be something injected accidentally
as an artifact somewhere inside of what
they’re doing right as part of the you
know the build procedures and i also
want to know you know where’s you know
any of the security and vulnerability
and analysis behind this because it
should be part of the process these are
things that we need to keep in mind of
course but on top of that you know
understanding you know the composition
of everything we do through the software
build materials it’s like having a
really excellent map and being able to
quickly go ahead and look instead of
trying to find out did somebody change
something
so
some of the misconceptions that we get
all the time uh you know i’ve actually
know you know is this a road map to the
attackers no it’s not because really
what it is it’s just a list of you know
materials i mean look at look at
something like i said you know um you
know the cake example included
instructions but what happens if i get a
software build materials in this case on
on twinkies right i look at twinkies i
look at the ingredients on that some of
the degrees i can’t even pronounce but
the thing is i know like there’s red
five or something you know like in some
things that i do or whatever you know at
the same time it’s just a list of
components without understanding how to
assemble those components it doesn’t
make a difference yeah there could be
information in there on a you know
susceptible library but hopefully you as
an organization has actually gone in and
attributed to that
no it doesn’t require source code you do
not need to give source code at all i
would actually if they anybody asks you
for your source code unless you sign
some sort of massive legal document that
says that they’re not going to do
anything with it i mean i have worked
with companies uh we have actually
in the past with another company that i
had they were like we want to keep your
your source code in escrow um just in
case you guys go out of business and i
thought it was the most bizarre request
i’ve ever had
but in this case you know it wouldn’t
that wouldn’t be part of it right the
software build materials does not
require software it’s a soft source code
it is simply the list of things that are
in there and then also too can expose my
intellectual property once again you’re
you know with it it’s like it’s like
being it’s like it’s like magic or
something right you know you can have
all the ingredients to make the wicked
brew that you want to do to turn
something into something else but unless
you have the actual instructions or you
know you might have some knowledge that
you want to do it you’re not going to be
able to do it you don’t have to expose
any intellectual properties you’re not
releasing your algorithic structures
you’re not releasing any of the magic
secret sauce that you’re doing with
these components that you’re doing
you’re simply supplying a list of
components for people to work with you
know to look at to say this is cool this
is not or you know if there’s software
change that have taken place does
something change also too in a lot of
cases now companies just want to have
these on record right because in the
case for you know we’re in any sort of
regulatory thing regulations in some
cases apply where they need to have this
on you know on location it also comes
down to if you have your own company and
you get into an m a situation and they
ask you for any information about your
software you don’t have to be like me
where we went through and spent six
weeks pulling this together and also
removing any potential things that might
have actually caused the deal to go sour
because we used maybe a library that was
a paid library and they don’t want to
you know use that took us weeks and
weeks to pull that together this we
could have actually if we had software
building materials we could have looked
at it hand it to them and their legal
team could have given us back to us in
like a day and a half or two days it was
terrible
so remember
number one if you’re working with the us
government you need this to work with
the us government now going forward it
needs to list all the foss and also paid
libraries so anything you use to compose
it right you need to you know in some
cases you need to have it you don’t
really need to give it to them but in
some cases they will ask you you know of
course how what and when something was
made right you know when was this done
what was used to compose it and how you
know how old is it in a lot of cases you
know are you giving me the latest
version or am i using a version from
last year um on top of that for
audibility traceability purposes a clean
accountability of all the software you
do and also for software license and
compliance
um
once again uh by the way in the
beginning we said you know here’s you
know for 25 first 25 people
everybody here today by the way you can
go ahead scan this qr code um this will
also allow you to go ahead and we say
you know book and shirt combo for the
win but there’s also another giveaway as
part of this um and then uh overshare
wait did i overshare uh but thank you
that’s all i got today guys um any
questions or anything i hope this has
been helpful um i don’t know if this is
what you exactly wanted but i tried
let’s see here it says we need to we
need more tooling to go through all the
s-bombs we receive and let us know what
the vulnerabilities are
yeah i’m gonna let you know so you just
let you know i mean like
you know i’ll just i’m not going to do a
product plug but i was going to say is
like in our product actually anytime you
actually publish a building to our
product artifactory um with our x-ray
product we actually have now the export
feature to do export spdx and cycle and
dx formats with all the components
behind it
um and it says what tools
do you use to automate s-bomb generation
um yeah there’s other ones out there too
right of course you can do
but like you know if you’re using like a
binary manager like us this is just an
easy model to follow um in a lot of
cases there are some things out there we
actually i think you know we’ll see more
tools that i’ll go through and do
parsing um so like uh we we actually i
have a like we have a cli tool where you
can actually go ahead and do a
comprehension model and component
analysis of all the pieces you’re using
and get back to various components but
you can also there’s other methodologies
there are other tools out there
that do that and it says what level of
granularity is required to create an
s-bomb
ah
so the thing is it’s everything so it’s
not just the implicit or the actual
direct imp dependencies that you’re
utilizing it’s also all the indirect
dependencies that come associated with
it it’s like it’s like inviting five
people to your house when you’re in high
school for a party and 200 people show
up right you need to make sure you get
everybody’s name of those 200 people for
better or for worse in case they trash
your house right i’m not saying i know
this from experience
um let’s see here we would love to see
we’d love to see jfrog join in here to
be sure artifactory supports the
workflow we were thinking
um
yeah so we’re actually we’re so we’re
governance board members by the way of
the uh of the cloud native foundation um
just so you know right so um you know
that’s one of the things that we are um
actually every time you got anytime you
guys use kubernetes you can thank one of
our employees named remus he’s actually
one of the co-creators of helm
um so we do a lot of this but yeah i
mean i would love to uh by the way if
you guys would you know if you guys are
working on this i see that there’s uh
you know uh
any time please feel i would love to be
part of this whole thing uh we’re we are
super into the community we’ve been
doing it for you know we are a major
contributor through you know open source
and stuff for for years now um you know
i personally have been part of uh you
know open source products in the past pm
for a company called antenna a long time
ago we produced the open source uh
compatibility stuff around parsing and
linting and all that stuff back in like
2007 2008 uh long ago
but um yeah i mean um any questions guys
i mean uh besides that i mean and thank
you for your time and i’m honored to be
your first talk of the year i hope you
guys have enjoyed it um having me here
so i’ve enjoyed doing this
cool cool yeah we absolutely appreciate
you being here and speaking with us and
wanna thank you for taking the time out
of your busy night uh or busy afternoon
uh
um the time difference thing is all
messed up now right yeah exactly oh
weird i’m a california guy i’m in san
jose california right now so right but
we do absolutely appreciate you coming
in and sharing your knowledge and it
seemed like you had some good
discussions here too so this was
i hope this isn’t working off 2022.
we’re on guys right on well thank you so
much just a few more things for
everybody else listening so we have a
talk i believe it’s going to be february
17th
that’s going to be uh mike mccabe ken
toller
it’s technically scheduled for the third
thursday in
february so i think that’s the 17th
usual time and they’re going to be
talking about scaling cloud security
so please uh come in and come back to
our regular regularly scheduled time
i’m also going to try and push this year
for more interaction with our slack
channel so
right now it’s just abdullah and sean
and i being kind of alone in there uh
trying trying to organize everything um
so i’ll post that
invite link in our um meeting
uh maybe throw it in a couple of emails
if i uh send those out for preparation
for next month
and one last thing we’re starting to
send feelers out there for
more speakers
and for
um potentially helping
us
manage this
owasp meetup chapter um and planning for
the upcoming meetups and helping us out
with social media twitter etc so if
anybody’s interested please reach out to
myself please reach out to abdullah
please go out to sean or you can do in
the slack channel i’m going to just
throw that one out there um
so
thanks again for all of you for joining
us and
thank you again bill for speaking
tonight let me just put the
invite link for slack in there um and
then
cool
so then we are all ready to go so
see you next time everyone and thanks
for joining us bye everybody thank you
thanks everybody enter the raffle get
your shirts guys yeah yeah serve
everyone rock on be safe be well
everybody cheers take care and see ya
thanks