What does a Java Developer Need to Know about DevOps? @Denver JUG

welcome everybody uh welcome to the boulder and denver java users group um boulder’s going to

do something in person and chris weiner will talk about that in a minute i’m going to continue with our shortened

format and if you don’t like it let us know but so far people have appreciated us just getting down to business

we’d like to meet again in person we’re trying to do that for the november meeting and we’ll do a broadcast as well

so if you can’t make it in person or don’t feel safe you can watch it from home if you have a large venue that

would accommodate us for having vent cat as a speaker without a lot of extra costs let us know

reach out to myself or matt or go through the meetup group so we can try to figure that out

email us any topics that you’d like to see reach out via the meetup group

big thank you to intellij agile learner we do giveaways

zeddy will handle those drawings thanks to matt rabel who is our speaker wrangler and to octa for hosting the

broadcast on through stream yard and then on youtube um thanks to jfrog

there should be a link in the chat they’re doing a giveaway as well just click on the link to do that

i’d like to go ahead and bring up chris weiner from the boulder java users group

with his announcements and then i’ll introduce the speaker hi everybody and thank you very much

greg um this should be a really good uh talk tonight melissa is an o’reilly author

and international speaker uh we’re fortunate enough that next monday we’ll be meeting at

rule four in boulder rule four is outside magazine’s place of the year to

work it’ll be a good meet up catered hot food

beverages beer you name it and it’ll be a combined meet up it’ll be

boulder devops colorado kubernetes and cloud native and also the boulder java usage group

so it’s just off the foothills parkway easy to get to if you have time to come up

and that’s it from us thanks greg thank you chris and obviously go to the

boulder meetup group and you can get details in the location uh and the address so

tonight’s speaker is melissa mckay and the topic is what does a java developer need to know about devops i’m excited

about this and um and and really it’s it’s any developer right but i know we’re going to talk about java

specifically because at the state of colorado we’re using azure devops as a way to

um you know build and deploy using devops tools to various environments and

uh using some different pipelines and so just really understanding that environment and how to pass you know

variables down uh and and automate it completely as of great interest to me and so i really hope you guys all enjoy

that so uh go ahead and let’s uh bring up melissa and uh hand it off to you thank you so

much for doing this tonight we really appreciate it hey hi everybody it is a pleasure to be here

really excited about this um i am really looking forward to i’m

talking about what does a java developer need to know about devops um this is a

you know kind of a culmination of some of my personal experiences uh we’ll talk you know a lot of high level things

that i’ll be talking about like what really matters really uh

i do a lot of speaking on various topics a lot of devops topics in general and uh

it’ll become pretty clear why that is in a moment but um it just it really matters to me um

given you know i i had to think way back when i first started as a java developer

this was probably over 20 years ago now uh where i was an intern and you know i had learned the

how to code i’d learned you know java as a language but everything that goes on

around it i did not know that was all stuff that i learned on the job and over time

so this is something that’s near and dear to my heart because i think anyone starting out and certainly anyone that

is more advanced in their career they’re going to need to know some of this stuff and some of these concepts so

this is kind of a combination of many of the different talks that i’ve given there was three that i chose

specifically to combine together to share with you tonight a lot to share so i’ll go

through some of it pretty quickly but i think you’ll get something out of this

tonight whether you’re a beginner or whether you’re more advanced we’re going to talk about what really

matters when we’re talking about devops processes and tooling we’ll talk about building and

understanding your dev delivery pipelines why that’s important managing dependencies i think is huge

and as a developer we really need to focus on that part a lot lastly i’m going to talk about building

your artifacts and we’ll get into some details about building containers specifically

i will focus on containers given some of my background and these are concepts i think are really important for us to

know but these are some techniques that can be applied across all languages as well

so some background on me i am a developer have been for many many years

i did start speaking a few years ago and i decided to join the developer

advocacy team developer relations team at jfrog i’m going on my third year so

this is really interesting timing i really enjoy teaching i enjoy sharing what i know i enjoy the public speaking

um however right when i started is when the pandemic happened so we all needed to

stay home and i got very versed in how to do video editing and how to do online

conferences and things like that um there’s some pros and cons there one thing that i enjoy the most is being

able to engage with people all across the world at any time and i still do that today

too i leave room in my schedule for that um like i said i’m a developer advocate

i’m also a java champion that was something that was an honor i received during the last

three years and also a docker captain so i try to keep up with the latest and greatest what’s going on with docker

what’s going on with containers and i have a few talks coming up at various conferences in the future on

those subjects so why are we here today what is devops

let’s start there and wikipedia has the best answer um you know good enough for

me anyway devops is a set of practices that combines its software development

dev and it operations ops it aims to shorten the system’s development

lifecycle and provide continuous delivery with high software quality okay

this is just a long way of saying help developers do better help us all as a

team and also other folks that are involved in a project

help us all be more successful um for required reading i think uh these

are books that were introduced to me many years ago and i actually should be uh

bringing these out again i think it’s time to reread these and uh get these concepts um you know remind myself of a

lot of things but these are the phoenix project the unicorn project and the

devops handbook really really exceptional books the phoenix project is

written from an perspective and it’s not just a an

academic book it’s written it’s like a novel so it’s really engaging it’s really fun

and it follows this team’s problems that they go through with a

particular software delivery and all of the solutions that they come up with and and you know how they start to

streamline their processes a little bit more and the sequel the unicorn project is actually from a developer’s perspective

and and it’s also written as a novel and there’s a lot of the same characters that you would recognize from the first

book the last one is more of an instructional manual on how to put devops principles

into practice and it talks about principles of flow and principles of feedback and principles of continual

learning and experimentation so like i said i always recommend these books i think

they should be required reading for anyone that is on a devops team

i would be remiss if i didn’t uh advertise my own book here so jfrog is

um has a raffle available and there should be a link coming into the comments um if

there isn’t already and uh this is a

form that you can fill out put in your email address i’ve got five of these books i’ll give them out to

the first you know five people that get pulled in a raffle and you’ll get contacted by email in order to get this

sent to you this is devops tools for java developers this was a book that

some colleagues and i got together and just decided to focus on what was most important in the devops processes for

developers from a developer’s perspective and i think it’s a really good introduction to the entire process

okay so this is a slide deck that i often see in corporate decks at jfrog

it is a pretty detailed step-by-step of the software development delivery

life cycle and it is really really busy but i like it because it does a really

good job of highlighting how overwhelming devops can be when presented with all of the stuff in

you know the ecosystem this space is huge and it’s really easy to get lost in the mix

this whole process is an attempt to answer the questions how do i build everything how do i test everything and

how do i deploy everything where do dependencies come from and how do i keep everything secure how do i

include automation to help increase efficiency and reduce human error and then as expected of course there’s quite

a few jfrog products in here that represent uh handling artifact management and storage and distribution

and security scanning on packages but this setup has really strong integrations with other components and

frameworks out there that you’re likely already familiar with and use today these are all the various build and

dependency managers like maven and npm gradle continuous deliver integration servers

like circle ci and jenkins and then of course provisioning tools like chef and

puppet now i don’t really expect you to see many of the logos on here it’s pretty

tiny but it just represents just how much there is in this space what is not represented on this slide is

everything that happens after delivery so a lot of your

you know observations that need to be taking place being able to monitor

everything that’s in production and in your systems and be able to provide feedback of how your service or

application is being used all of that needs to feed back into the beginning of

the whole life cycle for developers to continually maintain and add features

etc now one another thing that this slide really um brings attention to is how

much how much is in our wheelhouse like how much a developer really needs to know

and this is something that comes up all of the time with devops in particular i think there are

good and um wrong ways to have a devops team or to organize a

devops team and for me the whole idea the whole advantage is to be able to

utilize other people’s expertise i mean this is why we have teams that work

together closely it is there is not and should not be an expectation that one

person uh specifically a developer needs to know all of this in great detail

i think it is very important to have an awareness across the board of you know what’s available the purpose of certain

tools and things like that but to take a developer off their task of their craft

and to put them you know on other stuff i mean it really can be a full-time job

so i don’t think that that’s the best use of resources

as a developer keep that expertise but on your devops team you should also have

folks that have operations expertise as well so when you have concerns with

deployment there are specific people that deal with deployment when you have you know a bug that needs

to be fixed code needs to be checked in that’s in a developer’s purview

[Music] so that previous slide can be simplified

a little bit i was on a panel discussion

not too long ago and a question came up about the state of devops today and what might be coming down the line in the

future if you compare like existing devops tools that are out there and and

components to car manufacturing we’re still very early in development on

these things and what i mean when i say that is how many of us would be driving cars today if we had to build them

ourselves and sometimes that’s how it feels when you’re building a full pipeline from beginning to end it feels

like you’re you’re having to cobble together a lot of stuff on your own we’re still putting most of these pieces

together ourselves there’s a ton of levers to pull um but for a car for example we have a

general idea of how to make certain components how to categorize them so they’re easily interchangeable when you

go get your car repaired for example in software we’re still in the process of working out where the boundaries are

on some of these tools that we use how to make these components that make sense

but we have a pretty good idea of what we need to know now to get everything up and running and generally tools will

fall in to these core components or categories we can simplify this a bit break down

all these things we just saw in the previous busy slide into these core components and that includes your source

code repository um that’s your single source of truth for all of your dubs uh continuous

integration being able to build and test on on you know every

system that you have out there your package managers this will you know provide you a full bill of material so

that you understand clearly what’s in your package what’s in your artifacts that you’re producing

and security scanning this has become more and more important in recent years

they said deep scanning of dependencies and containers too if you happen to be using those and then uh continuous

deployment um being able to deploy you know out to the edge at any time

being able to update quickly and then also being able to deploy to any environment as needed

before you jump in um before you start implementing a bunch of new processes

or a bunch of new tools picking up all these things and trying to use them all at once get together as a team and determine

what your values are what are your goals what do you really care about and every

project is going to be different there is usually a driving factor that will help you to figure out what to

prioritize and this is where communication between teams comes into play as well

if your team is prioritizing something completely different than another another stakeholder you are undoubtedly

going to experience some friction and frustration and the best

self-help guidance out there applies to development teams as well you need to define what success looks like it can’t

always just be this vague let’s do better we need to have a you know a definition

of what that looks like so that you can set yourself up for success and know when you are successful

your team’s values and priorities are going to be different than everyone else’s you’re going to have a different

set of skill sets uh there’s really no one size that fits all solution um tonight i’m not going to tell you use

one tool over another uh that’s just not how it works prescribing a particular

tool set to your team would be akin to telling everyone to you know wear a uniform every day size 10. you know it

just doesn’t really make sense and um i’ll you’ll never see me get into an argument about languages either i don’t

especially when it comes to building like cloud native applications and things like that sometimes it comes

down to what what is the experience that that your team has to work with sometimes you need to go with that

change can also be expensive so choose those changes that will advance your team to that measure of

success that you have defined this picture that i have in this slide

is actually my car and i put this picture in here because i wanted it to remind us to know what is

good enough um this car is super old uh i’ve had it for years and years and years i’m just waiting for the doors to

fall off and uh but you know it’s not something i’m going to spend a lot of money on and repair so you know a few

years ago i got into a little accident and you can tell that there’s a little wound on the side of the car well

that was a 12 purchase on amazon to buy a magnetic band-aid and put it on there and you

know just go with that i’m not going to spend my resources to fix that on this

particular car sadly there’s some software that’s like that too so you really need to know what

um not just the goals of your team but all of the stakeholders involved there’s

always money involved there’s time and resources involved so all of those things will come into play when you make

your decisions on how you build your pipelines and where you spend your time

there are two really important aspects of devops and that is inter-team

communication and intra-team communication uh in a devops world

observability of your system is going to become extremely important and that’s often it could be a completely different

team than you and i put this slide in here just to remind me to tell you another interesting story and this is a real

life story about a disconnect that my team at the time had regarding a specific business value that we were

unaware of and most devs have seen a line like this before in their code base not going to

argue whether this should be there it was this is what we had and um i was

actually working on an efficiency issue in the code base um and i was very proud

of myself i implemented a really clever fix um it involved changing this log

statement though because this lock statement was no longer applicable for the very simple reason that the

function itself was no longer named transform media for example and it just didn’t make sense anymore

unbeknownst to me this little minor change caused a huge fuss and at the

time i was working in an open space but there were a lot of product and business

and marketing people that were on another floor in the building our floor had the better coffee so those folks

would always be down in our kitchen uh drinking the coffee and i just over

just happened to overhear a conversation between them saying well you know this

particular tool is not being used very much anymore i think we should discontinue it or you

know find something else that will work better um you know that kind of thing well my ears perked up because when they

named the particular product i knew exactly what they were talking about and i was really interested in why

why would they make this decision and apparently uh they had some dashboards out there

that i think they were splunk dashboards and they were utilizing log statements such as this in

order to populate that information well obviously since this log statement had changed it was no longer populating

what it should be so right away we had a disconnect they had a particular priority and a

particular value system that they were moving forward with and we did as well we wanted quality of code we wanted it

to be more efficient but we weren’t talking to each other about the goals of the product and the business value of

the product so we were you know not in lockstep we we missed we

missed that so thankfully i was able to overhear that and we were able to rectify that situation but that’s just

one interesting thing that can happen if you don’t have that intra team communication

[Music] so if you only get one thing out of this talk today it’s this avoid silos and i’m

not talking about just between members of a team where you know maybe a dev goes off in a corner when puts

headphones on that’s not what i’m talking about i’m talking about between teams

make sure that you have an understanding of what the goals of the project are and

what the values are if everyone is pulling in different directions you’re going to end up stepping on each other

much like what happened in my log story and when your team decides to charge a

hill make sure that everyone involved in that effort is charging the same hill that you’re all moving in the same

direction this means that communication about business goals needs to flow between all

stakeholders from the business and product level to the engineering and development teams

you’ll find there is a lot of focus on the end result when we talk about you know putting pipelines

together and stuff your service or application that’s running in production right now what is

the end result but something you’ll realize with experience is that your end result is going to depend a lot on

everything that happens before think about your software development life cycle itself as a product

consider the processes that you take to get your product to production find the waste what is slowing everyone

down and part of this is determining your priorities for your situation um is

shortening the time between releases the top priority or is there need to focus on quality or on security

and something that the devops handbook points out is that in the physical manufacturing industry it’s really easy

to see bottlenecks you know physically see them in technology it takes more effort we need

to implement those methods of making our work more visible and the best example of this is you know use of

kanban or sprint boards to make our activities more visible solutions like that will help the team

stay organized and aware of what team members are up to but sometimes we forget that these types of communication

are actually meant to help with the communication between teams as well keeping this information limited to

internal team members defeats half the purpose of boards like this now that’s the high level

another level is of course observability tools alerting monitoring

all of those things need to happen as well in order to monitor your success monitor your progress

but consider putting some of those tools to work not just on your end product

in production but maybe along the way as well collect some metrics like how long does it take for my application to build

during continuous delivery is there anything that can be improved there things like that

[Music] all right let’s talk pipelines

pipelines is not a magical word in devops everyone has a pipeline whether it’s

manual whether it’s automated whether it’s high-tech or low-tech it doesn’t matter this word is a bit overused many of the

products and services out there even use the word pipeline in that in the name

and i would um bet that most of those products that focus on pipelines they usually imply some sort of automation

that’s what’s involved in those particular products it’s just um that pipeline itself is really the steps all

of the steps that are required to get your code from development to production and every member of your team

should be familiar with your pipeline if not stop what you’re doing get everyone

together and document the steps there might be some surprises in there maybe

some manual things that are happening that only one person knows or um some

things that are redundant or you know some things that are unsafe

it’s definitely something that you you need to do is whiteboard this activity make sure that it’s documented too often

do i see documentation that explains how something is implemented and forgets to explain why

and the problem with that is you end up with you know members of your team who want

to make radical radical changes but if there’s no uh reason why

you could end up in this circle where a change is made everyone’s you know happy for a while and then you end up changing it back i

mean the reality is is that you know people move sometimes they move off

teams you get new people on and you don’t want to just keep repeating the same thing over and over so if you’ve

had a bad experience with a particular method write about it make sure it’s included in the documentation if you

changed a component out let’s say you chose to use a different build server make sure you explain why that way when

the next person comes along and says hey i’ve always used this previous build server in the past and was successful

let’s go ahead and change our project to use that then you’ll have this documentation to explain why maybe that

isn’t a good idea or you know at least don’t repeat the same mistakes

[Music] for each of those steps in your pipeline

look for improvements you can make this is a continuous activity

where can you get the biggest bang for your buck and we talked about making priorities and as you step through your pipeline

you might adjust your priorities but having priorities is not enough not enough just to say we want to do

better make sure to document your goals and determine what metrics that you can use

to figure out if you’ve met them and you’re successful start out small with goals that improve

some facet of your pipeline by one percent an example of a bad goal would be deploy faster

everyone wants to deploy faster you you need to start with a baseline say it takes us this long to deploy is there

anything we can do to improve that time is it is it worth it is there some other place in the pipeline that needs some

attention first this is an excellent time to consider different tools and frameworks that

promise to help with your flow when you’re considering your priorities and trying to make improvements but this the

most important thing that i want to stress is always have a reason for that change don’t just go switching out

technologies without having some metrics behind it that proves worthiness

it is fun to play with new stuff and there should absolutely be time devoted to that

but implementing new tools across a team needs to have serious thought and consideration behind it

and you need to have buy-in from your teammates on this as well so one thing i’ve learned over the years

with all of the new promising devops frameworks and technologies out there there is always a cost associated with

switching one out for another we are often guilty of seeing the pros uh we get advertised too you know and

convinced uh to make changes but then only after the upheaval of change do we

see the cons you’ve just signed up for different problems so um

i know i’ve seen that happen on teams a lot and i have plenty of stories about like switching out different um

continuous integration servers um that kind of stuff so

be careful um like i said it’s not you know it’s not a uh

if it’s not broken don’t fix it that’s not what i’m saying what i’m saying is time time box your exploration make sure

you always have time dedicated for that but limit your risk there if you’ve got

you know i would limit a percentage of your time a percentage of your resources for new things new technologies new

frameworks things like that [Music]

we this this um there’s a reason for this picture

i was on a team once where we we use source control sure

but we didn’t really have a good method of verifying our builds first i mean it was kind of a

we trust everyone to run a build first before they check something in and merge the code into into the main branch right

well you know when more and more teams were moving to get and stuff um

i remember moving uh to like a get flow type of

um you know method and and it was really enlightening to me because

what would happen to me in the past is uh you would come in in the morning you’d sit down at your computer and you

want to hold back on pulling the latest code because what you need to do first is check your email and make sure that

uh the build isn’t broken first and that you don’t just pull in broken code because then you’re stuck you’re stuck

with that code um you know unless you want to fiddle with trying to you know troubleshoot you know having an

entire development team and there were many many i don’t know 50 developers or whatever you know having everyone trying to

troubleshoot and figure it out you know is kind of a waste of time right so if anyone was guilty of doing this

you know late late late on a particular work day and they just want to get something in and they

do it without running a build first on their local machine they had to bring in donuts the next day

for everyone and uh depending on how much you value eating donuts you might not see that as

a huge problem right off i think a lot of us just accepted it as this is how it was but there is a better way to do that

and um this is where your you can find a lot of improvements i think um in your

continuous integration system so um have your build server take care of

this for you have automated builds don’t merge into the main branch immediately you know be able to have a mechanism so

that developers can check something in but they get notified if it’s not

going to work with the latest changes that are you know available in the code base

it is one more step but i do really really appreciate the

pull request system um being able to have that and to be able to have it set

up to be notified if there are any issues that need to be fixed first before it gets you know merged

in um it’s it’s all you know very helpful

and part of that process too uh this was also a symptom of this problem um

there were some of us especially ones that were new that um we just didn’t understand how the build

work worked um in this particular scenario of course the build server would do the

heavy lifting of a lot of building but there’s a lot of things that you can do as a developer to make yourself more

efficient and one of those is just understanding your build how every component is built together now

in my particular case we had a mono repo that’s a whole nother discussion but within that repo there were various um

modules in there and if you were just to do the basic you

know script to build everything it would take i took like 30 minutes or something to build if you’re making one change in

a module it would be helpful to know what else is dependent on that module you know so that you could be a little

more efficient with your time uh rather than just running a build every time you make a change um

you know running running a build on the the unit test for that module uh would be really helpful then you could you

know check that code in open your pull request and then let the build server handle you know the rest of

it to make sure there’s no compatibility issues with the rest of it it can save yourself a lot of time so

just that slow process that’s something that you can tackle with your junior devs by teaching them

mentoring them helping them learn how the build of the product actually works

not even just juniors but people that are being onboarded it’s very important for them to understand how all of the

components of the project work together

that brings me to avoid skimping on your test and qa

environments especially when it comes to integration testing um i’ve been on teams before where there

just was no test environment it was kind of like we test in production i don’t know how many of you have had

that experience but it can be pretty miserable if you’ve got a low budget project or something like that sometimes

that happens but it’s just hardly worth it and it’s an ex it’s expensive really it is

um it’s better to have some kind of test environment where you can really be able to tell

you know if there’s any integration issues it should resemble production as much as possible obviously you’re not

going to have all of the same resources but with the containerization resources that we have available to us today we

really have no excuse one example of this was a particular project i was on

relied on internal in-memory database unit testing

to make sure that you know everything was functioning correctly with a database well

enough of worked with different kinds of databases to know that that’s

not always the most reliable method and sure would have been nice to have a

test environment that spun up a um you know a postgres database for example the exact version

that we would be using in production to make sure that everything was okay before

we let any changes go by that was a pretty that was an unfortunate uh people a

people for our team at one point is realizing that uh there was a pretty significant portion that just wasn’t

working correctly just because of a difference in sql between the in-memory database and

what we were using at the time in production also

any of the artifacts that you want to deploy make sure they have gone through

the process of q a and test via promoting promotion of the exact same build rather

than rebuilding for a particular environment or for the release

if your environments require different configuration this should not equate to a completely

different build artifact that kind of defeats the purpose right because you’ve tested something but now you’re going to

build something else to put in production your build should happen once

and as that build goes through the stages um it should be that artifact

that backs that build should be promoted any configuration and stuff

that is different between certain environments that really shouldn’t be hard-coded within the project anyway it

needs to be managed outside of the build artifact either through flags

flags on launch for example or config files that kind of thing to make that

you know a little bit more resilient

then we get to deployment um humans are horrible at repeated tasks we

get bored we get complacent uh we always miss something um

i was on a project once where there was

uh a certain deployment uh and none of us you know we were just now you know

learning this is a brand new project it was being brought in from a third party contractor contractors being brought in

internal for our team to maintain and stuff and

uh we were unclear on how it was being deployed and it turned out that there were some scripts that were written and

these scripts were i say multiple scripts in multiple different places and multiple different versions of these

scripts so obviously not the base not the best way to be going about deploying something reliably and repeatably every

time and even more important is like we didn’t really have a good way of rolling

something back to a previous deployment um and prior to moving to a devops team i

did not really know how stuff was deployed i just kind of you know i did my job i committed my

code and i was done i wasn’t really involved in deployment so much but after learning how that worked at least for

this particular project um i realized how much i didn’t appreciate the trouble

that they were going through um just like knowing your build deeply know how your deploy works as

well list out exactly what it takes to deploy and whiteboard those steps if necessary

and most importantly automate what makes sense and you can do that in pieces uh

remember that one percent idea this isn’t something it’s it’s not all or nothing so

do your best you know find the places that you can automate and get that done and you will continuously improve

so let’s recap just that section uh understand your project’s business values and goals that means eliminating

those silos getting rid of those communication problems document your flow and pipeline you may

be surprised by some things there and this will be invaluable for when devs move on and off your project

prioritize and improve by one percent uh don’t get yourself overwhelmed trying to fix everything likely you will solve or

or improve one bottleneck to discover another and you need to move your priorities at that point

make your work and your metrics for success visible and that just means setting yourself up for up for success

by measuring a baseline before you make changes and set up realistic expectations for the outcome and this

will help you with prioritization if you realize during that exercise that a desired change turns out to have an

undesirable cost to benefit ratio that doesn’t match up with your project goals

and then master your ci cd your source control management your testing your artifact management and

your deployment know the details of your build where your artifacts are coming from and how things are being tested and

deployed and then of course automate where it makes sense don’t just automate for automation’s sake automate what is

going to give you the most bang for your buck and obviously this is going to depend on your priorities your

definition of success for the project but you might find that automating only a portion of your workflow is all that’s

necessary to meet or even exceed your goals at that point and you can move on to more fun stuff you know new feature

perhaps

okay we’re going to take a short break here and i’m going to just talk a little bit

about rube goldberg machines because i like them very much i think they’re cute they’re fun

this is a pretty easy internet search these are very popular but for anyone that is not familiar a rube goldberg

machine is a ridiculously complex and inefficient machine usually it’s composed of a high number of moving

parts and it only it sets out to achieve one specific goal

and rupe goldberg the person was a pulitzer prize winning inventor

he was a cartoonist and he made these machines popular through the cartoons

that he drew that was something that was new i didn’t realize that the one shown here is called the

self-operating napkin and you can see all of the necessary steps labeled obviously this isn’t a real life rube

goldberg machine we’ll see a couple of examples of those in a minute

but another interesting fact in about rube goldberg is that he is the only

person whose name is in the merriam-webster dictionary as an adjective

so i thought that was pretty clever another

another thing i uh was able to find out is there’s actually a contest held every

year for participants to create these um you can go to goldberg.org to find out more about that

and each year has a theme uh this this last year was um the focus was on

literacy and the challenge was to create a machine that would open a book and you can find videos of the top 10

selections of the machines that were built they’re super cute there’s a lot of very clever kids out there that are

creating these so this is fun um but where am i going with this

and this this is what i’m talking about uh we talked about goals for your software

delivery life cycle i’m going to make the argument that actually putting it together and implementing a pipeline is like a rube

goldberg machine but i want to expand a little bit more on that analogy and there’s three characteristics that i think of when i

make that comparison um there’s a lot of moving parts uh there’s inefficiencies

and there’s a tendency to be unreliable and unsafe so this book that i was involved in

writing um i empathize wholeheartedly that software

development and delivery is not an easy problem to solve especially if you’re new to today’s landscape if you’re

involved in cloud native applications all of those components and tool chains that could be potentially involved in

your delivery pipelines became very apparent to me during writing this book

this was published earlier this year devops tools for java developers and my co-authors and i came together with this

intent of gathering for readers much of the knowledge and background that’s required in order to develop and deliver

software efficiently and safely these are a lot of the topics that we

discussed in the book and this material gives you a really good idea of what you’re getting into and what

it’ll undoubtedly result in you wanting to take a deeper dive on many of these subjects in order to develop a solid

development and delivery pipeline we did discuss some of the most common tools used in the industry and we even touched

a bit on kubernetes here but what becomes abundantly clear is that the entire process can be overwhelming

significantly more so in a cloud native environment [Music]

i did a quick search for the largest rube goldberg machine and it turns out there was a video posted on december

10th uh 2021 last year this whole machine takes four minutes

and 26 seconds to run you can see how happy they are at the end

that’s how i felt after getting my first pipeline successfully deployed

this made it to the guinness book of world records and it had 427 steps involved

[Music] uh many of you have probably seen this already this is the cncf cloud native

landscape map this is from the cloud native computing foundation which is under the linux

foundation and i don’t expect you to be able to see a lot of the logos here it is massive

but it’s just a just to highlight one of these smaller sections um labeled orchestration and management there are

20 items that are categorized under scheduling and orchestration tools and frameworks there are seven under

coordination and service discovery there’s 10 under remote procedure call there’s 21

under service proxy that’s where you’d find something familiar like nginx

19 under api gateway i thought that was interesting and then there’s another 17

under service mesh and i love

jigsaw puzzles um that landscape became a meme and an earlier version of it made

the rounds on twitter a couple years ago as a thousand piece jigsaw puzzle

um i was thrilled when i went searching for this and it is actual it was an

actual thing available on etsy but apparently after selling 42 of them it’s

now sold out if it ever comes back i might consider

getting it i like jigsaw puzzles enough i’ve made the case for a lot of moving

car parts um but the fact that there is a lot of moving parts um that’s not necessarily a bad thing in and of itself

but what you choose to use for your particular use case and how you use them that can lead to a measure of

inefficiency which is the next part if you’re new to this landscape it’s going to take you a while to find your

place this is reasonable to expect it will take the expertise of the entire team from development to operations

although some have been involved across the board from beginning to end it’s usually expected to take some years of

experience to be able to do that it’s a tall order to ask one person to know everything and we discussed that at the

very beginning of this talk that that’s just not efficient to have one person know everything

um or be expected to anyway this is something that you have to grow over time with uh to add these things to your

toolbox um this is one of the misconceptions about the uh the whole

devops methodology that i come across all the time um you know putting that responsibility on

on everyone you know having developers know everything about ops and vice versa just doesn’t work out very well

just know that you will make mistakes but the important part is to learn and continuously evaluate your solutions as

you go [Music] this rube goldberg machine dates me a

little bit i actually remember using a typewriter i also remember running out of the white correction tape that you’d

have to load into the typewriter um this one uses a drill to press on the

space bar but it reminds me of something really specific with software delivery

pipelines and it’s one of the more interesting phenomena that i encounter most it’s that software tools

can be used in ways that were unanticipated and often in ways that are unintended by the creators and the

designers and this can be to to a number of different factors um

the first just a lack of due diligence when evaluating a tool for use or simply an attachment to a tool

a desire to make it be a one-stop shop make it work for everything

development of a pipeline that’s ad hoc that that implies a lack of planning or agreement on what actions are required

and where this can actually lead to several different tools being used that overlap

that are redundant perhaps the unavailability of a feature

that you want either the team doesn’t know where to look or they aren’t allowed to use available

solutions because of licensing or other corporate restrictions that can happen

and those are all common reasons but regardless of why this can cause an assortment of issues and one is that you

end up with glue code that can be a maintenance nightmare an example of that

of glue code is that it provides some kind of compatibility between two different tools that might

make sense at the time but then all you need is one of those tools to be updated and then either the tool

you know it could happen to either side and then your glue code is broken um that’s a maintenance nightmare

another is that projects are tools that originally had a specific focus

they start to become spaghetti code unmanageable and bloated

sometimes this can just be the result of an attempt to respond to community requests for features that are really

outside of their focus and it may be that a different tool altogether would

be better suited for those requests and there’s a lot to unpack there especially when it comes to

meeting the needs of the community and open source projects that’s always a challenge to push back

on things that can you know completely make sense for a specific use case but

maybe it doesn’t make sense for that particular project as a whole [Music]

so all of those things are listed those can lead to some pretty severe inefficiencies in your pipelines

this brings me back to the little shell scripts that i found scattered all over the place that were meant to deploy a particular product

in the past okay there’s one left this one is actually pretty serious when it comes to

pipelines and i’ll go more into detail on that especially as it pertains to containers

remember this slide bringing it up again and again this doesn’t even address what

happens after deployment this is all just up to deployment but even if you’ve managed to cobble

your system together with all of these different parts there’s likely points that include that glue code i was

speaking about earlier that handles various integrations between different tools and this can

lead to a measure of unreliability we discussed that as a developer there’s a phrase i’ve

heard often and it was called happy path coding or happy path

programming um meaning that you don’t consider all of the bad scenarios you just get it

working and get your happy path done um not always the best way to work

um and just like with happy path coding you can easily end up with a happy path

pipeline where everything needs to behave just so just like a rube goldberg machine

in order for a successful end result uh reliable pipelines more often than not come from a lot of trial and error

and much like i imagine that you know rube goldberg machine we looked at earlier um that would take a lot of time

too in fact the the team on that machine said it took three months to build that machine

glue code or not everywhere on this slide where you see one of those green connecting lines

those are places to start looking for weaknesses and putting some serious thought into what to do when there’s a break in one of those connections

is the result that the entire pipeline breaks down or are there some measures that you should be taking to handle

paths that are not happy paths

there’s clearly a ton involved in putting a full pipeline together

i’d be remiss if i didn’t mention those parts that are terribly unsafe if not addressed this diagram comes from the

salsa website um that stands for the supply chain levels

for software artifacts salsa it is a security framework and it’s basically a

checklist of standards and controls that you can use to improve integrity

secure your artifacts and packages and your infrastructure in your projects

it’s it’s a good um explanation of how you get uh from safe

enough to being you know in a better state um at any length in those change so any of

those weak points between your source code and delivery of your product or deployment of your

service there’s places you need to take extra care to protect your supply chain and that is indicated by each of these

red triangles those are opportunities for disruption and as a developer i focus on the ones

that are around the build process even around the continuous integration build

processes in a lot of the talks that i do i focus a lot on dependencies in particular that

is definitely in a developer’s realm so it makes sense

all of these pieces and parts of libraries and other artifacts that your code depends on this aspect becomes even

more important when using containers and one of the most crucial areas to harden is your build process during

continuous integration you must must ensure that the dependencies that you’re pulling in are

exactly what you expect them to be for example they need to have an expected version which means you need to

specify a version you also need to verify they’re using a trusted dependency and not a malicious

package that happens to have a similar name for example now

as java developers we’re a little spoiled here because i know there are

definitely been some problems one jumps to mind immediately for java developers but

um we at least are protected a little bit with you know maven um central like when we

use maven for uh you know to pull in our dependencies um at the very least

these uh have to abide by a namespace um you know the an owner of that namespace

maintains it um when you get into other artifacts like

uh python or javascript if you’re using npm it gets a

little more tricky because anyone can name a package anything and as long as

that name is available they can just throw it out there you know to npm and to

you know so that it’s accessible by pi pi and all of that

these are big problems and it has come to the forefront there’s a lot of vulnerabilities that have been

discovered out there that have to do with that type of behavior

so managing these dependencies responsibly a lot of this is just understanding your build and where these

dependencies come in since using a docker file is pretty common let’s start there talk a little

bit about a docker file this is an overused graphic i know but that concept is out there that

software is potentially made up of a ton of components that a developer doesn’t necessarily have first-hand knowledge of

that cannot be understated applications and services today are more complex than ever

developers generally don’t want to recreate the wheel if it isn’t necessary you know they want to continuously put

out new stuff this means reusing a lot of you know libraries and maybe open

source libraries or libraries even just from other internal teams i mean it makes a lot of sense to be able to do

that containers especially are an issue container images

when i started working with container images i was pretty naive in the beginning i just you

know wanted it to work i learned enough about docker to get my images built and then started you know

pushing them out there to see how they run but the more and more i looked into it the more i realized how much you can

shoot yourself in the foot if you aren’t careful and this is just a simple contrived docker

file and there’s several problems in here i’m not going to go over each one because there’s more details i’ll go over

later but there’s there’s a few so i mean obviously that first line from untrusted

parent image well what does that mean well a lot of these you know docker files um images can be built based on

another image on a parent image there’s that hierarchy there and

if you don’t know where your parent image is coming from how do you know what’s going to end up in your container when it gets launched in production i

mean these are things that you have control over there’s no tag here this is just the name of an image there’s no you

know version associated with it um even if there was images have this

crazy unreliable tagging mechanism that can be overwritten at any time your

processes can help with this of course but the reality is it’s very easy to override an image

tag with a completely different binary i mean they’re just pointers so

you might want to consider using a shaw or something

um lines two through four uh again not

specifying versions apparently whatever the base image had there’s still some os packages and stuff that were needed

and you know i often see lines like this in docker files but if you ever need to

build this image again you’re going to run into some you could potentially run into some

troubleshooting problems because any of these packages could be updated they could be updated in an incompatible way

and then you’re just gonna have to chase that down um line six

um that’s that could be a problem basically you’re copying everything uh you’re not being

real specific about what gets copied from you know your working directory or

from the directory that you’re building from what gets passed to the docker daemon

you can get a lot of stuff in your final product that you don’t want i’ll show you some more details on that

in a bit number seven line seven there’s an

external resource there that you don’t have control over we’ll talk about that more in a little

bit as well and then finally uh number nine

uh there’s no like user or group assigned so this just runs his route this little

start script here just runs his route these are just a few problems that i

come across frequently in docker files it’s a really good place to start but all in all this proves my point

software delivery pipelines are definitely a little bit like rube goldberg machines

so you’re probably wondering if there’s any real solutions to these problems let’s talk about that a little

bit here there are a ton of tools out there that will help you with the real case scenarios

as overwhelming as it is that cncf landscape for example at least lists

tools within categories that will be helpful to you if you’re just starting out you just have to pick one

for now quite a bit of research is required to vet and evaluate those solutions but we see how important that

is so make sure that you you know do that the hardest part is just getting all of

these pieces to fit and the industry right now is ripe for real holistic solutions that solve the basic problems

of continuously delivering and updating software as well as monitoring and troubleshooting in production

simple to say but there are a lot of one-off tools out there that have grown

out of very specific engineering problems and what we really need right now is

agreement in the industry on what best practices are so that we can wrangle some of these

tools in organizations are coming together to work on these issues

one important effort that was started is between jfrog and nginx

f5 and nginx they have like this open source project it’s called the mara project that they’ve been working on

it’s an acronym for modern application reference architecture and uh it was announced last year at their conference

sprint 2.0 um it’s basically a framework for deploying any

containerized application in a reliable and repeatable way and i’ve taken a look at it and there’s

definitely some things that we want to work on and improve and make more uh interchangeable more a little more

abstract to be used for different types of circumstances but this is just one example of

organizations like we’re working together on this one with that intention of producing open

source at least that will be available to everyone in order to push forward uh best practices

there’s also an effort with the cd foundation this is a sister organization to the cncf it is

also underneath the linux foundation they are working on

um determining and and defining best practices at this higher level at every

uh every business interest every stakeholder interest in order to define

why we’re doing what we’re doing in order to help guide and direct some of the efforts that are happening on the

engineering side of things [Music] this is the

landscape map for the cd foundation foundation the continuous delivery foundation it has a number of different

associated tools but it’s not quite as overwhelming as the cncf it has more of

a limited focus on continuous delivery explicitly

and they you know are really working hard on

discovering things or ways to make tools more interoperable

rather than strongly integrated that’s that you know avoiding those glue

code scenarios we talked about earlier and of course outlining best practices and there’s a number of organizations

that are involved in the cd foundation all of these meetings that they have especially with the special interest

groups they’re open to the public if you were to go to the cd foundation website they have a public calendar you

can definitely attend any of these meetings i’m a co-chair of the interoperability special interest group and we often have

these projects come and present to us explain to us why they exist

why what benefits do they offer that kind of stuff and that way you know we learn more as a

group and we publicize projects that we find that are that we

find very valuable a couple of examples of projects that have come out of these efforts um this

one cd events actually originated in the interoperability group um

this one uh basically you know it is is a

you know defining a way for all of the products to

submit and to read events from other tools

in a common format if any of you worked with cloud events uh that’s a very similar thing and

that’s really what this was based on with cloud events to start is just defining what that means

so that we have this common place and can avoid the glue code scenarios another project that came in

recently is persia this one is actually backed by jfrog but it is an open source project and it has

to do with decentralizing package registries

making sure that where possible if there’s you know access to source

that it can be verified between a few different nodes that taking that

source and building it ends up with the same artifact this gives you a little you know more

security knowing that you’re you’re getting what you asked for

that the artifact that you’ve requested hasn’t been manipulated in between you know the request and when you receive it

so we are developers we do need to talk more about dependencies

not just security issues are involved here there’s a lot of efficiency issues as well

and again i’m going to chat you know bring in

this slide all of these places where we end up dealing with

dependencies uh this is not reserved for just operations to worry about it is part

of our build process an integral part of our build process is to be able to pull in the dependencies that we need

and then knowing how uh something is going to be deployed later you know that’s going to affect

your the decisions of your developers on you know way back in the design stage and how they create these artifacts to

begin with [Music] this is a simplified version of the

pipeline you know beginning from development going through ci going through testing release process and then

deployment where is it in this whole

uh process should we be concerned about dependencies well pretty much everywhere

i mean this is where we live developers are constantly going to be bringing these in

and on to their local machines so they need to know where these dependencies are

coming from in the build server we also need to know where these dependencies are coming from

are they coming out from the wild west from some you know public repos or do we have some control over where

they’re coming from um same for testing this is when we retrieve our artifacts that we’ve

created in the previous steps so we care where the artifacts are coming from is that location has that

been protected and is that a trusted location [Music] and then for release you know we need to

be able to promote particular artifacts that have passed certain quality gates as they’re moving

through the stages of the pipeline and then ultimately deployment uh super important that what we’re deploying on

our production machines is what we expect one thing i’ve seen happen in the past

is maybe not enough concern given to permissions

when it comes to automation for a deployment and what i’ve seen happen is um accidents happen

not necessarily on purpose um where you know test artifacts were deployed on production by

accident uh that could be protected you know that could be prevented by protecting your

production you know making sure that your production machines only have access to production resources so that

whole idea of promoting as quality gates are passed and guarding those locations carefully

that can really help you out there okay i want to talk more about uh

images creating container images this is something that is becoming more and more important um

even as a java developer even if you’re not containerizing your java applications you’re likely going to be

soon if not java some other language so i think this is really important to

get a jump on if you have not done anything with containers in the past obviously there’s a ton of documentation

docker has a really good documentation i’d advise starting there obviously you don’t have to use docker

there are other tools out there but this is a good place to start to start recognizing where there could

be problems where you can make improvements there’s 10 of these i’m going to whip through

these really fast the dot docker ignore file i did not know that it existed when i first

started using containers what annoyed me the most about one particular project was how long it took

to build an image every time i made just a tiny change it seemed like it took forever to build and

that was because i did not understand how the docker context worked um this is another contrived example but

that line number three is the problematic line that we saw earlier in the docker

file that is copying everything into the working directory

and let’s be clear on how what’s actually happening there when you run a docker build command to build an image

using that docker file everything in that current directory is sent to the docker context

in order to make it available to run all of the commands that are in the docker file

everything is then copied into the image the docker ignore file is very much like

the get ignore file you can use it to avoid adding any test or user specific files or even

secrets that you might have stored locally and also consider that dot git directory that’s probably not something

that you want to be putting in a production image [Music]

so at first i thought i could just improve my build time by oh i’m not going to copy everything i’m just going to explicitly say what i want to copy

but in reality that doesn’t help enough because this is what’s happening in the background

again everything is being sent over to the doctor daemon that takes some time and then the doctor daemon

determines oh you only want these two files copied okay i’ll just take those over so the docker ignore file

helps you out with that very first step there it only sends over what’s needed can save you some time

[Music] so this is an example of what one looks like you can avoid adding readme files

that you don’t need passwords of course the dot get directory log files temp

files test files you know all these things that will just make a production image a little more bulky

next one is using untrusted base images we talked a little bit already about this

i just didn’t realize back when i started how base images worked

until our build broke one day with an error message saying that the base image didn’t exist well it turned out the base

image in our docker file was actually getting pulled from a private repo of the original third-party contractor

which was out of the picture for like a year by then so i mean there was no expectation that they would

always keep that up so again make sure you know where your dependencies like this where they’re

coming from make sure you bring them in internally somewhere so that you have control over them

uh this is another problematic base image obviously you’re not going to pick

something named evil image but my point here is just to pay attention where they’re coming from these are

dependencies just like anything else they should be pulled into artifactory or some other artifact management tool

that you use and managed along with all of the other binaries that you rely on for your build

using docker official images is a good idea uh docker has a dedicated team

that’s responsible for reviewing those uh they are open you can see exactly what’s in those and see how

they’re being built they are you know make sure that they are updated regularly

um that they work in collaboration with upstream software maintainers and security experts in the broader docker

community so those are good to start with but always do your due diligence and make

sure you know what is in your base images also are they coming from a private or a

public registry by default if you don’t say a registry if you don’t indicate a registry they’re going to come from

docker hub so you might consider you know caching them in a private registry

that you have control over what to use i remember

just when i was first starting and just trying to make a little container with some java application running in it

um i didn’t really know where to start um there’s all these names out there

you know if you’re a linux person you’re to figure this out and it isn’t you know magic it’s not all

unclear you’re going to see names like that in other images besides like java images

or open jdk images as well but there’s no mystery in here when you understand what they indicate

you can narrow your choices down by the java version that you want if you need to install additional

packages you might want to pay attention to the linux version that you’re using and then i put some notes in here about

alpine because if you need to look at the documentation when you’re pulling these from docker hub so if you go to

open jdk on docker hub they explained at one point that alpine

images were um you know they’re based off the alpine linux

project um but they weren’t really supported and according to their documentation they were only released

for early access open jdk version releases and they are not updated after

uh ga so they’re older images that you might not want to use now all of that said uh anyone who has gone to openjdk

and looks at it now uh they’re deprecated now so if you’re using these images you might want to go

consider you know replacing them with something else if you’re already paying for support for

a specific distribution like from azul using zulu images or

ibm or something or you know you can use the coretto ones from amazon

it makes sense to use their official images so pay attention and make sure that you’re updating appropriately

not defining versions um not much to say here here other than what i just said

make sure that you define your versions we talked about this earlier um you really should have a bill of

materials for your build know the exact version of everything that is installed

um and then you can eliminate multiple moving parts um

you know when you’re building your applications if you control the images one thing i would suggest a lot of i’ve

heard the reasoning of not defining an image because you always get the latest and greatest and you might as well just

run with that um the problem with doing that is it just like i said it’s too many moving parts

if you want to do things that way make sure that i would go ahead and explicitly use them use the versions

that you’re expecting and dedicate a build where you do just that because um

you potentially have a lot of troubleshooting to work out so make sure that you’re not risking you know getting

a feature out or getting a bug fix out just because you’re you know trying to figure out which

dependency got updated and when which one is incompatible [Music]

uh we talked a little bit about tags how unreliable uh image tags are using latest is just

asking for it like i said these are pointers latest is not always latest it doesn’t

always mean what you think it means uh it can always be overwritten it’s kind of like using a snapshot like when

you’re building your job application and you’re using snapshot images that that

it’s a good enough analogy i guess

uh never updating of course we need to update our stuff like i said make sure

that you dedicate some time to update things uh we know of a specific case where a particular company did not

update uh even knowing that there was a vulnerability involved and how embarrassing it was for them when they

actually suffered the consequences of that so block out time during every development cycle to spend on this kind

of maintenance [Music] so here’s an example um you know make

sure that you update your base images as well keep those current just like we talked about with openjdk

including build tools when i first started out with um java and and building containers and stuff i thought

well how convenient is it just to include one that had maven already right not have to worry about this and i see

that quite often it works but you end up seeing you end up with an image that’s quite a bit larger than it needs to be

and what do you need maven for on a production image you really don’t need all that extra stuff so it’s best also

to minimize your attack surface from a security perspective just the only thing that goes in your image

are is what is absolutely necessary to run your application um

so one thing to consider is utilizing multi-stage builds there’s an example of

that you see that there’s two from lines in here that’s a feature that docker added

um this way you can choose a very very slim minimal base image and you can just copy

you know the results of the build into that image so that’s one option for you

using external resources remember in that contrived example we had before there was a curl

statement i’ve seen this a lot i’ve seen curl i’ve seen wgets you know these kinds of things

it’s one thing if this is a resource that you have control over again that you update when you want

but i have seen cases and have experienced the case where the results of this statement is uh nothing because

whoever was maintaining it decided to move it decided to rename it decided to update it to something incompatible so

rather to be on someone else’s schedule like this it’s a better to pull in

that resource pull it in so that it’s managed internally in your artifact management

rather than reach out for it during a build

trust but verify of course um hard coding secrets that seems like an

honest you know obvious one uh but i see it a lot

uh unintentionally um one thing i’ve seen in the past is also this attempt to

remove secrets so as an image is built you know layers are are added

and this is an attempt to just remove the secrets that weren’t wanted in a base image

well um they’re still there though i mean just because you can’t you don’t you

can’t see them if you were to exact into the image they still exist this isn’t a good idea

in this case it would be better to rebuild that base image and make sure that those secrets weren’t added to

begin with um also this is another example of making sure that you

make configuration um stuff like this like this this port for example that you’re exposing maybe

maybe you need a different port for a different environment so um

you might want to use a variable instead in this case that way you don’t have to rebuild this

image for a particular environment [Music]

just more examples making sure you don’t have secrets and making sure that you’re

using variables [Music] i’m doing too much this is just using

these docker files as like a shell script not a good idea

this one’s somewhat contrived again for simplicity but it was modeled after an

actual docker file i’ve worked with in the past there’s some crazy side effects here because think about what happens

when a test fails um you really need item potent operations

in a docker file testing like this if nothing changes in a previous layer

it doesn’t get regenerated and rebuilt whenever you’re utilizing docker cache so

in this particular instance if you’re using docker cache um that cleanup database stuff is not going to be

repeated but you’ve already failed a test so that database is not in a good state for this

to ever work so um a better way to do this is keep this testing stuff out don’t be doing that in

a docker file build the image and then have an environment where you can

run your integration tests against that image as it is [Music]

yes so keep it simple all right um unnecessarily busting cash

uh this just slows everything down and when you start pulling apart fat jars for

example you discover that it is very amenable to you and your time to make

sure that stuff that changes frequently is lower in the doctor file then things

that don’t change as often so perhaps you might want to consider putting you know third party libraries in a previous

step and then copying over you know dynamic code or stuff that you are changing frequently

that is an active development you know those class files and stuff can come in later

good to keep your dev builds efficient this way there’s many different ways to build

java images for example whatever method you choose there’s a few out there

you can use that like a maven plugin you can use build packs you could use

jib understand how it’s actually building that image because things like this will

come into play [Music] so again you know keep stuff that

changes frequently as close to the bottom of your docker file as possible

all right lastly is relying on root

i feel bad for repeatedly saying this one over and over and over again

you really shouldn’t be doing this you shouldn’t have processes running i mean if if a process is running as rude it

could potentially have access to things it should not on your host um make sure that you’re specifying a

group and a user uh if you’re running a process um pay attention to official images that

you pull from docker hub because sometimes there’s this you know understanding that you’re going to be using a user or a group um

you it’s uh it’s just a an issue where

um the moment you know anyone if anyone can get into that container they immediately

have root access i i mean why make it that easy so if if it must have root

access make sure you have some good reasons for that um but um you know

use that principle at least privilege and make the right decisions there

so this is an example of that um this start script uh obviously is is running as root

[Music] and here’s an example of just adding a group and a user i think if you if you

were to look like a postgres docker file for example they’re pretty good about this

[Music] and there was a report that came out

this was a cystic report and i thought it was interesting that they just did an observation and 76 of the containers

that they observed using cystig were running as root had processes running as root so

i i think that everyone’s heard this already but even if that’s true it just it doesn’t appear that we’re taking that

risk seriously enough okay i prattled on enough about

containers uh let’s just do a quick summary about what a java dev should know about devops here

you should understand your software development life cycle goals and objectives

you should know where your dependencies are coming from that should not be a mystery to you for example when you’re

building on your local machine um are you pulling your dependencies

from maven central every time or are you pulling them from a private

uh repository or you know where are they coming from important to know um know how your artifacts and your

containers are built just like we went through understanding how a container image is built with a

docker file it’s important to understand that has huge effects on your efficiency

how well you’re going to be able to get through your pipeline from beginning to end

pay attention to the risk of vulnerabilities there are a lot of tools out there right now and i could show you

some if you’re interested jfrock has some really good security tools there’s a

jfrog x-ray does scanning and there is a plug-in for my ide that i can use and i

can just see right away before i even check my code in i can add a library to

the pom file for example or add add a package and it’ll tell me if there’s any known

vulnerabilities with that so that’s pretty cool docker desktop has some good scanning resources as well

um and then also you know the ability to do that within your pipeline too not only should you be doing it at the beginning

but you should also be doing it you know as the um as your software goes through the

different stages and then finally how to plug in and be active in your community remember what i

told you about the cd foundation it’s not private stuff going on there there’s a lot of a lot of happening

things and you know get yourself involved when you choose a new tool when you’re

looking and vetting a tool find out if there’s a community resource that you can get involved in you can ask questions

you can see what other problems other people are having that will help you with your research and help you with

your decision making lastly before i

sign off um is uh just make sure that you go and get

your free copy of this book there’s five of them available so go ahead and sign up and you’ll be contacted by email

within three business days so we can send you your copy

all right that is all thank you for listening feel free to dm me with any questions or comments you

have or if you have any questions today i’ve got some time here we can chat for sure

all right um and i’m not looking at the comment log but first of all thank you so much i really enjoyed this and um got

some ideas on sort of what i need to do in my environment with our devops pipeline we use um a yaml file and then

the pipelines that azure devops have but we definitely want to try to apply a lot of the stuff that you

said uh for our environment as well um chris what’d you think or do you have any

comments i thought it was an excellent um survey of the millions of different

things that you could have brought up and i thought you did a really nice job of condensing it down to some important

high points that people need to investigate further cool

thank you very much i’m gonna try to put up that comment uh log from the youtube

let me hang on a second i thought the thing about splunk was hysterical

it wasn’t so funny for me at the time yeah you know people building extraordinarily elaborate dashboards

harvesting logs you know and meanwhile we’re in there constantly changing the logging and

tweaking it or changing levels or changing the messages not even thinking that it’s going to

blow up this finicky um brittle thing somewhere exactly in a

data warehouse that we know nothing about yeah our team was not even aware that there were any metrics

you know that there were any measurements being taken so we didn’t even know to ask yeah no that’s that’s just beyond classic

all right and so the only thing i see is um well somebody commenting on the puzzle

as a great option my wife does a lot of puzzles as well in fact she has a mat that you can roll it up if you want to

pause and then roll it to continue on but um and then uh karina uh carrie gunther

um had some questions about uh produces and

privacy because the hip rules we have the same issue at cdot we have some data

that’s law enforcement data that is sigis is what they call it and we can’t share that and so we kind of have to set

up some test databases and load them up to do uh to do testing so that we can you know

not compromise the data and you know just abide by our security concerns yeah that is difficult i’ve run into

that too especially if you’re working with any you know any private data um any identi what is

it called the identifiable data that you’re not on yeah yeah which includes like

the email address is um obviously credit card information and things like that

yeah there’s not a good way to deal with that other than try to duplicate as much as possible in the

correct format uh in order to get the amount of data that you need but yeah that’s something

that needs to be developed over time too as you make changes you know add more and more types of data

to your test database one thing i will say about that um

i mean that is a hard problem to solve i don’t have an immediate solution for that but make sure you’re protecting your resources um between your

environment i don’t know if any of you have had that experience where you’re like oh no it’s almost as bad as

you know deleting a git repository or something on accident but it’s like you run your tests against prod

because you’re connected to prod or or um you accidentally delete the entire

test database on accident because you didn’t realize you know what you were connected to so these are things that

can be prevented if you really pay attention to uh you know permissions that you have to

various services yeah definitely i i mean for example for

work i do at cdot we have uh the production database shows up in a sort of like a red window dev is green and

then qa is yellow so that you know okay i know where this is going when i go to

do a query or something like that um a situation where

uh one of our devs it felt so bad for him accidentally launched like a

um message broker but accidentally connected to

uh production oops so he’s running tests and and in production it’s like what’s

happening to the messages the messages are just being collected and disappearing and it was a test

environment that was pulling them pulling them straight out of production so yeah

we’re just a few funny stories got a suggested meetup which was a test message

yeah so the other thing is um one of our attendees commented that they

really like the wonderful images uh for the context and point you were trying to explain and and they appreciated that

oh cool and there it is thank you matt all right matt i know you can’t you’re having some

technical challenges but if you want to put in our chat if there’s anything i missed but i think i got all the stuff

everybody liked the rube goldberg and i don’t know if you ever had this as a kid but i did and then we had it for our

kids there’s a game called mouse trap where you build a rube goldberg machine and the idea

to win the game you have to catch the mouse using this rube goldberg setup

i love it yeah so um yeah that’s all i got

thank you so much i really enjoyed it i guess i’m gonna wrap and let you everybody go eat their dinner or do whatever they’re doing next

so um thanks so much for coming out um i uh i tweeted some of the stuff that you mentioned if you could uh email

zeddy the slide deck and we’ll share it on our um on our meetup site but i also

shared on twitter um the uh some of the links for those tools that you had shared with us oh awesome

cool so and i tagged the denver jug so you guys should be able to see it all right um well thank you and have a

great night and uh for everybody at the attend we’ll this again in october our plan is for

november to be in person but please please please let us know a venue that we can meet at

that can accommodate vencat because usually there’s a big crowd some of the venues we’re looking at want

to charge like a few hundred bucks for security which if we can get a sponsor for that that’s

great but we’d rather not spend your money on that if you will i’d rather spend it on food for you and

drinks and other things like that so all right thank you and have a great night

good night