What does a Java Developer Need to Know about DevOps? @Denver JUG

JFrog's Melissa McKay talks with Denver Java Users Group!

September 14, 2022

< 1 min read

A day in the life of a Java dev includes coding up new features and fixing bugs. But what happens after you’ve committed your code, and why would/should a developer care?

The process of developing and delivering an application to production can be an overwhelming task with a number of steps. Melissa will explain the DevOps methodology, discuss DevOps implementations, and walk through what all is involved. After learning about dependency management, CI/CD, supply chain security, production packaging and updates, it will be clear why a Java developer would (and SHOULD!) want to be aware of what is going on in their software delivery pipeline. Learn how a strong knowledge of DevOps will inform and guide your decisions as a developer!


Melissa McKay

    Melissa McKay

    Melissa is a long-time developer/software engineer turned international speaker and is currently a Developer Advocate on the JFrog Developer relations team, sharing in the mission to improve the developer experience with DevOps methodologies. Her background and experience as a software engineer span a slew of languages, technologies, and tools used in the development and operation of enterprise products and services. She is a mom, Java Champion, Docker Captain, co-author of the upcoming book DevOps Tools for Java Developers, a huge fan of UNconferences, and is always on the lookout for ways to grow and learn. She has spoken at Kubecon, DockerCon, CodeOne, JFokus, Java Dev Day Mexico, the Great International Developer Summit, and is part of the JCrete and JAlba UNconference teams. Given her passion for teaching, sharing, and inspiring fellow practitioners, you are likely to cross paths with her in the conference circuit — both online and off!

    Video Transcript

    welcome everybody uh welcome to the boulder and denver java users group um boulder’s going to

    do something in person and chris weiner will talk about that in a minute i’m going to continue with our shortened

    format and if you don’t like it let us know but so far people have appreciated us just getting down to business

    we’d like to meet again in person we’re trying to do that for the november meeting and we’ll do a broadcast as well

    so if you can’t make it in person or don’t feel safe you can watch it from home if you have a large venue that

    would accommodate us for having vent cat as a speaker without a lot of extra costs let us know

    reach out to myself or matt or go through the meetup group so we can try to figure that out

    email us any topics that you’d like to see reach out via the meetup group

    big thank you to intellij agile learner we do giveaways

    zeddy will handle those drawings thanks to matt rabel who is our speaker wrangler and to octa for hosting the

    broadcast on through stream yard and then on youtube um thanks to jfrog

    there should be a link in the chat they’re doing a giveaway as well just click on the link to do that

    i’d like to go ahead and bring up chris weiner from the boulder java users group

    with his announcements and then i’ll introduce the speaker hi everybody and thank you very much

    greg um this should be a really good uh talk tonight melissa is an o’reilly author

    and international speaker uh we’re fortunate enough that next monday we’ll be meeting at

    rule four in boulder rule four is outside magazine’s place of the year to

    work it’ll be a good meet up catered hot food

    beverages beer you name it and it’ll be a combined meet up it’ll be

    boulder devops colorado kubernetes and cloud native and also the boulder java usage group

    so it’s just off the foothills parkway easy to get to if you have time to come up

    and that’s it from us thanks greg thank you chris and obviously go to the

    boulder meetup group and you can get details in the location uh and the address so

    tonight’s speaker is melissa mckay and the topic is what does a java developer need to know about devops i’m excited

    about this and um and and really it’s it’s any developer right but i know we’re going to talk about java

    specifically because at the state of colorado we’re using azure devops as a way to

    um you know build and deploy using devops tools to various environments and

    uh using some different pipelines and so just really understanding that environment and how to pass you know

    variables down uh and and automate it completely as of great interest to me and so i really hope you guys all enjoy

    that so uh go ahead and let’s uh bring up melissa and uh hand it off to you thank you so

    much for doing this tonight we really appreciate it hey hi everybody it is a pleasure to be here

    really excited about this um i am really looking forward to i’m

    talking about what does a java developer need to know about devops um this is a

    you know kind of a culmination of some of my personal experiences uh we’ll talk you know a lot of high level things

    that i’ll be talking about like what really matters really uh

    i do a lot of speaking on various topics a lot of devops topics in general and uh

    it’ll become pretty clear why that is in a moment but um it just it really matters to me um

    given you know i i had to think way back when i first started as a java developer

    this was probably over 20 years ago now uh where i was an intern and you know i had learned the

    how to code i’d learned you know java as a language but everything that goes on

    around it i did not know that was all stuff that i learned on the job and over time

    so this is something that’s near and dear to my heart because i think anyone starting out and certainly anyone that

    is more advanced in their career they’re going to need to know some of this stuff and some of these concepts so

    this is kind of a combination of many of the different talks that i’ve given there was three that i chose

    specifically to combine together to share with you tonight a lot to share so i’ll go

    through some of it pretty quickly but i think you’ll get something out of this

    tonight whether you’re a beginner or whether you’re more advanced we’re going to talk about what really

    matters when we’re talking about devops processes and tooling we’ll talk about building and

    understanding your dev delivery pipelines why that’s important managing dependencies i think is huge

    and as a developer we really need to focus on that part a lot lastly i’m going to talk about building

    your artifacts and we’ll get into some details about building containers specifically

    i will focus on containers given some of my background and these are concepts i think are really important for us to

    know but these are some techniques that can be applied across all languages as well

    so some background on me i am a developer have been for many many years

    i did start speaking a few years ago and i decided to join the developer

    advocacy team developer relations team at jfrog i’m going on my third year so

    this is really interesting timing i really enjoy teaching i enjoy sharing what i know i enjoy the public speaking

    um however right when i started is when the pandemic happened so we all needed to

    stay home and i got very versed in how to do video editing and how to do online

    conferences and things like that um there’s some pros and cons there one thing that i enjoy the most is being

    able to engage with people all across the world at any time and i still do that today

    too i leave room in my schedule for that um like i said i’m a developer advocate

    i’m also a java champion that was something that was an honor i received during the last

    three years and also a docker captain so i try to keep up with the latest and greatest what’s going on with docker

    what’s going on with containers and i have a few talks coming up at various conferences in the future on

    those subjects so why are we here today what is devops

    let’s start there and wikipedia has the best answer um you know good enough for

    me anyway devops is a set of practices that combines its software development

    dev and it operations ops it aims to shorten the system’s development

    lifecycle and provide continuous delivery with high software quality okay

    this is just a long way of saying help developers do better help us all as a

    team and also other folks that are involved in a project

    help us all be more successful um for required reading i think uh these

    are books that were introduced to me many years ago and i actually should be uh

    bringing these out again i think it’s time to reread these and uh get these concepts um you know remind myself of a

    lot of things but these are the phoenix project the unicorn project and the

    devops handbook really really exceptional books the phoenix project is

    written from an perspective and it’s not just a an

    academic book it’s written it’s like a novel so it’s really engaging it’s really fun

    and it follows this team’s problems that they go through with a

    particular software delivery and all of the solutions that they come up with and and you know how they start to

    streamline their processes a little bit more and the sequel the unicorn project is actually from a developer’s perspective

    and and it’s also written as a novel and there’s a lot of the same characters that you would recognize from the first

    book the last one is more of an instructional manual on how to put devops principles

    into practice and it talks about principles of flow and principles of feedback and principles of continual

    learning and experimentation so like i said i always recommend these books i think

    they should be required reading for anyone that is on a devops team

    i would be remiss if i didn’t uh advertise my own book here so jfrog is

    um has a raffle available and there should be a link coming into the comments um if

    there isn’t already and uh this is a

    form that you can fill out put in your email address i’ve got five of these books i’ll give them out to

    the first you know five people that get pulled in a raffle and you’ll get contacted by email in order to get this

    sent to you this is devops tools for java developers this was a book that

    some colleagues and i got together and just decided to focus on what was most important in the devops processes for

    developers from a developer’s perspective and i think it’s a really good introduction to the entire process

    okay so this is a slide deck that i often see in corporate decks at jfrog

    it is a pretty detailed step-by-step of the software development delivery

    life cycle and it is really really busy but i like it because it does a really

    good job of highlighting how overwhelming devops can be when presented with all of the stuff in

    you know the ecosystem this space is huge and it’s really easy to get lost in the mix

    this whole process is an attempt to answer the questions how do i build everything how do i test everything and

    how do i deploy everything where do dependencies come from and how do i keep everything secure how do i

    include automation to help increase efficiency and reduce human error and then as expected of course there’s quite

    a few jfrog products in here that represent uh handling artifact management and storage and distribution

    and security scanning on packages but this setup has really strong integrations with other components and

    frameworks out there that you’re likely already familiar with and use today these are all the various build and

    dependency managers like maven and npm gradle continuous deliver integration servers

    like circle ci and jenkins and then of course provisioning tools like chef and

    puppet now i don’t really expect you to see many of the logos on here it’s pretty

    tiny but it just represents just how much there is in this space what is not represented on this slide is

    everything that happens after delivery so a lot of your

    you know observations that need to be taking place being able to monitor

    everything that’s in production and in your systems and be able to provide feedback of how your service or

    application is being used all of that needs to feed back into the beginning of

    the whole life cycle for developers to continually maintain and add features

    etc now one another thing that this slide really um brings attention to is how

    much how much is in our wheelhouse like how much a developer really needs to know

    and this is something that comes up all of the time with devops in particular i think there are

    good and um wrong ways to have a devops team or to organize a

    devops team and for me the whole idea the whole advantage is to be able to

    utilize other people’s expertise i mean this is why we have teams that work

    together closely it is there is not and should not be an expectation that one

    person uh specifically a developer needs to know all of this in great detail

    i think it is very important to have an awareness across the board of you know what’s available the purpose of certain

    tools and things like that but to take a developer off their task of their craft

    and to put them you know on other stuff i mean it really can be a full-time job

    so i don’t think that that’s the best use of resources

    as a developer keep that expertise but on your devops team you should also have

    folks that have operations expertise as well so when you have concerns with

    deployment there are specific people that deal with deployment when you have you know a bug that needs

    to be fixed code needs to be checked in that’s in a developer’s purview

    [Music] so that previous slide can be simplified

    a little bit i was on a panel discussion

    not too long ago and a question came up about the state of devops today and what might be coming down the line in the

    future if you compare like existing devops tools that are out there and and

    components to car manufacturing we’re still very early in development on

    these things and what i mean when i say that is how many of us would be driving cars today if we had to build them

    ourselves and sometimes that’s how it feels when you’re building a full pipeline from beginning to end it feels

    like you’re you’re having to cobble together a lot of stuff on your own we’re still putting most of these pieces

    together ourselves there’s a ton of levers to pull um but for a car for example we have a

    general idea of how to make certain components how to categorize them so they’re easily interchangeable when you

    go get your car repaired for example in software we’re still in the process of working out where the boundaries are

    on some of these tools that we use how to make these components that make sense

    but we have a pretty good idea of what we need to know now to get everything up and running and generally tools will

    fall in to these core components or categories we can simplify this a bit break down

    all these things we just saw in the previous busy slide into these core components and that includes your source

    code repository um that’s your single source of truth for all of your dubs uh continuous

    integration being able to build and test on on you know every

    system that you have out there your package managers this will you know provide you a full bill of material so

    that you understand clearly what’s in your package what’s in your artifacts that you’re producing

    and security scanning this has become more and more important in recent years

    they said deep scanning of dependencies and containers too if you happen to be using those and then uh continuous

    deployment um being able to deploy you know out to the edge at any time

    being able to update quickly and then also being able to deploy to any environment as needed

    before you jump in um before you start implementing a bunch of new processes

    or a bunch of new tools picking up all these things and trying to use them all at once get together as a team and determine

    what your values are what are your goals what do you really care about and every

    project is going to be different there is usually a driving factor that will help you to figure out what to

    prioritize and this is where communication between teams comes into play as well

    if your team is prioritizing something completely different than another another stakeholder you are undoubtedly

    going to experience some friction and frustration and the best

    self-help guidance out there applies to development teams as well you need to define what success looks like it can’t

    always just be this vague let’s do better we need to have a you know a definition

    of what that looks like so that you can set yourself up for success and know when you are successful

    your team’s values and priorities are going to be different than everyone else’s you’re going to have a different

    set of skill sets uh there’s really no one size that fits all solution um tonight i’m not going to tell you use

    one tool over another uh that’s just not how it works prescribing a particular

    tool set to your team would be akin to telling everyone to you know wear a uniform every day size 10. you know it

    just doesn’t really make sense and um i’ll you’ll never see me get into an argument about languages either i don’t

    especially when it comes to building like cloud native applications and things like that sometimes it comes

    down to what what is the experience that that your team has to work with sometimes you need to go with that

    change can also be expensive so choose those changes that will advance your team to that measure of

    success that you have defined this picture that i have in this slide

    is actually my car and i put this picture in here because i wanted it to remind us to know what is

    good enough um this car is super old uh i’ve had it for years and years and years i’m just waiting for the doors to

    fall off and uh but you know it’s not something i’m going to spend a lot of money on and repair so you know a few

    years ago i got into a little accident and you can tell that there’s a little wound on the side of the car well

    that was a 12 purchase on amazon to buy a magnetic band-aid and put it on there and you

    know just go with that i’m not going to spend my resources to fix that on this

    particular car sadly there’s some software that’s like that too so you really need to know what

    um not just the goals of your team but all of the stakeholders involved there’s

    always money involved there’s time and resources involved so all of those things will come into play when you make

    your decisions on how you build your pipelines and where you spend your time

    there are two really important aspects of devops and that is inter-team

    communication and intra-team communication uh in a devops world

    observability of your system is going to become extremely important and that’s often it could be a completely different

    team than you and i put this slide in here just to remind me to tell you another interesting story and this is a real

    life story about a disconnect that my team at the time had regarding a specific business value that we were

    unaware of and most devs have seen a line like this before in their code base not going to

    argue whether this should be there it was this is what we had and um i was

    actually working on an efficiency issue in the code base um and i was very proud

    of myself i implemented a really clever fix um it involved changing this log

    statement though because this lock statement was no longer applicable for the very simple reason that the

    function itself was no longer named transform media for example and it just didn’t make sense anymore

    unbeknownst to me this little minor change caused a huge fuss and at the

    time i was working in an open space but there were a lot of product and business

    and marketing people that were on another floor in the building our floor had the better coffee so those folks

    would always be down in our kitchen uh drinking the coffee and i just over

    just happened to overhear a conversation between them saying well you know this

    particular tool is not being used very much anymore i think we should discontinue it or you

    know find something else that will work better um you know that kind of thing well my ears perked up because when they

    named the particular product i knew exactly what they were talking about and i was really interested in why

    why would they make this decision and apparently uh they had some dashboards out there

    that i think they were splunk dashboards and they were utilizing log statements such as this in

    order to populate that information well obviously since this log statement had changed it was no longer populating

    what it should be so right away we had a disconnect they had a particular priority and a

    particular value system that they were moving forward with and we did as well we wanted quality of code we wanted it

    to be more efficient but we weren’t talking to each other about the goals of the product and the business value of

    the product so we were you know not in lockstep we we missed we

    missed that so thankfully i was able to overhear that and we were able to rectify that situation but that’s just

    one interesting thing that can happen if you don’t have that intra team communication

    [Music] so if you only get one thing out of this talk today it’s this avoid silos and i’m

    not talking about just between members of a team where you know maybe a dev goes off in a corner when puts

    headphones on that’s not what i’m talking about i’m talking about between teams

    make sure that you have an understanding of what the goals of the project are and

    what the values are if everyone is pulling in different directions you’re going to end up stepping on each other

    much like what happened in my log story and when your team decides to charge a

    hill make sure that everyone involved in that effort is charging the same hill that you’re all moving in the same

    direction this means that communication about business goals needs to flow between all

    stakeholders from the business and product level to the engineering and development teams

    you’ll find there is a lot of focus on the end result when we talk about you know putting pipelines

    together and stuff your service or application that’s running in production right now what is

    the end result but something you’ll realize with experience is that your end result is going to depend a lot on

    everything that happens before think about your software development life cycle itself as a product

    consider the processes that you take to get your product to production find the waste what is slowing everyone

    down and part of this is determining your priorities for your situation um is

    shortening the time between releases the top priority or is there need to focus on quality or on security

    and something that the devops handbook points out is that in the physical manufacturing industry it’s really easy

    to see bottlenecks you know physically see them in technology it takes more effort we need

    to implement those methods of making our work more visible and the best example of this is you know use of

    kanban or sprint boards to make our activities more visible solutions like that will help the team

    stay organized and aware of what team members are up to but sometimes we forget that these types of communication

    are actually meant to help with the communication between teams as well keeping this information limited to

    internal team members defeats half the purpose of boards like this now that’s the high level

    another level is of course observability tools alerting monitoring

    all of those things need to happen as well in order to monitor your success monitor your progress

    but consider putting some of those tools to work not just on your end product

    in production but maybe along the way as well collect some metrics like how long does it take for my application to build

    during continuous delivery is there anything that can be improved there things like that

    [Music] all right let’s talk pipelines

    pipelines is not a magical word in devops everyone has a pipeline whether it’s

    manual whether it’s automated whether it’s high-tech or low-tech it doesn’t matter this word is a bit overused many of the

    products and services out there even use the word pipeline in that in the name

    and i would um bet that most of those products that focus on pipelines they usually imply some sort of automation

    that’s what’s involved in those particular products it’s just um that pipeline itself is really the steps all

    of the steps that are required to get your code from development to production and every member of your team

    should be familiar with your pipeline if not stop what you’re doing get everyone

    together and document the steps there might be some surprises in there maybe

    some manual things that are happening that only one person knows or um some

    things that are redundant or you know some things that are unsafe

    it’s definitely something that you you need to do is whiteboard this activity make sure that it’s documented too often

    do i see documentation that explains how something is implemented and forgets to explain why

    and the problem with that is you end up with you know members of your team who want

    to make radical radical changes but if there’s no uh reason why

    you could end up in this circle where a change is made everyone’s you know happy for a while and then you end up changing it back i

    mean the reality is is that you know people move sometimes they move off

    teams you get new people on and you don’t want to just keep repeating the same thing over and over so if you’ve

    had a bad experience with a particular method write about it make sure it’s included in the documentation if you

    changed a component out let’s say you chose to use a different build server make sure you explain why that way when

    the next person comes along and says hey i’ve always used this previous build server in the past and was successful

    let’s go ahead and change our project to use that then you’ll have this documentation to explain why maybe that

    isn’t a good idea or you know at least don’t repeat the same mistakes

    [Music] for each of those steps in your pipeline

    look for improvements you can make this is a continuous activity

    where can you get the biggest bang for your buck and we talked about making priorities and as you step through your pipeline

    you might adjust your priorities but having priorities is not enough not enough just to say we want to do

    better make sure to document your goals and determine what metrics that you can use

    to figure out if you’ve met them and you’re successful start out small with goals that improve

    some facet of your pipeline by one percent an example of a bad goal would be deploy faster

    everyone wants to deploy faster you you need to start with a baseline say it takes us this long to deploy is there

    anything we can do to improve that time is it is it worth it is there some other place in the pipeline that needs some

    attention first this is an excellent time to consider different tools and frameworks that

    promise to help with your flow when you’re considering your priorities and trying to make improvements but this the

    most important thing that i want to stress is always have a reason for that change don’t just go switching out

    technologies without having some metrics behind it that proves worthiness

    it is fun to play with new stuff and there should absolutely be time devoted to that

    but implementing new tools across a team needs to have serious thought and consideration behind it

    and you need to have buy-in from your teammates on this as well so one thing i’ve learned over the years

    with all of the new promising devops frameworks and technologies out there there is always a cost associated with

    switching one out for another we are often guilty of seeing the pros uh we get advertised too you know and

    convinced uh to make changes but then only after the upheaval of change do we

    see the cons you’ve just signed up for different problems so um

    i know i’ve seen that happen on teams a lot and i have plenty of stories about like switching out different um

    continuous integration servers um that kind of stuff so

    be careful um like i said it’s not you know it’s not a uh

    if it’s not broken don’t fix it that’s not what i’m saying what i’m saying is time time box your exploration make sure

    you always have time dedicated for that but limit your risk there if you’ve got

    you know i would limit a percentage of your time a percentage of your resources for new things new technologies new

    frameworks things like that [Music]

    we this this um there’s a reason for this picture

    i was on a team once where we we use source control sure

    but we didn’t really have a good method of verifying our builds first i mean it was kind of a

    we trust everyone to run a build first before they check something in and merge the code into into the main branch right

    well you know when more and more teams were moving to get and stuff um

    i remember moving uh to like a get flow type of

    um you know method and and it was really enlightening to me because

    what would happen to me in the past is uh you would come in in the morning you’d sit down at your computer and you

    want to hold back on pulling the latest code because what you need to do first is check your email and make sure that

    uh the build isn’t broken first and that you don’t just pull in broken code because then you’re stuck you’re stuck

    with that code um you know unless you want to fiddle with trying to you know troubleshoot you know having an

    entire development team and there were many many i don’t know 50 developers or whatever you know having everyone trying to

    troubleshoot and figure it out you know is kind of a waste of time right so if anyone was guilty of doing this

    you know late late late on a particular work day and they just want to get something in and they

    do it without running a build first on their local machine they had to bring in donuts the next day

    for everyone and uh depending on how much you value eating donuts you might not see that as

    a huge problem right off i think a lot of us just accepted it as this is how it was but there is a better way to do that

    and um this is where your you can find a lot of improvements i think um in your

    continuous integration system so um have your build server take care of

    this for you have automated builds don’t merge into the main branch immediately you know be able to have a mechanism so

    that developers can check something in but they get notified if it’s not

    going to work with the latest changes that are you know available in the code base

    it is one more step but i do really really appreciate the

    pull request system um being able to have that and to be able to have it set

    up to be notified if there are any issues that need to be fixed first before it gets you know merged

    in um it’s it’s all you know very helpful

    and part of that process too uh this was also a symptom of this problem um

    there were some of us especially ones that were new that um we just didn’t understand how the build

    work worked um in this particular scenario of course the build server would do the

    heavy lifting of a lot of building but there’s a lot of things that you can do as a developer to make yourself more

    efficient and one of those is just understanding your build how every component is built together now

    in my particular case we had a mono repo that’s a whole nother discussion but within that repo there were various um

    modules in there and if you were just to do the basic you

    know script to build everything it would take i took like 30 minutes or something to build if you’re making one change in

    a module it would be helpful to know what else is dependent on that module you know so that you could be a little

    more efficient with your time uh rather than just running a build every time you make a change um

    you know running running a build on the the unit test for that module uh would be really helpful then you could you

    know check that code in open your pull request and then let the build server handle you know the rest of

    it to make sure there’s no compatibility issues with the rest of it it can save yourself a lot of time so

    just that slow process that’s something that you can tackle with your junior devs by teaching them

    mentoring them helping them learn how the build of the product actually works

    not even just juniors but people that are being onboarded it’s very important for them to understand how all of the

    components of the project work together

    that brings me to avoid skimping on your test and qa

    environments especially when it comes to integration testing um i’ve been on teams before where there

    just was no test environment it was kind of like we test in production i don’t know how many of you have had

    that experience but it can be pretty miserable if you’ve got a low budget project or something like that sometimes

    that happens but it’s just hardly worth it and it’s an ex it’s expensive really it is

    um it’s better to have some kind of test environment where you can really be able to tell

    you know if there’s any integration issues it should resemble production as much as possible obviously you’re not

    going to have all of the same resources but with the containerization resources that we have available to us today we

    really have no excuse one example of this was a particular project i was on

    relied on internal in-memory database unit testing

    to make sure that you know everything was functioning correctly with a database well

    enough of worked with different kinds of databases to know that that’s

    not always the most reliable method and sure would have been nice to have a

    test environment that spun up a um you know a postgres database for example the exact version

    that we would be using in production to make sure that everything was okay before

    we let any changes go by that was a pretty that was an unfortunate uh people a

    people for our team at one point is realizing that uh there was a pretty significant portion that just wasn’t

    working correctly just because of a difference in sql between the in-memory database and

    what we were using at the time in production also

    any of the artifacts that you want to deploy make sure they have gone through

    the process of q a and test via promoting promotion of the exact same build rather

    than rebuilding for a particular environment or for the release

    if your environments require different configuration this should not equate to a completely

    different build artifact that kind of defeats the purpose right because you’ve tested something but now you’re going to

    build something else to put in production your build should happen once

    and as that build goes through the stages um it should be that artifact

    that backs that build should be promoted any configuration and stuff

    that is different between certain environments that really shouldn’t be hard-coded within the project anyway it

    needs to be managed outside of the build artifact either through flags

    flags on launch for example or config files that kind of thing to make that

    you know a little bit more resilient

    then we get to deployment um humans are horrible at repeated tasks we

    get bored we get complacent uh we always miss something um

    i was on a project once where there was

    uh a certain deployment uh and none of us you know we were just now you know

    learning this is a brand new project it was being brought in from a third party contractor contractors being brought in

    internal for our team to maintain and stuff and

    uh we were unclear on how it was being deployed and it turned out that there were some scripts that were written and

    these scripts were i say multiple scripts in multiple different places and multiple different versions of these

    scripts so obviously not the base not the best way to be going about deploying something reliably and repeatably every

    time and even more important is like we didn’t really have a good way of rolling

    something back to a previous deployment um and prior to moving to a devops team i

    did not really know how stuff was deployed i just kind of you know i did my job i committed my

    code and i was done i wasn’t really involved in deployment so much but after learning how that worked at least for

    this particular project um i realized how much i didn’t appreciate the trouble

    that they were going through um just like knowing your build deeply know how your deploy works as

    well list out exactly what it takes to deploy and whiteboard those steps if necessary

    and most importantly automate what makes sense and you can do that in pieces uh

    remember that one percent idea this isn’t something it’s it’s not all or nothing so

    do your best you know find the places that you can automate and get that done and you will continuously improve

    so let’s recap just that section uh understand your project’s business values and goals that means eliminating

    those silos getting rid of those communication problems document your flow and pipeline you may

    be surprised by some things there and this will be invaluable for when devs move on and off your project

    prioritize and improve by one percent uh don’t get yourself overwhelmed trying to fix everything likely you will solve or

    or improve one bottleneck to discover another and you need to move your priorities at that point

    make your work and your metrics for success visible and that just means setting yourself up for up for success

    by measuring a baseline before you make changes and set up realistic expectations for the outcome and this

    will help you with prioritization if you realize during that exercise that a desired change turns out to have an

    undesirable cost to benefit ratio that doesn’t match up with your project goals

    and then master your ci cd your source control management your testing your artifact management and

    your deployment know the details of your build where your artifacts are coming from and how things are being tested and

    deployed and then of course automate where it makes sense don’t just automate for automation’s sake automate what is

    going to give you the most bang for your buck and obviously this is going to depend on your priorities your

    definition of success for the project but you might find that automating only a portion of your workflow is all that’s

    necessary to meet or even exceed your goals at that point and you can move on to more fun stuff you know new feature


    okay we’re going to take a short break here and i’m going to just talk a little bit

    about rube goldberg machines because i like them very much i think they’re cute they’re fun

    this is a pretty easy internet search these are very popular but for anyone that is not familiar a rube goldberg

    machine is a ridiculously complex and inefficient machine usually it’s composed of a high number of moving

    parts and it only it sets out to achieve one specific goal

    and rupe goldberg the person was a pulitzer prize winning inventor

    he was a cartoonist and he made these machines popular through the cartoons

    that he drew that was something that was new i didn’t realize that the one shown here is called the

    self-operating napkin and you can see all of the necessary steps labeled obviously this isn’t a real life rube

    goldberg machine we’ll see a couple of examples of those in a minute

    but another interesting fact in about rube goldberg is that he is the only

    person whose name is in the merriam-webster dictionary as an adjective

    so i thought that was pretty clever another

    another thing i uh was able to find out is there’s actually a contest held every

    year for participants to create these um you can go to goldberg.org to find out more about that

    and each year has a theme uh this this last year was um the focus was on

    literacy and the challenge was to create a machine that would open a book and you can find videos of the top 10

    selections of the machines that were built they’re super cute there’s a lot of very clever kids out there that are

    creating these so this is fun um but where am i going with this

    and this this is what i’m talking about uh we talked about goals for your software

    delivery life cycle i’m going to make the argument that actually putting it together and implementing a pipeline is like a rube

    goldberg machine but i want to expand a little bit more on that analogy and there’s three characteristics that i think of when i

    make that comparison um there’s a lot of moving parts uh there’s inefficiencies

    and there’s a tendency to be unreliable and unsafe so this book that i was involved in

    writing um i empathize wholeheartedly that software

    development and delivery is not an easy problem to solve especially if you’re new to today’s landscape if you’re

    involved in cloud native applications all of those components and tool chains that could be potentially involved in

    your delivery pipelines became very apparent to me during writing this book

    this was published earlier this year devops tools for java developers and my co-authors and i came together with this

    intent of gathering for readers much of the knowledge and background that’s required in order to develop and deliver

    software efficiently and safely these are a lot of the topics that we

    discussed in the book and this material gives you a really good idea of what you’re getting into and what

    it’ll undoubtedly result in you wanting to take a deeper dive on many of these subjects in order to develop a solid

    development and delivery pipeline we did discuss some of the most common tools used in the industry and we even touched

    a bit on kubernetes here but what becomes abundantly clear is that the entire process can be overwhelming

    significantly more so in a cloud native environment [Music]

    i did a quick search for the largest rube goldberg machine and it turns out there was a video posted on december

    10th uh 2021 last year this whole machine takes four minutes

    and 26 seconds to run you can see how happy they are at the end

    that’s how i felt after getting my first pipeline successfully deployed

    this made it to the guinness book of world records and it had 427 steps involved

    [Music] uh many of you have probably seen this already this is the cncf cloud native

    landscape map this is from the cloud native computing foundation which is under the linux

    foundation and i don’t expect you to be able to see a lot of the logos here it is massive

    but it’s just a just to highlight one of these smaller sections um labeled orchestration and management there are

    20 items that are categorized under scheduling and orchestration tools and frameworks there are seven under

    coordination and service discovery there’s 10 under remote procedure call there’s 21

    under service proxy that’s where you’d find something familiar like nginx

    19 under api gateway i thought that was interesting and then there’s another 17

    under service mesh and i love

    jigsaw puzzles um that landscape became a meme and an earlier version of it made

    the rounds on twitter a couple years ago as a thousand piece jigsaw puzzle

    um i was thrilled when i went searching for this and it is actual it was an

    actual thing available on etsy but apparently after selling 42 of them it’s

    now sold out if it ever comes back i might consider

    getting it i like jigsaw puzzles enough i’ve made the case for a lot of moving

    car parts um but the fact that there is a lot of moving parts um that’s not necessarily a bad thing in and of itself

    but what you choose to use for your particular use case and how you use them that can lead to a measure of

    inefficiency which is the next part if you’re new to this landscape it’s going to take you a while to find your

    place this is reasonable to expect it will take the expertise of the entire team from development to operations

    although some have been involved across the board from beginning to end it’s usually expected to take some years of

    experience to be able to do that it’s a tall order to ask one person to know everything and we discussed that at the

    very beginning of this talk that that’s just not efficient to have one person know everything

    um or be expected to anyway this is something that you have to grow over time with uh to add these things to your

    toolbox um this is one of the misconceptions about the uh the whole

    devops methodology that i come across all the time um you know putting that responsibility on

    on everyone you know having developers know everything about ops and vice versa just doesn’t work out very well

    just know that you will make mistakes but the important part is to learn and continuously evaluate your solutions as

    you go [Music] this rube goldberg machine dates me a

    little bit i actually remember using a typewriter i also remember running out of the white correction tape that you’d

    have to load into the typewriter um this one uses a drill to press on the

    space bar but it reminds me of something really specific with software delivery

    pipelines and it’s one of the more interesting phenomena that i encounter most it’s that software tools

    can be used in ways that were unanticipated and often in ways that are unintended by the creators and the

    designers and this can be to to a number of different factors um

    the first just a lack of due diligence when evaluating a tool for use or simply an attachment to a tool

    a desire to make it be a one-stop shop make it work for everything

    development of a pipeline that’s ad hoc that that implies a lack of planning or agreement on what actions are required

    and where this can actually lead to several different tools being used that overlap

    that are redundant perhaps the unavailability of a feature

    that you want either the team doesn’t know where to look or they aren’t allowed to use available

    solutions because of licensing or other corporate restrictions that can happen

    and those are all common reasons but regardless of why this can cause an assortment of issues and one is that you

    end up with glue code that can be a maintenance nightmare an example of that

    of glue code is that it provides some kind of compatibility between two different tools that might

    make sense at the time but then all you need is one of those tools to be updated and then either the tool

    you know it could happen to either side and then your glue code is broken um that’s a maintenance nightmare

    another is that projects are tools that originally had a specific focus

    they start to become spaghetti code unmanageable and bloated

    sometimes this can just be the result of an attempt to respond to community requests for features that are really

    outside of their focus and it may be that a different tool altogether would

    be better suited for those requests and there’s a lot to unpack there especially when it comes to

    meeting the needs of the community and open source projects that’s always a challenge to push back

    on things that can you know completely make sense for a specific use case but

    maybe it doesn’t make sense for that particular project as a whole [Music]

    so all of those things are listed those can lead to some pretty severe inefficiencies in your pipelines

    this brings me back to the little shell scripts that i found scattered all over the place that were meant to deploy a particular product

    in the past okay there’s one left this one is actually pretty serious when it comes to

    pipelines and i’ll go more into detail on that especially as it pertains to containers

    remember this slide bringing it up again and again this doesn’t even address what

    happens after deployment this is all just up to deployment but even if you’ve managed to cobble

    your system together with all of these different parts there’s likely points that include that glue code i was

    speaking about earlier that handles various integrations between different tools and this can

    lead to a measure of unreliability we discussed that as a developer there’s a phrase i’ve

    heard often and it was called happy path coding or happy path

    programming um meaning that you don’t consider all of the bad scenarios you just get it

    working and get your happy path done um not always the best way to work

    um and just like with happy path coding you can easily end up with a happy path

    pipeline where everything needs to behave just so just like a rube goldberg machine

    in order for a successful end result uh reliable pipelines more often than not come from a lot of trial and error

    and much like i imagine that you know rube goldberg machine we looked at earlier um that would take a lot of time

    too in fact the the team on that machine said it took three months to build that machine

    glue code or not everywhere on this slide where you see one of those green connecting lines

    those are places to start looking for weaknesses and putting some serious thought into what to do when there’s a break in one of those connections

    is the result that the entire pipeline breaks down or are there some measures that you should be taking to handle

    paths that are not happy paths

    there’s clearly a ton involved in putting a full pipeline together

    i’d be remiss if i didn’t mention those parts that are terribly unsafe if not addressed this diagram comes from the

    salsa website um that stands for the supply chain levels

    for software artifacts salsa it is a security framework and it’s basically a

    checklist of standards and controls that you can use to improve integrity

    secure your artifacts and packages and your infrastructure in your projects

    it’s it’s a good um explanation of how you get uh from safe

    enough to being you know in a better state um at any length in those change so any of

    those weak points between your source code and delivery of your product or deployment of your

    service there’s places you need to take extra care to protect your supply chain and that is indicated by each of these

    red triangles those are opportunities for disruption and as a developer i focus on the ones

    that are around the build process even around the continuous integration build

    processes in a lot of the talks that i do i focus a lot on dependencies in particular that

    is definitely in a developer’s realm so it makes sense

    all of these pieces and parts of libraries and other artifacts that your code depends on this aspect becomes even

    more important when using containers and one of the most crucial areas to harden is your build process during

    continuous integration you must must ensure that the dependencies that you’re pulling in are

    exactly what you expect them to be for example they need to have an expected version which means you need to

    specify a version you also need to verify they’re using a trusted dependency and not a malicious

    package that happens to have a similar name for example now

    as java developers we’re a little spoiled here because i know there are

    definitely been some problems one jumps to mind immediately for java developers but

    um we at least are protected a little bit with you know maven um central like when we

    use maven for uh you know to pull in our dependencies um at the very least

    these uh have to abide by a namespace um you know the an owner of that namespace

    maintains it um when you get into other artifacts like

    uh python or javascript if you’re using npm it gets a

    little more tricky because anyone can name a package anything and as long as

    that name is available they can just throw it out there you know to npm and to

    you know so that it’s accessible by pi pi and all of that

    these are big problems and it has come to the forefront there’s a lot of vulnerabilities that have been

    discovered out there that have to do with that type of behavior

    so managing these dependencies responsibly a lot of this is just understanding your build and where these

    dependencies come in since using a docker file is pretty common let’s start there talk a little

    bit about a docker file this is an overused graphic i know but that concept is out there that

    software is potentially made up of a ton of components that a developer doesn’t necessarily have first-hand knowledge of

    that cannot be understated applications and services today are more complex than ever

    developers generally don’t want to recreate the wheel if it isn’t necessary you know they want to continuously put

    out new stuff this means reusing a lot of you know libraries and maybe open

    source libraries or libraries even just from other internal teams i mean it makes a lot of sense to be able to do

    that containers especially are an issue container images

    when i started working with container images i was pretty naive in the beginning i just you

    know wanted it to work i learned enough about docker to get my images built and then started you know

    pushing them out there to see how they run but the more and more i looked into it the more i realized how much you can

    shoot yourself in the foot if you aren’t careful and this is just a simple contrived docker

    file and there’s several problems in here i’m not going to go over each one because there’s more details i’ll go over

    later but there’s there’s a few so i mean obviously that first line from untrusted

    parent image well what does that mean well a lot of these you know docker files um images can be built based on

    another image on a parent image there’s that hierarchy there and

    if you don’t know where your parent image is coming from how do you know what’s going to end up in your container when it gets launched in production i

    mean these are things that you have control over there’s no tag here this is just the name of an image there’s no you

    know version associated with it um even if there was images have this

    crazy unreliable tagging mechanism that can be overwritten at any time your

    processes can help with this of course but the reality is it’s very easy to override an image

    tag with a completely different binary i mean they’re just pointers so

    you might want to consider using a shaw or something

    um lines two through four uh again not

    specifying versions apparently whatever the base image had there’s still some os packages and stuff that were needed

    and you know i often see lines like this in docker files but if you ever need to

    build this image again you’re going to run into some you could potentially run into some

    troubleshooting problems because any of these packages could be updated they could be updated in an incompatible way

    and then you’re just gonna have to chase that down um line six

    um that’s that could be a problem basically you’re copying everything uh you’re not being

    real specific about what gets copied from you know your working directory or

    from the directory that you’re building from what gets passed to the docker daemon

    you can get a lot of stuff in your final product that you don’t want i’ll show you some more details on that

    in a bit number seven line seven there’s an

    external resource there that you don’t have control over we’ll talk about that more in a little

    bit as well and then finally uh number nine

    uh there’s no like user or group assigned so this just runs his route this little

    start script here just runs his route these are just a few problems that i

    come across frequently in docker files it’s a really good place to start but all in all this proves my point

    software delivery pipelines are definitely a little bit like rube goldberg machines

    so you’re probably wondering if there’s any real solutions to these problems let’s talk about that a little

    bit here there are a ton of tools out there that will help you with the real case scenarios

    as overwhelming as it is that cncf landscape for example at least lists

    tools within categories that will be helpful to you if you’re just starting out you just have to pick one

    for now quite a bit of research is required to vet and evaluate those solutions but we see how important that

    is so make sure that you you know do that the hardest part is just getting all of

    these pieces to fit and the industry right now is ripe for real holistic solutions that solve the basic problems

    of continuously delivering and updating software as well as monitoring and troubleshooting in production

    simple to say but there are a lot of one-off tools out there that have grown

    out of very specific engineering problems and what we really need right now is

    agreement in the industry on what best practices are so that we can wrangle some of these

    tools in organizations are coming together to work on these issues

    one important effort that was started is between jfrog and nginx

    f5 and nginx they have like this open source project it’s called the mara project that they’ve been working on

    it’s an acronym for modern application reference architecture and uh it was announced last year at their conference

    sprint 2.0 um it’s basically a framework for deploying any

    containerized application in a reliable and repeatable way and i’ve taken a look at it and there’s

    definitely some things that we want to work on and improve and make more uh interchangeable more a little more

    abstract to be used for different types of circumstances but this is just one example of

    organizations like we’re working together on this one with that intention of producing open

    source at least that will be available to everyone in order to push forward uh best practices

    there’s also an effort with the cd foundation this is a sister organization to the cncf it is

    also underneath the linux foundation they are working on

    um determining and and defining best practices at this higher level at every

    uh every business interest every stakeholder interest in order to define

    why we’re doing what we’re doing in order to help guide and direct some of the efforts that are happening on the

    engineering side of things [Music] this is the

    landscape map for the cd foundation foundation the continuous delivery foundation it has a number of different

    associated tools but it’s not quite as overwhelming as the cncf it has more of

    a limited focus on continuous delivery explicitly

    and they you know are really working hard on

    discovering things or ways to make tools more interoperable

    rather than strongly integrated that’s that you know avoiding those glue

    code scenarios we talked about earlier and of course outlining best practices and there’s a number of organizations

    that are involved in the cd foundation all of these meetings that they have especially with the special interest

    groups they’re open to the public if you were to go to the cd foundation website they have a public calendar you

    can definitely attend any of these meetings i’m a co-chair of the interoperability special interest group and we often have

    these projects come and present to us explain to us why they exist

    why what benefits do they offer that kind of stuff and that way you know we learn more as a

    group and we publicize projects that we find that are that we

    find very valuable a couple of examples of projects that have come out of these efforts um this

    one cd events actually originated in the interoperability group um

    this one uh basically you know it is is a

    you know defining a way for all of the products to

    submit and to read events from other tools

    in a common format if any of you worked with cloud events uh that’s a very similar thing and

    that’s really what this was based on with cloud events to start is just defining what that means

    so that we have this common place and can avoid the glue code scenarios another project that came in

    recently is persia this one is actually backed by jfrog but it is an open source project and it has

    to do with decentralizing package registries

    making sure that where possible if there’s you know access to source

    that it can be verified between a few different nodes that taking that

    source and building it ends up with the same artifact this gives you a little you know more

    security knowing that you’re you’re getting what you asked for

    that the artifact that you’ve requested hasn’t been manipulated in between you know the request and when you receive it

    so we are developers we do need to talk more about dependencies

    not just security issues are involved here there’s a lot of efficiency issues as well

    and again i’m going to chat you know bring in

    this slide all of these places where we end up dealing with

    dependencies uh this is not reserved for just operations to worry about it is part

    of our build process an integral part of our build process is to be able to pull in the dependencies that we need

    and then knowing how uh something is going to be deployed later you know that’s going to affect

    your the decisions of your developers on you know way back in the design stage and how they create these artifacts to

    begin with [Music] this is a simplified version of the

    pipeline you know beginning from development going through ci going through testing release process and then

    deployment where is it in this whole

    uh process should we be concerned about dependencies well pretty much everywhere

    i mean this is where we live developers are constantly going to be bringing these in

    and on to their local machines so they need to know where these dependencies are

    coming from in the build server we also need to know where these dependencies are coming from

    are they coming out from the wild west from some you know public repos or do we have some control over where

    they’re coming from um same for testing this is when we retrieve our artifacts that we’ve

    created in the previous steps so we care where the artifacts are coming from is that location has that

    been protected and is that a trusted location [Music] and then for release you know we need to

    be able to promote particular artifacts that have passed certain quality gates as they’re moving

    through the stages of the pipeline and then ultimately deployment uh super important that what we’re deploying on

    our production machines is what we expect one thing i’ve seen happen in the past

    is maybe not enough concern given to permissions

    when it comes to automation for a deployment and what i’ve seen happen is um accidents happen

    not necessarily on purpose um where you know test artifacts were deployed on production by

    accident uh that could be protected you know that could be prevented by protecting your

    production you know making sure that your production machines only have access to production resources so that

    whole idea of promoting as quality gates are passed and guarding those locations carefully

    that can really help you out there okay i want to talk more about uh

    images creating container images this is something that is becoming more and more important um

    even as a java developer even if you’re not containerizing your java applications you’re likely going to be

    soon if not java some other language so i think this is really important to

    get a jump on if you have not done anything with containers in the past obviously there’s a ton of documentation

    docker has a really good documentation i’d advise starting there obviously you don’t have to use docker

    there are other tools out there but this is a good place to start to start recognizing where there could

    be problems where you can make improvements there’s 10 of these i’m going to whip through

    these really fast the dot docker ignore file i did not know that it existed when i first

    started using containers what annoyed me the most about one particular project was how long it took

    to build an image every time i made just a tiny change it seemed like it took forever to build and

    that was because i did not understand how the docker context worked um this is another contrived example but

    that line number three is the problematic line that we saw earlier in the docker

    file that is copying everything into the working directory

    and let’s be clear on how what’s actually happening there when you run a docker build command to build an image

    using that docker file everything in that current directory is sent to the docker context

    in order to make it available to run all of the commands that are in the docker file

    everything is then copied into the image the docker ignore file is very much like

    the get ignore file you can use it to avoid adding any test or user specific files or even

    secrets that you might have stored locally and also consider that dot git directory that’s probably not something

    that you want to be putting in a production image [Music]

    so at first i thought i could just improve my build time by oh i’m not going to copy everything i’m just going to explicitly say what i want to copy

    but in reality that doesn’t help enough because this is what’s happening in the background

    again everything is being sent over to the doctor daemon that takes some time and then the doctor daemon

    determines oh you only want these two files copied okay i’ll just take those over so the docker ignore file

    helps you out with that very first step there it only sends over what’s needed can save you some time

    [Music] so this is an example of what one looks like you can avoid adding readme files

    that you don’t need passwords of course the dot get directory log files temp

    files test files you know all these things that will just make a production image a little more bulky

    next one is using untrusted base images we talked a little bit already about this

    i just didn’t realize back when i started how base images worked

    until our build broke one day with an error message saying that the base image didn’t exist well it turned out the base

    image in our docker file was actually getting pulled from a private repo of the original third-party contractor

    which was out of the picture for like a year by then so i mean there was no expectation that they would

    always keep that up so again make sure you know where your dependencies like this where they’re

    coming from make sure you bring them in internally somewhere so that you have control over them

    uh this is another problematic base image obviously you’re not going to pick

    something named evil image but my point here is just to pay attention where they’re coming from these are

    dependencies just like anything else they should be pulled into artifactory or some other artifact management tool

    that you use and managed along with all of the other binaries that you rely on for your build

    using docker official images is a good idea uh docker has a dedicated team

    that’s responsible for reviewing those uh they are open you can see exactly what’s in those and see how

    they’re being built they are you know make sure that they are updated regularly

    um that they work in collaboration with upstream software maintainers and security experts in the broader docker

    community so those are good to start with but always do your due diligence and make

    sure you know what is in your base images also are they coming from a private or a

    public registry by default if you don’t say a registry if you don’t indicate a registry they’re going to come from

    docker hub so you might consider you know caching them in a private registry

    that you have control over what to use i remember

    just when i was first starting and just trying to make a little container with some java application running in it

    um i didn’t really know where to start um there’s all these names out there

    you know if you’re a linux person you’re to figure this out and it isn’t you know magic it’s not all

    unclear you’re going to see names like that in other images besides like java images

    or open jdk images as well but there’s no mystery in here when you understand what they indicate

    you can narrow your choices down by the java version that you want if you need to install additional

    packages you might want to pay attention to the linux version that you’re using and then i put some notes in here about

    alpine because if you need to look at the documentation when you’re pulling these from docker hub so if you go to

    open jdk on docker hub they explained at one point that alpine

    images were um you know they’re based off the alpine linux

    project um but they weren’t really supported and according to their documentation they were only released

    for early access open jdk version releases and they are not updated after

    uh ga so they’re older images that you might not want to use now all of that said uh anyone who has gone to openjdk

    and looks at it now uh they’re deprecated now so if you’re using these images you might want to go

    consider you know replacing them with something else if you’re already paying for support for

    a specific distribution like from azul using zulu images or

    ibm or something or you know you can use the coretto ones from amazon

    it makes sense to use their official images so pay attention and make sure that you’re updating appropriately

    not defining versions um not much to say here here other than what i just said

    make sure that you define your versions we talked about this earlier um you really should have a bill of

    materials for your build know the exact version of everything that is installed

    um and then you can eliminate multiple moving parts um

    you know when you’re building your applications if you control the images one thing i would suggest a lot of i’ve

    heard the reasoning of not defining an image because you always get the latest and greatest and you might as well just

    run with that um the problem with doing that is it just like i said it’s too many moving parts

    if you want to do things that way make sure that i would go ahead and explicitly use them use the versions

    that you’re expecting and dedicate a build where you do just that because um

    you potentially have a lot of troubleshooting to work out so make sure that you’re not risking you know getting

    a feature out or getting a bug fix out just because you’re you know trying to figure out which

    dependency got updated and when which one is incompatible [Music]

    uh we talked a little bit about tags how unreliable uh image tags are using latest is just

    asking for it like i said these are pointers latest is not always latest it doesn’t

    always mean what you think it means uh it can always be overwritten it’s kind of like using a snapshot like when

    you’re building your job application and you’re using snapshot images that that

    it’s a good enough analogy i guess

    uh never updating of course we need to update our stuff like i said make sure

    that you dedicate some time to update things uh we know of a specific case where a particular company did not

    update uh even knowing that there was a vulnerability involved and how embarrassing it was for them when they

    actually suffered the consequences of that so block out time during every development cycle to spend on this kind

    of maintenance [Music] so here’s an example um you know make

    sure that you update your base images as well keep those current just like we talked about with openjdk

    including build tools when i first started out with um java and and building containers and stuff i thought

    well how convenient is it just to include one that had maven already right not have to worry about this and i see

    that quite often it works but you end up seeing you end up with an image that’s quite a bit larger than it needs to be

    and what do you need maven for on a production image you really don’t need all that extra stuff so it’s best also

    to minimize your attack surface from a security perspective just the only thing that goes in your image

    are is what is absolutely necessary to run your application um

    so one thing to consider is utilizing multi-stage builds there’s an example of

    that you see that there’s two from lines in here that’s a feature that docker added

    um this way you can choose a very very slim minimal base image and you can just copy

    you know the results of the build into that image so that’s one option for you

    using external resources remember in that contrived example we had before there was a curl

    statement i’ve seen this a lot i’ve seen curl i’ve seen wgets you know these kinds of things

    it’s one thing if this is a resource that you have control over again that you update when you want

    but i have seen cases and have experienced the case where the results of this statement is uh nothing because

    whoever was maintaining it decided to move it decided to rename it decided to update it to something incompatible so

    rather to be on someone else’s schedule like this it’s a better to pull in

    that resource pull it in so that it’s managed internally in your artifact management

    rather than reach out for it during a build

    trust but verify of course um hard coding secrets that seems like an

    honest you know obvious one uh but i see it a lot

    uh unintentionally um one thing i’ve seen in the past is also this attempt to

    remove secrets so as an image is built you know layers are are added

    and this is an attempt to just remove the secrets that weren’t wanted in a base image

    well um they’re still there though i mean just because you can’t you don’t you

    can’t see them if you were to exact into the image they still exist this isn’t a good idea

    in this case it would be better to rebuild that base image and make sure that those secrets weren’t added to

    begin with um also this is another example of making sure that you

    make configuration um stuff like this like this this port for example that you’re exposing maybe

    maybe you need a different port for a different environment so um

    you might want to use a variable instead in this case that way you don’t have to rebuild this

    image for a particular environment [Music]

    just more examples making sure you don’t have secrets and making sure that you’re

    using variables [Music] i’m doing too much this is just using

    these docker files as like a shell script not a good idea

    this one’s somewhat contrived again for simplicity but it was modeled after an

    actual docker file i’ve worked with in the past there’s some crazy side effects here because think about what happens

    when a test fails um you really need item potent operations

    in a docker file testing like this if nothing changes in a previous layer

    it doesn’t get regenerated and rebuilt whenever you’re utilizing docker cache so

    in this particular instance if you’re using docker cache um that cleanup database stuff is not going to be

    repeated but you’ve already failed a test so that database is not in a good state for this

    to ever work so um a better way to do this is keep this testing stuff out don’t be doing that in

    a docker file build the image and then have an environment where you can

    run your integration tests against that image as it is [Music]

    yes so keep it simple all right um unnecessarily busting cash

    uh this just slows everything down and when you start pulling apart fat jars for

    example you discover that it is very amenable to you and your time to make

    sure that stuff that changes frequently is lower in the doctor file then things

    that don’t change as often so perhaps you might want to consider putting you know third party libraries in a previous

    step and then copying over you know dynamic code or stuff that you are changing frequently

    that is an active development you know those class files and stuff can come in later

    good to keep your dev builds efficient this way there’s many different ways to build

    java images for example whatever method you choose there’s a few out there

    you can use that like a maven plugin you can use build packs you could use

    jib understand how it’s actually building that image because things like this will

    come into play [Music] so again you know keep stuff that

    changes frequently as close to the bottom of your docker file as possible

    all right lastly is relying on root

    i feel bad for repeatedly saying this one over and over and over again

    you really shouldn’t be doing this you shouldn’t have processes running i mean if if a process is running as rude it

    could potentially have access to things it should not on your host um make sure that you’re specifying a

    group and a user uh if you’re running a process um pay attention to official images that

    you pull from docker hub because sometimes there’s this you know understanding that you’re going to be using a user or a group um

    you it’s uh it’s just a an issue where

    um the moment you know anyone if anyone can get into that container they immediately

    have root access i i mean why make it that easy so if if it must have root

    access make sure you have some good reasons for that um but um you know

    use that principle at least privilege and make the right decisions there

    so this is an example of that um this start script uh obviously is is running as root

    [Music] and here’s an example of just adding a group and a user i think if you if you

    were to look like a postgres docker file for example they’re pretty good about this

    [Music] and there was a report that came out

    this was a cystic report and i thought it was interesting that they just did an observation and 76 of the containers

    that they observed using cystig were running as root had processes running as root so

    i i think that everyone’s heard this already but even if that’s true it just it doesn’t appear that we’re taking that

    risk seriously enough okay i prattled on enough about

    containers uh let’s just do a quick summary about what a java dev should know about devops here

    you should understand your software development life cycle goals and objectives

    you should know where your dependencies are coming from that should not be a mystery to you for example when you’re

    building on your local machine um are you pulling your dependencies

    from maven central every time or are you pulling them from a private

    uh repository or you know where are they coming from important to know um know how your artifacts and your

    containers are built just like we went through understanding how a container image is built with a

    docker file it’s important to understand that has huge effects on your efficiency

    how well you’re going to be able to get through your pipeline from beginning to end

    pay attention to the risk of vulnerabilities there are a lot of tools out there right now and i could show you

    some if you’re interested jfrock has some really good security tools there’s a

    jfrog x-ray does scanning and there is a plug-in for my ide that i can use and i

    can just see right away before i even check my code in i can add a library to

    the pom file for example or add add a package and it’ll tell me if there’s any known

    vulnerabilities with that so that’s pretty cool docker desktop has some good scanning resources as well

    um and then also you know the ability to do that within your pipeline too not only should you be doing it at the beginning

    but you should also be doing it you know as the um as your software goes through the

    different stages and then finally how to plug in and be active in your community remember what i

    told you about the cd foundation it’s not private stuff going on there there’s a lot of a lot of happening

    things and you know get yourself involved when you choose a new tool when you’re

    looking and vetting a tool find out if there’s a community resource that you can get involved in you can ask questions

    you can see what other problems other people are having that will help you with your research and help you with

    your decision making lastly before i

    sign off um is uh just make sure that you go and get

    your free copy of this book there’s five of them available so go ahead and sign up and you’ll be contacted by email

    within three business days so we can send you your copy

    all right that is all thank you for listening feel free to dm me with any questions or comments you

    have or if you have any questions today i’ve got some time here we can chat for sure

    all right um and i’m not looking at the comment log but first of all thank you so much i really enjoyed this and um got

    some ideas on sort of what i need to do in my environment with our devops pipeline we use um a yaml file and then

    the pipelines that azure devops have but we definitely want to try to apply a lot of the stuff that you

    said uh for our environment as well um chris what’d you think or do you have any

    comments i thought it was an excellent um survey of the millions of different

    things that you could have brought up and i thought you did a really nice job of condensing it down to some important

    high points that people need to investigate further cool

    thank you very much i’m gonna try to put up that comment uh log from the youtube

    let me hang on a second i thought the thing about splunk was hysterical

    it wasn’t so funny for me at the time yeah you know people building extraordinarily elaborate dashboards

    harvesting logs you know and meanwhile we’re in there constantly changing the logging and

    tweaking it or changing levels or changing the messages not even thinking that it’s going to

    blow up this finicky um brittle thing somewhere exactly in a

    data warehouse that we know nothing about yeah our team was not even aware that there were any metrics

    you know that there were any measurements being taken so we didn’t even know to ask yeah no that’s that’s just beyond classic

    all right and so the only thing i see is um well somebody commenting on the puzzle

    as a great option my wife does a lot of puzzles as well in fact she has a mat that you can roll it up if you want to

    pause and then roll it to continue on but um and then uh karina uh carrie gunther

    um had some questions about uh produces and

    privacy because the hip rules we have the same issue at cdot we have some data

    that’s law enforcement data that is sigis is what they call it and we can’t share that and so we kind of have to set

    up some test databases and load them up to do uh to do testing so that we can you know

    not compromise the data and you know just abide by our security concerns yeah that is difficult i’ve run into

    that too especially if you’re working with any you know any private data um any identi what is

    it called the identifiable data that you’re not on yeah yeah which includes like

    the email address is um obviously credit card information and things like that

    yeah there’s not a good way to deal with that other than try to duplicate as much as possible in the

    correct format uh in order to get the amount of data that you need but yeah that’s something

    that needs to be developed over time too as you make changes you know add more and more types of data

    to your test database one thing i will say about that um

    i mean that is a hard problem to solve i don’t have an immediate solution for that but make sure you’re protecting your resources um between your

    environment i don’t know if any of you have had that experience where you’re like oh no it’s almost as bad as

    you know deleting a git repository or something on accident but it’s like you run your tests against prod

    because you’re connected to prod or or um you accidentally delete the entire

    test database on accident because you didn’t realize you know what you were connected to so these are things that

    can be prevented if you really pay attention to uh you know permissions that you have to

    various services yeah definitely i i mean for example for

    work i do at cdot we have uh the production database shows up in a sort of like a red window dev is green and

    then qa is yellow so that you know okay i know where this is going when i go to

    do a query or something like that um a situation where

    uh one of our devs it felt so bad for him accidentally launched like a

    um message broker but accidentally connected to

    uh production oops so he’s running tests and and in production it’s like what’s

    happening to the messages the messages are just being collected and disappearing and it was a test

    environment that was pulling them pulling them straight out of production so yeah

    we’re just a few funny stories got a suggested meetup which was a test message

    yeah so the other thing is um one of our attendees commented that they

    really like the wonderful images uh for the context and point you were trying to explain and and they appreciated that

    oh cool and there it is thank you matt all right matt i know you can’t you’re having some

    technical challenges but if you want to put in our chat if there’s anything i missed but i think i got all the stuff

    everybody liked the rube goldberg and i don’t know if you ever had this as a kid but i did and then we had it for our

    kids there’s a game called mouse trap where you build a rube goldberg machine and the idea

    to win the game you have to catch the mouse using this rube goldberg setup

    i love it yeah so um yeah that’s all i got

    thank you so much i really enjoyed it i guess i’m gonna wrap and let you everybody go eat their dinner or do whatever they’re doing next

    so um thanks so much for coming out um i uh i tweeted some of the stuff that you mentioned if you could uh email

    zeddy the slide deck and we’ll share it on our um on our meetup site but i also

    shared on twitter um the uh some of the links for those tools that you had shared with us oh awesome

    cool so and i tagged the denver jug so you guys should be able to see it all right um well thank you and have a

    great night and uh for everybody at the attend we’ll this again in october our plan is for

    november to be in person but please please please let us know a venue that we can meet at

    that can accommodate vencat because usually there’s a big crowd some of the venues we’re looking at want

    to charge like a few hundred bucks for security which if we can get a sponsor for that that’s

    great but we’d rather not spend your money on that if you will i’d rather spend it on food for you and

    drinks and other things like that so all right thank you and have a great night

    good night