Service Mesh Solutions: A Crash Course @ London Microservices – 2020

first thing i just want to say thank you to everyone for uh coming and uh for the organizers for

putting these things together it’s not the easiest thing in the world just to uh throw meet-ups and um even

conferences and stuff together so i feel privileged to be here i’m glad to speak to you guys

this is the first remote meetup group i’ve spoken at which is pretty interesting considering the

state of affairs this year you’ll probably see me speaking at various

online conferences and things like that i was just speaking with daario before this

call about how conferences having the online conferences have both good and

bad aspects and one of the good ones is the ability to be able to you know reach across the pond i’m i

live in denver colorado so it’s pretty amazing for me to be speaking at a um london microservices meetup group pretty

cool stuff um also i’m able to um you know via this

um digital persona able to uh um speak with you know people from all

over the world and um interact with people that maybe don’t go to formal conferences very often

anyway so anyway i’m really excited for this opportunity um i gave this a version of this talk

and this was the very first one i ever did in my career in speaking and i i gave it at code one

um a few years ago i’ve updated it a little bit i mean there’s there’s been some changes

in the uh ecosystem since then so i’m excited to share some of that

with you um let’s keep going here um also you can um find me on twitter

i’ve got my twitter handle there um before we get any farther we do have

a raffle that um we that jfrog has given us uh here’s a bitly link for

that or pull out your phone and get a picture of this qr code and you get a chance to win some apple

airpods pro headphones which is pretty awesome the winner of this will be selected in

two business days and uh contacted via email thank you ari for setting this up through jfrog

a little bit about me um i am a mom i’ve got two children um they

are almost out of the house i’ve got a son that’s a senior and my daughter is in college now she’s a sophomore this

year i’ve been a developer for the past goodness over 20 years now

in various companies various tools various frameworks my primary coding language is

java server-side so i i do have a bias toward that but most of my speaking

comes from a developer point of view um i did move to jfrog

in february and i’m now a developer advocate there completely different way of um

of doing things now before as a developer you know was focused on feature

and bug fixes and um more you’re in that being

i now have to you need to share them with other people because some of your

lives a little bit easier this all started for me several years ago i have to put a plug in

for contest again something if you hear chance 10

and a conference definitely do at least once very very difficult normal con

you don’t have talking at you it’s more of a group discussions

sessions are not planned in advance sessions uh depend on who is there

so if you have a particular expert there in a particular area that’s what you might want to talk about

um there’s no waiting in line for sessions or hoping that there’s still room if you didn’t sign up in time

usually these conferences are very small and you have the opportunity to meet a lot

of really cool people that’s what happened to me completely changed my career i think it’s been eight or nine years

now since the very first unconference i attended in jay crete in greece

a lot to choose from hopefully they’ll start popping up again second half of next year let’s keep our fingers crossed on that

one of the most important things and i’m going to skip over this very quickly but the biggest part about unconferences is

the set of rules that you follow whoever shows up are the right people whenever it starts as the right time

whatever happens is the only thing that could have wherever it happens is the right place

even if you’re like out swimming in the ocean which is pretty cool been in situations like that

um when it’s over it’s over so i like um more interaction um that

unconferences let you do using this rule set

all right so this talk about service meshes what i want you to get out of this is basically to understand the concepts

behind a service mesh learn the key differentiators i want you to help you get an idea of the types of

things that you might be concerned with and what to look for ultimately i want you to be able to

develop an educated opinion about whether this is really for you whether it’s applicable for your

infrastructure or your services whatever makes sense

this is not a super deep dive into integrating existing services with a service mesh or

differing deployment scenarios or anything but i’m going to provide you with a ton of resources to learn more

i encourage you to take advantage of them i know i’m not going to have enough time to show you everything that i want

to so you may end up with some homework

so how how did this happen and um there’s a reason i’m telling this story and i’ll explain it

in a bit um how i got to this point this is very simplified with cute little

simple graphics to make it easy um i was a developer i had recently been

moved on to a devops team so now i was all of a sudden instead of being in my little

silo behind my desk um just worried about coding features now i had to worry about things like

being on call which for a particular service at the time was quite a miserable experience because

now i had to be concerned with whether that service was running and reliable and in this

particular instance it was not so um from the ops perspective perspective what was

happening is we had this little service out there that was accepting a ton of requests and

normally on a normal day it was fine but as they started selling more and more um

of this service to customers those requests started growing exponentially and it got to the point

where that particular service began to misbehave

and we were missing some of the slas that we promised these customers so right away um you know we just looked

at the number of requests okay so more requests all of a sudden these services start falling over

in an ops position i mean what do you do the first thing you do is um you know you don’t want to call

the developer that’s expensive and especially me because i’m pretty cranky at night

so the first thing that they tried was to bounce it so they just you know

bounce it every time that it would fall over well this just repeatedly kept happening

so the next solution would be to scale it right i mean that’s the only other thing

left in their toolbox to do unfortunately replicating that service

and scaling it still didn’t solve the problem so it was time to call in a developer

and at the time i was the lead on this team so it was my responsibility to take problems like this and to triage them

to figure out you know what could be done if anything performance and reliability problems

like this super difficult if there isn’t good visibility within the system and in this particular case there wasn’t

um also a bug might be an architectural problem not necessarily

code um that’s exactly the case here

so since replicating didn’t solve the problem um the next step was to start looking at

logs unfortunately that was all that i had to work with i had raw logs well not

not completely raw at least they were being put into their location do some analysis

be able to figure out maybe what for production logs were not

bug [Music]

[Music]

um

you would take is make this problem smaller there was no way to simulate production

load but since scaling up didn’t seem to help the load itself wasn’t the problem so um

i went ahead and and made a smaller environment to look at

um one one option that uh is really nice to have is maybe like recording production

environment traffic in order to simulate that and test uh we didn’t have that tool

either so i had to figure out what kind of traffic would reproduce this problem and um

digging in i had to it was wasn’t only the number of requests but start looking at

the characteristics of the load what type of the requests were coming in

so it turned out i was able to figure out that um there there was a particular type of

request that was using up resources and such to the point that other api requests would

get dropped so how would you solve this um i am

at a microsoft microservices meetup group so i’m pretty sure you’re all shouting this answer out immediately but

um what we had was a tiny monolith of a service we needed to break up this service into

pieces and for each type of request scale those requests differently that’s exactly what we did

we did go ahead and we were able to break it up that was a it was a great move forward

but we had a problem our load balancer that was sitting in front of this

service or now services was a hardware load balancer

it wasn’t smart enough to be able to route specific requests to specific services and uh

for reasons that i won’t get into here we are also unable to change the end point that

these requests were coming in so they had to use the same endpoint but we needed to be able to look at the

the characteristics of the message and then route it appropriately based on that

um our load balancer was handled by a completely different infrastructure team um

so my this is my first time with this kind of a problem but i knew

this had to happen i knew there had to be a solution out there so um the very first thing is that you know

i went online and started looking around to see what i could find

well um we needed another layer we needed that way to route traffic at a higher

level than what our load balancer appliance would do we needed something operating on layer 7.

we also wanted to have control over managing it we didn’t want to have to hand this off to another infrastructure team this

and many of you already know this answer this was an api gateway and i as i was doing research i

stumbled upon a lot of solutions there were quite a few this was several years ago

but there were still a ton of available off-the-shelf options that i could use um i was just

looking for the simplest thing that worked the path of least resistance i grabbed one of these it was easy to

install it had the nice side effect of exposing me to a lot of different ways of filtering

traffic um different options for traffic management and routing that were new to me

as a developer because i was just never concerned with that side of things before

it also gave us the ability to handle like rolling upgrades and stuff we we now were able to

replace just part of the services and experiment with him in that way we were able to add rate

limiting to a few of these um the canary being able to add canary nodes was was pretty awesome

we weren’t able to do that in the past and then the blue green deployment deployment being able to you know

upgrade our services in groups it was it was pretty awesome um i didn’t have to coordinate any of this

with our infrastructure team and uh i was able to manage

these deploys uh you know within our team it gave us a lot of power and flexibility and it was great i was happy

well at some point during this research and to api gateways istio came up

i wasn’t familiar with it it was pretty young it didn’t meet my immediate needs at the time so i shelved

it um i fixed the immediate problem the fire was put out and we all just went on our way now there’s

an important reason that i shared this story with you um the number of tools and frameworks

that are out there are really overwhelming and um over 20 years i’ve gained a lot of experience with

a lot of different tools and frameworks some of them not even used anymore unfortunately today well fortunately in some cases

and i often get asked and i was someone that asked as a younger developer

how do you keep up how do you keep up with all this and how do you know what you need to learn and and what tools are out there that

you can benefit from and the most important thing that was shared to me and the beginning of my

career was always always step back and look at the problem that you’re trying to solve

this is how you accrue knowledge over time you always need that back story of something that you’re trying to solve

before you just jump into any tool in any framework just for the sake of it

that can lead you down some rabbit holes you may learn something and be excited about it but have no use for it so you

quickly forget it and then become obsolete in that knowledge right away because things

[Music] the

a little while later um we had a meetup group in denver and at that point uh racing who is a

google developer advocate he came out and he did a session on istio well i saw this come across my feet and

i’m like oh istio yeah i recognize that name um i’ve heard of it i want to go to this

thing i want to i want to learn uh what it’s about um so he was able to

show a lot of the features he demonstrated running istio on a kubernetes cluster

it had grown a lot and i was super excited about it so i had that serious

fear of missing out i wanted to understand what the difference was between what i had implemented

just the api gateway versus this service mesh magic um had i missed

something was there something better i could have done so that was my goal there um i went back

to the internet and i found this article by zach jory that explained pretty succinctly what

the difference was between api gateways and service meshes specifically and the key difference that

i noted was the um the traffic how how to handle the traffic gateways handle north south traffic

service meshes handle east west traffic the north-south traffic is

external client to service communication your east-west traffic is service to service communication

so there’s been a lot of development of service meshes over the last few years and that has resulted in

a lot of common functionality we’re probably going to see more of that as we move forward and maybe

these concepts will merge into a single solution in fact undoubtedly there are service meshes out

there right now that um have all of the same functionality as an api

gateway but for now i i found that the primary difference between the two was the

intent the intent was different with the type of traffic that was being managed

so knowing they were different i still didn’t really understand what it was

i went the api gateway that i found it was just a simple application that i was able to install and deploy

i configured it to point to my services but what a service mesh actually was it still you

know wasn’t quite gelling with me so i went out and looked for a bunch of

definitions and i’ll share quite a few with you just because i’m never satisfied with just one definition they

all must agree so here’s a good one

a service mesh is a dedicated infrastructure layer that controls service to service communication over a

network pretty simple makes sense um here’s a longer one kind of made me tired reading

it service mesh is a configurable low latency infrastructure layer designed to handle a high volume

of network-based inter-process communication among application infrastructure services using

application programming interfaces okay any definition that needs a breath in the middle of it is too long

this is a good one i appreciate tldr a service mesh is a dedicated infrastructure layer for making service

to service communication safe fast and reliable this one is from william morgan he

actually is with buoyant which is behind linker d the lingerid service mesh

he actually reached out to me and i’m asked if he could give me a demo and this was back before i had

given this talk at code one he wanted to make sure that i was giving linker d

um enough you know time in in my talk so uh he reached out gave me a demo

pretty awesome pretty awesome to watch um here’s another one a service mesh

brings security resiliency and visibility to service communications so developers don’t have to

i like that one anything you know that can help a developer out is pretty attractive this one gives

you kind of implies that a service mesh will just take care of everything for you sounds great now lastly this one’s from

istio’s website a simple explanation the term service mesh is used to describe the network of microservices

that make up such applications and the interactions between them

all good stuff uh real world analogy um it’s the best way for me to wrap my mind

around things um i find that the most successful inventions and innovations have been

ones that most closely resemble a naturally occurring process and uh here’s one

so how about the human circulatory system i find i think it’s a pretty good analogy um all the little boxes equal services

and all those little veins and arteries that’s controlling communication routing rate limiting even

this is pretty legit totally makes sense another good analogy i found recently is like our mail delivery system i don’t

have to worry about all the details of getting a letter from here to there i just you know put an address or and a stamp

on the envelope and stick it in the mailbox now that i had some good definitions i

started to do a bunch of reading and this is where we get into the marketing bit so here is a list of

uh service meshes promises that um i found that were pretty attractive to me so i

just started you know going down these lists um figuring out um which which had the

most value for our team and what would make the most sense to have

um the first one uh service discovery um especially in our cloud native

environments and stuff and having like auto scaling you may not know the ip address

of a service we no longer hard code those or host names even so having a way

to discover your services pretty valuable observability this is huge

no just knowing when your service is up or down or when it’s arrowing out or if it’s constantly restarting some

pretty important stuff rate limiting this could be valuable for subscription

services where you have different performance plans that are offered you can also use it to

prevent starvation circuit breaking so you don’t want one

service in your system failing or misbehaving to cause the entire system to fail

you want at least some of your requests to go through so you this is the intent here is to stop the

domino effect obviously if your service keeps breaking you’ve got other problems this is a last

resort but pretty important when you don’t want to bring an entire system down especially with

all of the interdependencies traffic shifting this was huge and this

is something that i got just from our api gateway being able to do canary deploys being

able to do blue green deploys i can put out a new version and only

route some traffic through it to see how it goes before switching out the entire thing pretty

valuable load balancing this is i mean really simple

obviously being able to balance your requests across your services is important um to keep you know one

service from getting all of them and falling over authorization and authentication this is

in the context of services speaking to each other by default um this must be secure

and why is this important well some services should only have access to some

things um i have kind of a funny story not really funny it’s actually pretty sad

um we had a production environment that was polluted by the test environment i’m sure none of

you have ever had that happen before but we had a situation where we had a production amq

broker it was running like it was supposed to cueing up tasks and services and um you know grabbing those tasks

like like it intended then someone started up a

broker on their local machine for testing but accidentally used a

production url um took us a while to figure out what was going on i mean jobs were

disappearing and we didn’t know where they were going if we had a service mesh

if we had it configured so that local and test environments can’t access prod that would have been nice you need to

protect those production resources that’s an accident waiting to happen

another sad story we had was with a database you know our testing database

a run of tests would delete that database and start over and uh that actually connected to a

production database and wiped the whole thing i these are these are very newbie problems but

there’s always a newbie on the team so you’ve got to figure out ways to protect yourself

from these problems because they happen they happen they’re to be expected so you need to put in those protections

you don’t need a service mesh to accomplish any of that but if we had had one and had it

configured appropriately that would have prevented these

distributed tracing being able to track a request from service to service especially if you have a lot of

interdependencies that’s pretty valuable you can measure how long it takes for a particular

service to process a request and send it off it’s also a pretty powerful troubleshooting

mechanism if you start at the beginning it fails at the end with distributed tracing you can figure out exactly where that

failure occurred uh this is this list is a pretty good list um there’s there’s a lot more out there

about around like reliability retry mechanisms but um this is good enough for us to

start with so at this point reading all this i’m super excited

i mean a service mesh is going to get me all of this this is awesome remember my original

architecture problem didn’t really have much to do with this

i mean there there’s some pieces and parts there that i could use right but um i kind of gotten a little bit of

the weeds here because we had some of these you know some of these problems i didn’t even

know we had at the time i was i bought i was bought you know

hook line and sinker i wanted all of it i wanted to play with all of it um i now cared more about these things

than i ever did in the past i couldn’t believe we didn’t already have a service mesh you know why on earth

didn’t we have one why didn’t everyone have one well how many of you have gotten you

know really excited about something and then you get your your bubble burst a little bit it

happens all the time it can happen with a senior engineer can happen with a peer

it could happen with your spouse it’s it’s not fun when you’re really excited about something

um and somebody kind of you know takes you down a few notches i’m getting used to it now maybe just

because i’m getting older i don’t know that a good solution is just don’t get excited about things

anymore but just recognize that and and remember how it feels and try

not to step on someone else’s excitement too much anyway uh this quote i came across

if you’re wondering about service mesh you don’t need one period if you’ve reached the scale and

microservice maturity level that requires a service mesh you will be actively perhaps desperately

searching for a solution and it will be abundantly obvious that a service mesh is necessary so this this was within an

article just talking about the benefits of of service mesh and it gave me pause

and then there was another quote in there by matt klein um he he you know

i think we have a tendency to chase the shiny object in the sense that x company does y

therefore i must do y even though i don’t have any of x company’s problems

so that goes back to you know going back to what problem is it that you’re actually trying to solve and and being efficient

with your research resources there so i’m sure some of you here maybe

have gone to conferences um i’ve i’ve done it um early on in my

career um i came back with these huge to-do lists um and a lot of them started out with well

ibm is doing this we should do this microsoft is doing this we should do this google’s doing this we need to do

this too we need to do it all and conference speakers are very good marketers i know this now

more than ever um when you’re coming up with problems like this and evaluating

your solutions always revisit the actual problem you’re trying to solve and don’t overkill it’s it’s just not

worth it so let’s go back to our originally really cool list

don’t we have solutions for a lot of this already there’s there’s a bunch of tools available already and i’m sure

um if you wanted to throw in chat tools that you use already to solve a lot of these problems

um there’s there’s at cd um there’s eureka there’s zookeeper for service discovery console

console is actually now uh can be categorized as a full-on service mesh with console connect

for observability you know we or telemetry we have stuff like prometheus grafana we’ve got libraries like statsd

to work with uh there’s commercial products out there datadog does an awesome distributed

tracing by the way it’s pretty neat and visualization of your microservices

cloudwatch we have proxies that can do some traffic shifting work

for us like nginx aha proxy these have been around a long time for logging you know we can get products

like splunk it’s pretty overwhelming a lot of these things overlap

so let’s go back to our original problem um turns out this matt klein guy who uh

i quoted earlier pretty important guy he developed envoy proxy and envoy proxy is the default sidecar

for the istio implementation of a service mesh i’ll explain a little bit more about that later i found this podcast

by matt klein he was being interviewed he described how he got into this situation

when you’ve been an engineer long enough you see a a cyclical repeat in your career if i

had to guess maybe every 10 years um we have monoliths oh it’s time to

split it up um too much we need to start bringing stuff back in we divide up responsibilities to

different services oh well well now we need all these responsibilities in one place so the team can work on them in the same code

base so you’ll constantly see this push pull of these ideas

maybe who knows maybe in 10 years we’ll start hearing talks on why why monoliths are better

matt colleen described how we got here we started with the monolith then we went to microservices

now all these pieces can be wildly different from each other they can even be different languages they can be um worked on by different

teams even this is not just a digital divide it’s it’s a whole a team structure management change

all of these pieces now have to communicate with each other and you now have all this extra traffic between the services

i now worry about security and all the other things inherent to service service

communication all of these changes happened over time we now have

smaller teams with autonomy over smaller pieces of code that’s an advantage of microservices

some of the biggest problems here are around observability and reliability we have different teams with different

code bases we have partial implementations of logging or like the logging is different

monitoring solutions are different you might have failures because one service is set up with circuit breaking but

another one is not so this is where service mesh comes in

this is some of the problems that service mesh tries to solve it is implemented with the sidecar

that’s where how most of them are it’s worth noting that linker d the first release of linker d

was actually it didn’t follow the sidecar per service pattern

it was written in scala and that first version it because of the

the consumption the memory consumption and things like that because of the jbm that was required it was often like

maybe one proxy per node so that was a little bit different but these days with the modern

the modern microservice or modern service mesh architectures you’re going to see a sidecar per

service this side car handles all of the communication to your service

the link id 2 is basically a rewrite that’s what’s out

right now a lot of features have been added to linker d2 and i have a good list for you to go

check that out in a minute um that did they even change the language from scala to

rust and go um the other part of a service mesh is

the control plane this is uh what configures the data plane collects metrics and specifies

authentication policies

so um if you want to get out there and get started with um service mesh learn how to do it

um these are the updated links to each of them i did include envoy here because i think it’s

important just to understand how the proxy itself works um especially since istio uses it other

service meshes use the envoy proxy um this is actually the envoy proxy that

istio uses it’s it’s their version of envoy linker d does not use envoy it uses its own that’s written in

rust and that might have a lot to do with um a different difference in performance between istio

and linguity

all right so some of the prereqs um as a developer i didn’t want to have like a whole environment to go out and try all this i just wanted to

set some stuff up on my own computer and i’m going to go quickly here because i think i’m running out of time

but um this is another reason why you might want to think again about

integrating a service mesh you need to know a little bit you need to have all these tools

docker docker compose i put docker compose in there because um envoy stuff envoy examples they use docker

compose you need some kubernetes basics understand the command line how that works you need to understand a little

bit about helm charts and templates um definitely get used to yaml if you haven’t been introduced to that yet

as a developer um you’re gonna need to make just you know some minor changes to how um

how docker runs on your machine i just used a docker for mac that it was enough for me to set up and

use all of the examples that istio provides and that linker d provides pretty awesome

if you don’t know anything about kubernetes or this is new to you you might want to do some of this

this is a link that i found provided by microsoft 50 days from zero to hero with kubernetes

uh right away even just that title lets you know how much effort and time you need to spend on this um each one this isn’t like a

full day of activity each one of these items is is just a short video or a short

exercise so that you don’t get too overwhelmed day one is my favorite it’s a children’s book

um i picked a hard copy of this up at kubecon the last time i was there

really fun really fun and if you go to phippydio

you can buy the books there you can get digital downloads of the book as well it’s pretty cute it describes uh a lot

of the uh puts into metaphor a lot of the kubernetes concepts

pretty cute here’s some pages here okay that last sentence there uh with uh cube cuddle

they they um have definitely it is now written down it is pronounced cube cuddle

that that made me chuckle when i got to that page um some of the metaphors are a little

bit forced so there is a little bit of you know technical explanation of what’s going on

some instructions to explain the concepts

so on to istio this is uh some of the description of istio and what it’s all about

um it’s written in go apache 2.0 license that’s important for

some um it’s backed by google red hat and ibm i’m noting that because this is a service

mesh you hear a lot of and i’m telling you the reason is because of the marketing they’ve got a big backing behind them

it is also very very complex there are a lot of features to choose

from a lot of toggles and switches you can shoot yourself in the foot if

you don’t pay attention there’s a lot of different deployment model options you’re going to

need to consider that they do have you know a lot of defaults for you

when i was just trying this out i just used all of the defaults i would highly recommend doing that i tried to do it a different way and

just you know spent a lot of time futzing around on my personal machine trying to get things to work so

definitely when you’re first starting to learn it you want to start at the the basic level and just solve one one

problem the community is pretty huge lots of information out there

they have lots of examples to work through that are are really valuable

the documentation is is really good this is a picture of their architecture

you can see here the you know the services with the um the proxies um their proxy is envoy

um you can technically plug in like an entirely different proxy if you wanted and and that’s something

that istio does is they focus on plugability quite a bit

all right linker d2 a lot of the same stuff i found they do support a lot of the same

features but they are more opinionated in some places

this is uh when i went to their documentation and set that up it’s super

easy i mean within minutes i had it up and running and was able to go through their examples um very very very cool they’re um

less modular but they’re you know they go for simplicity over complexity they’re not quite as

feature rich especially since they did the huge rewrite from the first version of lingardy which was

conduit or not conduit conduit actually became the second version of linkerity

um they are they have done a lot of work and most recently in their latest 2.9

release they added mtls which is on by default for all tcp it used to be just http so this is

this is good stuff they are doing a lot of work and catching up really fast

this is a picture of their architecture how they work with the control plane in the data plane

you can see you know some of the stuff like the telemetry stuff they’re a little bit opinionated on

this is a website i’m going to leave this for you for homework um kinvulk is a berlin-based company

that was asked by linkery to perform benchmark testing on both istio and linkery note that with

any service mesh solution latency is going to be introduced and this is um there’s an interesting

discussion here definitely go and look this over um they talk about their methodology

and and how you know they do things and there’s even a response from istio explaining why they might be

seeing some of these results depending on how sdo was configured so

definitely worth going and checking that out it’s not enough to say from this graph

for example oh you know linker d is definitely better than istio as far as performance

be careful with making a statement like that without understanding the methodology here

here’s a quick quick comparison there’s a lot the same between the two

uh main differences uh basically just around uh complexity

and how opinion dated they are um how difficult it is to set up there’s a really good comparison chart

on um d-zone i put a link to it here um the only thing uh that wasn’t quite

up to date was the fact that the latest version of linguity 2.9 supports mtls over all tcp connections

all right we’re almost done um i want to bring up the service mesh interface

so there are a ton of service meshes out there it’s gotten to the point where we now need a new abstraction layer and this is it this is

a team that has gotten together representatives from multiple companies

and they’ve created this spec i put a link to the spec here it um basically it is a standard

interface for service meshes on kubernetes it’s a basic feature set for the most common service master use cases

including traffic policy traffic telemetry and traffic management

it has the flexibility to support new ones over time and kind of you know allows the folks in

the ecosystem to innovate with new service mesh technologies

so conclusion when you’re deciding what to do make sure you’re making you’re choosing

a solution that addresses real problems for your system also consider

your code base are all of your service homogenous maybe you don’t need to draw out cross-cutting concerns

that a service mesh would because already you’re already sharing solid libraries there

the idea is that no code changes should be required but in some cases you do need to make code changes um headers may be needed to

uh in order to enable distributed tracing for example there may be a performance cost that you

don’t want to accrue depending on what industry you’re in um in my particular case we chose not

to implement a service mesh uh we just weren’t at the scale or the number of services where it made

sense where the effort made sense for us in one case it was good enough to

just you know put in a nginx proxy and and deal with some of our services that way

so always consider other things as well all right thank you very much um don’t

forget to sign up for your air pods and um let me know if you have any

questions you can reach out to me anytime i’m really sorry for going over i know this was a long talk that i tried to trim down a little

bit and wasn’t very successful um anyway i’m gonna pass it back to you

and uh yeah thank you melissa thank you very much for your talk

um are there any questions what’s your twitter what’s my twitter

that’s directed to you or something melissa j mckay let me see if i can type it in here

right any other questions

do i have a real world example where you used to smash successfully yes um we had a a team

that used uh istio for some other services they weren’t services that i personally worked on

what ended up happening though is we ended up designating an entire team to maintain that cluster

um you said that you don’t use service mesh can you go into detail on what you use

um for us we just had a few different services that we needed to load balance and rate limit things like that we used

nginx to do that for us that that was sufficient

um as far as you know other tools i did list some of the other tools that we used i mean we definitely you know plugged in

prometheus and grafana we did use um dog at one point that was really a

powerful solution i hate to push like a commercial solution at a community event though but

um it was it was pretty nice to be able to do the distributed tracing and to be able to

view the map of the services it was pretty nice

can the service mesh pattern be used without containers

so most of the service mesh right now are focused on kubernetes clusters um

istio does support vms um so i mean

the pattern itself um like what i described before you know the the idea of having this proxy that takes

care of your communication to your service i suppose you know you aren’t limited to only using

a container for that

are you still using java in 2021 do you plan to move so always an awesome question

um i know of a lot of teams that are still on java 8.

and that’s all that’s a big discussion especially in the java community how we can push folks forward especially

with uh cloud native infrastructures and stuff um java 8 isn’t the best

version to be wrapping in containers and such so java 11 in the first lts release that

fully supports containerization is java 11. there are still a ton of java developers

there’s still a ton of work um i have seen teams especially when they’re starting to break out microservices that

seems to be a huge deciding factor on whether to stick with java and and move to a later

version or whether to try something else and really what it has to do with is what type of resources you have

the developers that you have for these teams are they coming fresh out of college do they even have any experience

with java i mean would they much prefer you know node and npm things like that

it’s just going to depend on what resources you can pull from

so long answer yes i still use java in 2021.

how did you immediately solve the issue where you were having a thousand requests overload you said you end up splitting the

application to microservices the first couple of solutions was to um

have ops just keep booting those services keep bouncing them that lasted for

you know a while but then they replicated them adding more resources adding more

services and installing um you know more instances of those

and that works to a point but even in a kubernetes cluster you’re going to get to a point where even your auto scaling

you’ve reached your limits right you’ve got to figure out how to deal with those overloaded requests so ultimately we had

to split up those requests right away and by putting in the apa gateway and being able to

to route the request of a particular type over to another set of services

that were scaled differently that was huge