Mutation Testing – Start Hunting the Bugs @ London Java Community Meetup

let me introduce myself for those of you who don’t know my name is Abby I work with Barry Cranford who’s the founder of
the London Java community and I have the privilege of Hosting today’s event
um the London Java Community is just one of the communities that is run by recworks we are a tech recruitment
company with a massive massive difference so as a company we believe that recruitment really can be a force
for good in the industry Beyond just placing people in jobs we have a
particular focus on learning mentoring and personal development if people want
to learn and others want to teach we are happy to connect you through our communities and events we see this as
giving back to those we’ve worked with in the past but also paying it forward to those we hope to work with in the
future to date uh for the ljc we’ve run over 700 events for engineers developers
students graduates ctOS we have run conferences lightning talks hackathons
all sorts in January of this year we made our 4 000th introduction through
our meet and Mentor program and we’re really excited about that and seeing where that’s going to go next
I’ve already said we love to give opportunities for people to connect and this event is part of that if you would
like to know more about any of our other communities we currently have about 15. um please do get in touch with myself or
Barry or Helen Lewis if you know her and we can send you the link for the right groups that you’re interested in and
take it from there um the recording of today’s session will be on YouTube so I’ll be able to put the
link in the chat later and also you’ve got the option just message me and say
Hey can I have that video and I’ll send it to you as well um with no further Ado I’m going to hand
over to Harry who’s going to have a little word with us about jfrog and then
we’ll hear from Sven over to you Harry thank you so much Abby I really appreciate it hi everybody my name is
Ari Waller and I am the Meetup event manager for jfrog and we are so glad to
be a sponsor uh with the London Java Community this year just really appreciate the community efforts uh that
Barry and the whole team uh just put into uh making it special and uh really uh uniting actually the recruiting world
with the uh Meetup World which is not which is not an easy task uh because uh
you know giving back to the community is more important because recruiters are usually just about jobs but this shows a
different heart and a different approach which is really really awesome so kudos to Abby to you and the team for those of
you who are not familiar with jfrog we are the devops software company we’ve been around since 2008 and many people
including you may know us by our Flagship product artifactory which was originally written in Java and it’s
considered by most as the gold standard for managing your artifacts and dependencies jfrog has always been
committed to do the Java Community we developed a free Cloud version of jfrog
for you specifically to enjoy and it’s free to use for all of your projects or
just to play around with if you’re getting if you’re getting your feet wet and devops a little bit no credit card
is required at all and it’s not a trial version that runs out it’s one that if you continue to use it you can continue
to use it as long as you like as long as it meets your needs I’ll drop the link into the chat shortly but something else
I’d like to share if I can share my screen really quick is a really cool offering
um in the spirit of community happiness which is one of the uh virtues of jfrog
uh that I get to live and breathe on a daily basis we have can everyone see my slide okay
great uh we have a um a raffle just for attendees today jfrog is going to give
you a chance to enter and win a pair of airpod Pros um it’s going to be given to somebody who is here today or tonight rather
um and um all you have to do is enter with the QR code or the bitly link which will also drop in the chat shortly now
because it’s the internet I can’t do live drawings and sweepstakes it’s a it’s a compliance thing however a winner
will be randomly selected within two business days of the Meetup and then once the winner formally accepts the
prize we will share their name with the entire Meetup community and uh celebrate
with them but uh again thank you and without any further Ado I’m going to turn it over to Savannah Rupert who’s
going to be talking to us about mutation testing hunting the bugs
hello hello so thanks for the introduction and now we
want to start to talk a little bit more about the technical things and today yeah talking about hunting books and uh
this was quotations so um not the critical task I have to share my
screen so that they can start what
so hopefully you can see my screener I’m now searching for the chat window
because it’s always disappearing if I’m starting to share something and
let me see all in the meeting Okay so by the way if you have any questions
recommendations or whatever during the talk um please just use the chat window don’t
use this q a thing I’m obviously just looking at this chat window
and if I if I’m at the right time in my talk then I will keep immediately an answer maybe
a little bit later but um yeah I try to give answers to everything what I’m seeing in the chat
and that is more or less related to The Talk and uh yeah take feel free to the
chat please so um I’ll teach you testing
by the way my name is Ben I’m from Germany and uh as you can see a
picture uh every three minutes on spinning Outdoors somewhere in the woods
and uh since a big pandemic I was a pleasure to combine my hobby with my
with my business or was my job because I thought um doing all my recorded talks out in
the books if you want to see something about Java or you have Sega or cyber security topics then it would be a
pleasure for me to see you on my YouTube channel it’s a tiny one but yeah I’m
still working on it so if you want to see some outdoor style article talks check out my YouTube channel and I
really would like to see you as many subscribe okay good no we’re talking about rotation testing
rotation testing I I want to go through several points here and
um the basic thing is more or less why I quite this talk this talk was created
um out of pain so yeah I I was head of an r d department and uh I had to to work on a
product a legacy product nothing fancy uh startup six months old thing with the
latest Technologies it was really an old Legacy project and it was way over 10
years old and um yeah it it was a formerly product that was converted from
PHP based Java and all the Developers or the most developers that worked on
this project were still from the first team all converted Java developers from
the PHP world and they learned Java inside this project with this project during this project and it was a product
that was hold to the companies and yeah well but the cool
thing was it that was really zero test coverage so during the time I jumped
into the project it was my task to clean it up make it new make it fancy and all
this stuff and um yeah so there was really zero line of
test coverage and there was no dedicated refactoring budget so I and they were
just paid for features and not for cleaning up and all this stuff I I mean the most of you know what what arguments
and discussions you can have with some people in your company and um for sure
during the 10 years of Learning and Development the complexity of this piece
of code was just amazing really amazing so uh for me it was it was more or less
a few questions I had say what what’s the right place to start inside this
project um to get an overview where where the biggest risk is this huge crude base and
um one of the big things was how to motivate this team to to change
something to work on it it makes their life easier I have to to say that this team was
I think calculate 70 or 75 percent of the
developers in this team where in their last 10 years before they are
retired so it was really a dream to start in this project and um yeah so
um I had a bunch of experts there and I haven’t I’ve been test for you for you
by yourself so please grab a piece of paper and a pencil and then
give me give me answers to these two questions this is a typical piece of business
coach why because it’s completely random there’s no
um rule no mathematical background that you can use to verify this okay it’s just
some piece of business code and um it’s a small part and you all have to
do with with code reviews and then you have way more complex code I hope and
that you’re looking at this code and then you have to decide quite fast uh is the test coverage good how many tests
you need and what are the right input values of this piece of code
so just a test for yourself we will go through this example a little bit later
um try write down just how many tests you need because we need this amount for
form some calculation and then just write down the pairs of input values you would
use and I guess a lot of you are quite seniors so it should be an easy task and
just write it down um I mean you have for this amount of code maybe a minute maybe two
uh to your code review you will have way more code in way less time so yeah write
it down um give give me a hands up if if you have done it so that I can just see that
that it’s done more or less or that if you have done and uh in terms of time I
well let’s start yeah working a little bit so please
rep a minute write it down and just give me hands up if you had
a hands up if you don’t want to do it then just signaling okay
I can’t see something okay okay first one I got the first one
okay all right I gotta that’ll be okay so let’s let’s work on that
um I mean why why do we need this what what what is the importance of of this
of these numbers uh why you should extract this uh so fast
um we if you have a test coverage we we
have to find out um what has coverage how much how much
tests are enough to test this piece of code okay and uh in the same way we have
to identify is this a good value okay so we we want to guarantee a constant quality of of
this test testing environment or this test coverage so how to find out what will be enough so how many tests I need
to to be on the safe side and how to find out what what are the right place
um and what does it mean so in this at every day so on Monday morning on Friday in the middle of the night uh if if you
have challenges at home if you are not feeling well if you are highly motivated whatever we we have to to deliver
constant quality and this is something okay so I think everybody had enough
time to write his own examples down or the secret values and demand and so I I
started for this piece of code with with these two tests so just um zero zero and three and zero and
it says good this is right this is something we can improve this would be
the first question uh or we we have to find an answer to this so before we are
going through this example I want to talk about the basic idea behind mutation testing so mutation testing
itself is something that is a structural testing method it means um you try to make sure that the tests
are fitting as best against your code it is not a test method that will help you
to check if the requirements are got right or whatever it’s just the
combination between the the test code and the the code you want to test and the test should be as padded down to
this source code as as possible it’s it’s a way to find good tests technically good tests we are not
talking about the use case or um if if the use case makes sense really just we’re talking about the technical aspect
so and the good thing is that the machine will help you to find these good
tests and this is called mutations so what what is limitation a mutation is
a change inside your own code and it’s small enough so that’s the
defect what the defect is we’re talking in a few minutes ago so it it is changed
so you have the original program and then you have it program two
and this is with one small change compared to the origin okay so we have p
is a program with physics test collection so the amount of all tests we have so the minimum that we need is one
test okay even if a one test I can start with mutation test and then I need a
sequence of modified programs and this is called one two and so on so and in
every notation I have just one notation okay I I have
never two defects in one program I’ve always just one single rotation and then
I’m testing against this single mutation what’s good what’s bad so if Green is
if at least one test will fail okay from a test Suite red is if all tests agree
what does it mean if you’re changing your program from P to P1 with Once in a
defect and you let your test Suite run and all tests are green
the result is that you’re not able to decide if you’re delivering be your P1 or P2 or P3 okay so you’re not able to
identify that there is a change and the test coverage should be so near
so totally bound to your source code that even the smallest change could be
detected okay so the optimum would be that I’m killing 100 of all mutations
okay this is an Ideal World the same with all other test coverage we’re
mostly not reaching 100 but this is the ultimate goal we try to reach so as many uh mutations as possible to take now the
big question is how should I create all these different mutations and what mutations are here say what what can we
do here so the the general workflow is quite easy I am generating all the mutations
then I’m running all tests and then I’m checking reports and this is more or
less a classical tdd process so I don’t have to to change anything I don’t have
to to do separate steps I’m just doing what I’m mostly doing with the tiny
add-on that I decide now it’s time for augmentation testing please generate all the mutations okay but I’m using exactly
the same paths I’m using junit um what what I’m using here so and this
the cycle is that after checking the report I’m just changing the test writing more or removing tests so it’s
really until my quality gate is reached so it’s it’s no big change in your process
um okay if if I want to compare or the first thing what I can do if if I’m now
able to identify how many mutations I have in 1000 lines of code or
dedicated amount of lines of code what I can do here I I can use this as some
some Metric assume that I have three modules and in this modules I have a
different amount of rotations possible notations what I can do is that I can
use this as a complexity metric or some kind of complexity metric and see if I have a module with hundreds
of possible notations and Alpha model was just 10 possible amount of notations then this module was 100 amount of
possible notations and it is by far more complex we’ll see why it is like this
but if if I have this one I can use even if I’m not even killing one rotation if
I’m just generating the report of mutations I can decide now in different
modules where is the biggest technical risk in terms of amount of rotation and
this is mostly a direct indicator to complexity or not well written code okay
so um if you have this just have in mind that that you can start structuring so
if you have different modules focus on whether maest mutations possible mutations and then you have even a
strict order between modules but what what is a mutation and what are
different motivational groups language depends language it depends so what what can be a notation for
example we have value mutations the limitations is that inside a loop
boundary for example I’m changing the the number run until three run until four without
Now using some language constructed it’s more or less language independent because all languages with loops can
have this mutation changing constants I can just change them from -10 to -1 or
to whatever I have decision notations decision mutations are smaller to smaller equal
so logical operators that I’m just changing okay but I’m negation on
something like this the same uh and and what I have that is language independent
is something like statement notation it means that I’m swapping deleting lines of code so it’s something that you have
sometimes from refactoring or other stuff you’re changing lots of code or duplicating things or whatever so on
this you can do and this is called this imitation okay the limitation decision mutation statement mutation and
um these are language independations because you can do it with following honey and then you have language
specific notations and now we are talking about more Java specific things by the way the the most stuff I’m
talking here saying is for kotlin as well or other GBM languages and um yeah
we will see how to use it so for Java uh think about changing modified or you can
delete uh initialization of members or changing between static non-static
normal stuff um what’s really nice and what amazing
people are not thinking about is for example my my favorite one is argument order change
I had um oh I had hands of clean code in this team clean coat was everything but
clean coat was a little bit misunderstood at this point so I had methods for example with a signature
um method name and then the arguments were integer a integer C into integer B
and they had the same range of possible values and then clean code was very nice
and they sold okay changing the arguments would be interesting because ACB is not good to read I’m changing it
now to a b and c great it was really great so finding
these parts was was paid for free time so all these are Java specific computations uh depending
on implementation testing framework you have more specific ones and sometimes you can create your owns but this is
more or less the idea behind it so you have source code and then you’re changing the tiny bits and pieces this
this code creating the new version of the program and then let the source code
uh the test Suite run against exactly this modified program so
uh quotation testing as I mentioned is at on to the normal regular pdds though
so if you’re desperately have the test print development you’re on on a very good track and then you can just add
rotation testing on it because it’s just net on it’s not a change
um you will see that we have tool that is supporting it very well and um there’s a downside of it I will show
you examples of data in terms of time uh generating and running all tests are
very time consuming so it means um if if you have an 100 000 lines of
code project it’s not a good idea to start mutation testing on your CI environment on Monday morning of all
packages with all mutations will just not end okay so start in in tiny modules
and see what is the runtime behavior for your project before you’re just using
protection testing for the whole project okay you can do it on a dedicated machine but not but if you have just one
CI environment with one agent or something like this so have in mind that
the runtime augmentation testing is something exponential with the amount of lines of code and the possible amount of
well not the amount of lines of code more or less the amount of possible notations okay so but uh mutation
testing will definitely affect your project structure if you start working with this so um okay we we have this
mutation term we we have now the the different rotations that can be done
I’ve shown a few examples so what what is a church for example in Java to to work on this and what Frameworks we we
have here so there’s an old framework that is just for research purpose never
never use it okay please don’t use it it’s it’s end of life don’t use it but
if you want to learn more about how to write quotation testing tools it’s a
good thing to check so um I thought that a lot of projects started around 2003
2004 um I guess it was fancy at conference to talk about it was uh I think a
sponsored by companies research institutions whatever so a lot of
Frameworks started 2003 but the basically mathematically Basics behind rotation testing is from the 70s so the
theoretical part is very old in computer science ages but in Java it started 2003
then a lot of Frameworks started and they all died around 2008.
seven eight nine so in this age the Mouse project night I don’t know maybe
they are all having the same results or whatever and this framework what I’m showing here uh had the approach that
they stopped modifying the source code and then tried to compile and let it run
and the challenge here is if if you’re creating rotation and this will break
the compile process um you you have to decide is this now an error that is done by wrong syntax or
Isis and runtime error or what kind of error it is and this is sometimes not so
easy so it means rotation testing you can use most for the most languages from
the theoretical point of view but if you have languages that you have to compile first before you can run then it’s not
very effective so for C C plus plus but I know motivation testing is for example used for PHP as well and
inside such area so this project is still available so in terms of source code and then it can have a view on how
to decide how to create mutations which is interesting and the good mutation test framework and the quality you will
see um in in the way how many mutations that are repetitive or overlapping are just
not executed because this is a quality smartphone formation testing but here’s
another framework that is very active and um they started around 2012 if I’m involved
right and it started um as a parallel integration test framework so this is
where the name time is coming from PI test it’s an open source framework uh the main code of the inventor of this
project um got his Java Champion for this um I don’t know two three years ago I’m not
sure about it but but it’s Java Champion for this and since 2014 is very active so if you have a new Java version
um then it’s very fast adapted to the new bytecode if we call the big change here compared to the other project is
that this sort of project is working on bytecode so it’s it’s getting rid of
this compiled context in terms uh or in in a way that it’s just changing the
bytecode and then let it run and this is a very good approach for Java or all jvm
based languages and we will see that we can run it on Cotton as well and so so uh Pi test um
comes from the parliament integration testing that means uh the the project was very strong in isolating class
loaders ramping up different processes uh let stuff run uh check if the process
is still active in all this and this helps a lot with that mutations run concurrently and yeah
it’s it’s very very cool so looking at it how it looks for you as a
developer we’re going back to this example so we have this method with two ends with just
a few logical operations and now the question is how many tests do I need to
test this piece of code if you are thinking about what is the total amount of mutations here what are the pieces
that can be changed in what way and how to isolate this one with one single test
so now the situation is um what how many tests I need and what are
the right input values same if you’re talking about line coverage Pure Line coverage
then I need exactly two tests why
firstly people are looking at the source code and this is a good advice the first thing what you have to see is how many
paths you have through this source code the maximum dominant part here and this
first code is this if construction and this related to two pauses so you see okay I have two possible pulses so at
least for a hundred percent line coverage I need at least two tests if I have just one test I’m below this border
to my test coverage is by definition not good why should I think about the
minimum amount of needed tests if I have the rule that every test will cover just
one case I need to understand immediately how many possible paths in this piece of
code how many tests I have and this is the lowest barrier I need if I have less
tests I’m below the minimum quality okay that’s it so the next thing is hot can I
identify the upper border so what is the amount of tests that are needed without
yeah writing too many tests I saw for example for this example
um I I saw sometimes that people are writing 30 or 40 tests to test different
things and imagine that if you would have this this
amount of tests for this tiny piece of code and this in a bigger scale for project it would create a way more
source code as your original program it’s okay but you have to maintain all this stuff so it’s not good if you’re
just adding tests if you’re just constantly adding tests just imagine if you want to switch from an junit version
to the next one and you have to migrate your test if you have to migrate to new language features behaviors whatever so
make sure that you are writing the good amount of tests but create this test code as it is your production code so
test code is your production code and with this you’re producing the production code for your customer okay so
um you need something to to identify how to remove this test so let’s start with
these two tests zero zero and three zero so what what is what is good what is bad
if you’re talking about line coverage that I will get with your Coco by the
way yakuka is a very good tool it’s best tool for line coverage I know including brand coverage I will go we got um I
will get with this test here with zero zero exactly this report what I see is this diamond so with pure land coverage
I would just see that six eight and nine is green the rest is maybe white or red
is brand coverage I will get at least the idea that I’m not covering All
Passes okay but that’s it if I’m adding the second test three and zero I have a
hundred percent green band coverage report okay
but with this test I can do a lot of that now looking at this mutation testing stuff
um I’m heading now to this Pi test
configuration I’m just using the default purchase configuration I have this conditional boundary increments return
value mathematical operators and so on and so on a bunch of rotators you can
switch up and down if you want to add more or want to do and that was made I
created this Pi test mutation coverage tool so what what it’s saying to me is it’s generated
54 theoretical notations for this tiny piece of code okay we killed exactly
three wow that’s that’s amazing so but talking about the quality of
mutation testing Frameworks from the theoretical amount of mutations reduce
it to effective mutations and this means that this huge amount of notations
theoretical notations it was reduced to four possible notations so a good
mutation test Frameworks are excellent in reducing the theoretical pump to the amount of mutations with a really active
rotation Zone so what what I see here is that my line
coverage will show Green okay and my rotation test coverage says okay
I have three notations out of seven active mutations so you will get this
HTML report out of five test and then you have this class name and you have these two metrics and if you’re clicking
on this class name this service the Java it will jump insert this report to this detailed View and then you see per line
different things for example you will see that line number eight will have two
rotations in this one and what kind of notation you have for example here you
have change conditional boundary and negative conditional so these are the two possible notations that are
identified with configured materials for this line number eight and I killed
exactly with my test the second notation so if I have this one what I’m doing now
is I’m looking at this report then I identify okay what what I want to kill us next then I’m focusing on one
notation I will want to kill and I’m changing a test or I’m adding a test in
this case I just switched from the values three rule 3 and 0 to 2 and 0 because I want
to go with my values as near as possible to this logical operator smaller equal or smaller so
that if this is changed immediately it would pay okay so now line number eight
is green both mutations are killed the next is I’m removing zero zero and
adding the test one one because I’m killing the mutation number one in line
number nine okay replace integer is subtraction okay so the next one would
uh be killing the input value is zero to uh two and zero and now I have the pair
one one and TT and with this I am killing
line number eleven replace integer with subtraction okay check now I have two tests with two
input values definitely different than I what I had before doing the tunnel was working just with
land coverage and now I’m looking at this one you will get mathematician testing immediately or you will get some
points like oh here’s the rotation I can’t kill here the mutation I come till is multiplied by minus one because
divided by -1 and multiplied by minus one is the same result so you will get here to this Pawn that you can’t kill
this mutation uh you will have more complex examples in in your business code later on but you have to make
there’s always the decision how to deal with it and you have different possibilities first of all you can just
ignore it the other possibility is that you try to rewrite your code so mathematation
testing immediately you start writing easier structured code to get rid of
this not reachable notations Okay so
what we learned here was more or less this is a piece of code from this Legacy project
with everything what we learned so far how many tests I need for this piece of
code this was a utility function and it was spread of the whole project sorry I I got along to to use this piece
of code for for demonstration so I I took this piece of code and I took uh
one of his old engineers and told them okay write the best source code again uh
test coverage against it and we want to refactor this piece of code to to understand what’s going on and make it
more simple okay so how many tests I need what are the right input values for this test
um well if you want to refactor this one you have different possibilities you can just prefect her because you know
exactly what this should be doing okay well basically it’s not the case huh and then you start
modifying the source code in the end what you need is the semantic equivalent um implementation of the original code
if you just want to replace it with a more cleaner code so you can use two implementations let it run forever
against each other and check if there’s any difference well on the other side
the best thing your best safety belt here is just a good strong test coverage
so what’s mutation testing and this piece of code we were able to
um okay with this piece of code this isolated piece of code we we have done a Brute Force attack which was these two
implementations we just let those run with identity how many input values against each other to compare after the
test phase and refactoring phase if it is the same actually what you’re not really doing in production but for this
training purpose is possible so but in the end writing again this one the best
test coverage you can and then rewrite this source code and this was a result so this is the same
functionality in a different piece of code cleaned up so that you are able to
kill the most mutations here and if you’re comparing various you will see
that the complexity in terms of if and else and all this subfold
strategies here they they are way easier so the version one was polo
with if and if and Cascade if stuff and the second version was one if else one
for loop with one if inside and two additional if else independent if else
structures so what you can see was rotation testing is if you start with
mutation testing you start writing a simpler code with the same behavior and
this will lead you to the limitation test coverage with strong rotation test coverage or test coverage in general
easier to maintain code and this is done by a developer because he’s just want to
get a good test coverage okay it’s not a definition because you have to or whatever
see rotation tests are often leading to cleaner code compared to junit only or
line coverage and it will affect the structure so people the developers start writing smaller modules this is more or
less based on the exponential runtime Behavior mutation testing so if they are
lazy if they’re just making shorter modules so that your working class and there’s one interesting thing if you’re
looking at this piece of code it’s a typical piece of code and normally you
don’t have this red green highlighting but here is useless code in this piece
of code is there’s a line that’s completely useless so what what it is so how to find it
um well if you’re looking at this code maybe you will count it maybe not because it’s a little bit more
far from each other away then then you’re not able to see it straight but with mutation testing we’re able to find
it quite fast if you’re looking at this sorry I’m just zooming in to line number 18 answer
quotation testing will say to you you’re not able to kill transportations why sure it makes no sense so asking if
something is bigger than zero and then iterating of it makes no sense so what what you will find is more or less that
courage that is just inside your application so it will not only help you
to make it cleaner or more structured will help you to identify useless Parts as well sometimes small sometimes less
but it’s nice nicer two talking about all this why why why is
motivation testing helping and what was a good thing if I’m talking or checking
um what what I’m doing as a developer I’m writing my own code and then I’m adding a bunch of dependencies okay this
is what I’m normally do I’m adding a dependency I’m I’m adding the dependency of a PDF Library instead of writing a
PDF Library by myself if I need it basically I’m doing overall tickets but we are focusing now on on the
application layer okay I have two things that can hurt me and
this is compliance issues with dependencies okay wrong license or they change license and I have to remove it I
have to replace it with a semantic equal implementation I need a very strong test coverage that’s supporting it if I want
to get rid of vulnerabilities it’s more or less a daily job for us because we
have found new vulnerabilities I have to change dependencies now I need a very strong test coverage so that I can
change my dependencies not only the top level dependencies even um transient dependencies or indirect
dependencies because then I there’s a whole test we’d run if everything is green it’s not so as an example if if
you’re inside your ID you see that my pencil will have some vulnerabilities and I can’t change to the new version
because it’s not supported so far vulnerability is not clear as well whatever then I can kill maybe vulnerabilities if
I’m working on the vulnerabilities or the 20s that are indirectly used and
then I need very strong test coverages okay so that means even if my left side
of the security or cyber security related talks I’m mentioning that rotation testing is one way to have a
very efficient dependency management because this dependent management or this test
against functionality you are really using it’s not only how to test my code
what I’m writing but you have in the same way the possibility to change the
dependencies the composition of the same dependencies with different versions in a very effective way and this is mostly
helping killing vulnerabilities and and compliance issues as well so good test
coverage is not only for testing your own code a strong test coverage is good
for a lot of different things so quality and security for example going really hand in hand here at this point
so how can you start in your own project with this
starting a project here if you have a unit and you want to start you need the
Spy test plugin inside your Maven definition you are just adding it to your project
the configuration I will show you is just a copy paste it’s really easy and change your creating the reference in a
clean install or clean verify it depends all are good here your load check it is so
um and then you need something with zero red test okay so uh mutation testing is
not possible or it’s can’t be used if you have failing tests okay so you need
a green test suite for the origin program and then you’re running this uh Target
mutation coverage and then you will find the report on the toilet that reports uh
you can have it in XML and HTML and so it means you can use it for your daily if it is done by the CR environment or
you can feed other reports or tools for this the configuration itself is just copy paste we will see on the op-ed from
this tool that they have exactly this Snippets in it once the Declaration of the plugin and then the plugin of this
reporting the only thing what you have to do is to decide XML and or HTML and
you have to Define your target class your production classes and your test
classes okay check reporting plugin that’s it then
start with some tests we need just one test then your generator is report and
you start writing the next test and that’s it and if you have time you can use mutation testing in the same way
that you try to eliminate useless tests that are overlapping from the values okay
okay and this step by step about runtime um I have a hand project for myself and
it’s Dynamic dependency injection framework I use it for for some some projects and I started with mutation
testing during the time I started talking about it and here what you can see is that the junit runtime is nearly
40 seconds okay not too much but with mutation testing I have three and a half
minutes of same code base and we are talking a few kilobyte of good so really make sure that you try to minimize the
amount of uh of the scope during the first time you’re running rotation testing so that you’re not
running an entrance time so mostly it’s in a way like source code you’re working on yourself so this tiny one you’re
working with material testing on this one and then in a regular Cadence uh you can use the CIS system to do this long
running rotation testing of the big project or bigger part of the project
I see that sometimes people start comparing different line coverage to it
just makes no sense so you will find different results somehow for example if
you’re comparing from um the cocoa and from PI test or whatever tool you’re using different
line coverage then it can be that you have different numbers don’t try to make
them equal don’t calculate decide if you want to use LINE coverage or if you want to use mutation coverage and then just
use this one so don’t try to compare this metrics against each other
here in a Java class I just created this one and I created as a theme in kotlin
so object oriented version in kotlin what you can see is that Pi test is working on those so Java side as well as
some kotlin so you can use it for transition for migration or for mixed project as well because it’s very very
convenient so here’s the output of the Java class we’ll complain this integral
multiplication with division division and the same is complaint in the kotlin
code it’s a little bit different in terms of lines that are complaining or at what place the mutation is possible
but you will get more or less same result on the same report so this is very convenient because you can use it
for different tools as well so um
and what Leslie done was the topics I want to explain it’s a short overview I
really can recommend going to pytest.org to to see uh how it works how to
configure it in a very straightforward thing and the initial effort to
try mutation testing mostly is just a few minutes because you have adding copy
paste something to a maple file then you just let it run so that’s it if you have
any questions I checking in the chat no questions so far but if you want you can
connect to me later by Twitter LinkedIn I really would like to see you on my
YouTube channel for sure and the only thing that is really bad is email email
is just not working for me okay so then I always stop sharing my screen
I’m done fantastic thank you Sven that was brilliant love it I love it does anybody
have any questions
there is one that’s just come up in there yeah yeah Simon say what was language
specification changing public to private safety economic uh compilation will fail
so uh the thing um I test is not compiling your source code it’s
just modifying bytecode and it could be that just crashing yes but then you are
you’re killing a rotation because it’s it’s crashing okay but it’s definitely
because the budget is or the the result is not runable that’s it okay so it’s a
big difference if if you try to multitask host code and then doing the compiled processing instead of just
modifying spot code I hope I could give you uh one thing so
I’m sure I would like have liked to have seen a comparison with using switch instead
yeah so I keep it simple I keep it short in this talk so that everybody from just
starting with Java up to an expert is able to follow
so sorry for having this very reduced in terms of complexity of reduced examples
but um just found out it’s it’s easy and the report is giving you the line numbers
and all this stuff so and then it it depends on what Java language you’re using so what what are the features how
to do records and all that stuff but in the end uh High test is following up the bytecode very costly mostly if you have
a new jdk if it is done already or just in a few days mostly it’s adopted and this is
really cool so if if you need this tool okay
um are there any IDE integration following the limitation Loop yes so frontality there is a plugin available
I didn’t know the name because I’m so lazy I’m just using material thing because I’m just saying generate this
one and then if my browser was mine at gmail but I think there was more
IntelliJ at least plug-in it was working with j15 giving you some kind of line
coverage indicator and all this stuff so
okay integration with IntelliJ uh yes so uh integration is more or less it’s it’s
based on your build system so if you’re using gradual maiden or whatever it’s just part of the definition it’s just
the target you’re running so your integration is there uh what kind of integration we could have maybe
something like a highlight if this is covered or not covered I’m not sure if this plugin is doing it right now
because I’m not using it so sorry it’s um but I think someone will work on this
topic for sure and if you found something please let me know just mention you go on Twitter or whatever and paying it back please
so any other questions no question so far
no I just know okay okay well all that’s left for me to do then is to say uh if
you want the YouTube video have a look um probably early next week it should be
on our YouTube channel or message me directly on LinkedIn and I’ll be able to get that sent to you thank you so much
then for sharing what you know and thank you so much for uh sponsoring us as well jfrog it’s always always been a pleasure
so far and it’s always great to hear you speak as well Sven and thank you to
everybody who’s joined us today and we look forward to seeing you at our next ljc event thank you very much have a
great evening take care all right thanks guys