Conan Package Manager for C++ in Practice @ Core C++ (Mevasserat Ziyyon, Israel) Meetup

well yes thank you adi and inball and
everyone for having us today
and thank you everyone else here for
attending my name is jerry wilsey and
i’m a developer
on the conan team at jfrog for today’s
talk i’ll be
providing some high level explanations
about the fundamentals of conan
but also going through some actual code
and live exercises
to reflect what working with conan is
like in practice
the link in the qr code shown here on
this slide will take you to
all the resources for this talk the
slides will be posted there and links to
the video eventually once the video has
been posted
you can also rate this presentation and
enter a raffle to win
a conan t-shirt so we will now post the
link
in the zoom chat as well for those
interested because i won’t stay on the
slide
very long and but yeah that could be
we would love to have your feedback and
we would love to send you a t-shirt
we also want to share uh with you the
dates of another event where i’ll be
speaking about conan
that is jfrog’s annual devops conference
called swampa
so we’ll be holding the event online
this year and running it twice
one’s for the us and once for europe and
middle eastern and african time zones
and those dates are shown here for emea
also ari has some additional information
to share
about swamp up and some great great tips
there so ari please
go ahead if you like
if you are ready yup no i appreciate
that thanks uh thanks jerry just uh very
briefly uh
i’m not sure how many of you have come
to swamp up before but
it is an amazing global devops
conference and
uh we’re gonna be doing a virtual this
year which will certainly make it
accessible to so many more people
um on june the second which is the main
conference day
um that is a free pass day but we also
wanted to do something special for the
meetup community as well
so what we’ve done is i put some
instructions in the chat
but we have a deep dive technical
tutorial day
that’s going to be taking place on june
the 3rd that is a there is a cost
involved
however uh for those in the meetup
community
um that uh we’re visiting uh before the
conference
i’ve just given you instructions in the
chat on how to get that day for free
so uh feel free to if you click that
link the code is already there for you
all you have to do is go to the
registration process and um
do what i said there in terms of
selecting um the
registration um correctly and you’re
going to get that uh
49 extra day uh for free so
enjoy it thanks jerry awesome great tips
thank you all right that’s a great
resource
um okay so uh we can move on to the
exercises now the fun part
for these exercises i have created a
reproducible environment
which contains all the files for all the
examples and i will use it to run the
commands
the goal of this environment is that
anyone can clone and run the repository
or clone the repository and run the
exercises rather
on their own on their own machine the
commands are all contained
in the pdf of this presentation so that
should be all you need to get started
of note it is based on docker so you
will need to have docker installed
but docker does make it extremely easy
to do this sort of thing so that’s why
it was chosen
as long as you have docker it should
work on windows mac or linux
so if you try things out and you have
any difficulties with
with the demo on your local machine
please feel free to open a github issue
on that repository
now word warning
you know this is live demo so when you
whenever you do live coding demos and
exercises especially involving docker
uh things can go awry and they have gone
uh poorly in the past so if we have
technical difficulties
we apologize in advance and uh and we
will continue to
try to work those out over time so thank
you in advance for your patience
so i’m going to bring up a
a session now a terminal
and i want to make sure you all can see
that is that good
so now i’m going to run the commands
that i showed on the previous
slide and we are all going to
do this adventure together and and both
prove that it works and there’s no
slidewear
and uh you know prove that uh
that it can be done by humans so this is
going to run
and take perhaps a little bit of time
nope
and here we will run so that built the
docker images it will take a little
longer on your machines potentially
if you run this locally but it isn’t
that long even in the worst case
so this has given us a terminal a linux
virtual machine to run some conan
commands and to run the conan client
or a docker container rather and also
another docker container which is
running
artifactory which will host our
repository our conan repository that
we’ll use
all right there’s a question if does
this exercise
is it supported on windows yes it should
work on windows linux and mac
so as long as you have docker and can
run docker you can run these exercises
so before i start doing the in-depth
part of the examples though i do want to
provide a brief introduction to conan
if you don’t know already conan is a
package manager for c and c
plus it is open source and is published
with an mit license
it is also multi-platform meaning it can
run on linux windows and mac os
and actually other places anywhere
python can run basically so
various flavors of unix can run it and
are known to do so
so the support is very broad conan is
designed to support
all of the popular build systems in cnc
plus plus
as well as custom build systems which is
not an uncommon occurrence in enterprise
software development
conan is stable which means that there
are no breaking changes within major
versions
and that is to say that packages built
for conan 1.0
will work for any future versions of
conan 1.0 or 1.x
and if there’s a breaking change to
conan or to the packages it won’t be
until conan 2.x
is released so it is it does uh
fulfill the promises of semver the the
stability guarantees of semver
conan is also very active it has several
full-time developers which are sponsored
generously by jfrog
and as well as a very active user
community
which submits dozens of pull requests
each month
um so uh our slack channel has over 1200
members
and it is one of the most active
channels in the c
plus community slack so that’s worth
noting
finally the conan team has created
multiple interactive
self-paced training courses for conan
these courses are all completely free
and courtesy of jfrog on the online
training platform
known as the jfrog academy so if you’d
like to go
and learn more in depth and see more
exercises
you can visit the link below at the
academy.jfrog.com
and take those exercises
some great questions in the chat i have
some members of the conan team with me
here
and they will field all of those however
if there’s any that
require more elaborate responses we can
get to those at the end so we do save
some time at the end for q
a so the high-level design of conan
is fairly straightforward compared to
other well-known package managers
such as npm pipi maven
apt and yum it features common concepts
such as a local repository which will be
used for local conan operations
and package operations and remote
repositories for sharing packages
with easy to use upload and download
commands and capabilities
there’s also a central moderated public
repository
for open source packages and that’s
known as conan center
so artifactory is a server application
provided by jfrog
which can host remote conan repositories
and that’s what we’ll be using today for
all of our exercises as i said in a
docker container
also of note conan center is simply a
conan repository hosted on artifactory
but it’s hosted in the cloud with a lot
of resources and cdn assigned so
conan center is hosted in the same way
you could host your own private
repositories
so the hosting and the moderation
service and all the costs are provided
and maintained by jfrog
so while conan is similar to other
package managers
in these ways with its high level
architecture
it’s very unique in its package model so
this is what gets interesting
uh the string shown here in the box of
pkg
0.1 at user channel
this is an example of what we call a
recipe reference in conan
so this is the most simple and
straightforward logical identifier there
is
for packages in connect this is what
they look like
so one of the unique things about this
recipe reference
compared to other package managers is
that it can represent
any number of separate physical binaries
with unique
build configurations so you take one
recipe reference
and you can build any number of physical
binaries related to
and stored behind that reference so this
package model
in which one identifier represents any
number of unique build configurations
is one of conan’s major innovations
so here’s how conan implements that in
simple terms
conan assigns a unique id for each
unique configuration
this unique id is called the package id
and conan uses that
string for various things throughout the
build and packaging process
for example the package id is used as a
fundamental part of the directory
structure
for storing conan packages in the local
cache and it’s also
visible on the server implementations
it’s used for many other things as well
but this graphic depicts how
the differences in the build the typical
differences we see in build
configurations so for example
if you’re building a shared library
versus a static library
and all other settings are the same
you’ll still get two different binaries
so we still end up with two different
package ids
and those two package ids are tracked
and stored and transferred separately
so that is the theory and again one of
the fundamental innovations behind
conan’s
package model so the client and server
also use the package
id as the matching mechanism when
uploading and download donate binaries
from the remote repositories
so for most operations you do in conan
you are implicitly or
explicitly specifying a build
configuration
and fetch the appropriate package id for
each package
and download them all in one operation
also once you’ve built or downloaded a
specific binary package
on your local machine that binary can be
used across
any number of projects on the same
machine with the same configuration
so with conan there’s no need to rebuild
the same
library binary multiple times with the
same configuration
just to use it in a bunch of different
consumer projects
so for something like openssl you can
build it just once or once per
configuration
and you can use that openssl binary in
any number of consuming projects
so this cache provides a very efficient
binary sharing strategy
as i said conan uses the build
configuration as the input for
calculating these package ids
here are two in-depth examples of conan
build configurations which we generally
refer to as profiles in conan
so this is the text based format you can
also specify profiles as command line
arguments
or you can mix the two
so users can store these as text files
in this format and then just pass the
file names to the various conan commands
so this is a much more manageable
approach to
passing these values when compared to
passing each of these values as an
individual argument
so we will use both of these profiles
here in our upcoming exercises today
we want to demonstrate conan’s
cross-platform capabilities
so we’re going to do everything that we
do once on linux
and then once on windows with a
microsoft compiler
one final note about profiles is that
they also allow you to do something else
which is very powerful it’s not
necessarily what you would consider a
build configuration
it’s something a little different
profiles allow you to specify build
tools
which you wish to be made available in
your builds
so this is done with the build requires
section of the profile
as highlighted here in yellow in this
example we specified that we want to use
cmake version 3.19.0 for all the builds
in our graph which requires cmake
this implies that there’s been a conan
package created which contains cmake
3.19.0
and conan will make the cmake executable
from that package
available via the path environment
variable
throughout the package build process
so you can then therefore use build
requires
as an alternative to installing these
build tools such as cmake and others
directly into your systems and your user
systems
so for teams who are very distributed or
for organizations who have
want to have tool consistency across a
number of automated build servers and
things like this
creating conan packages of build tools
and
using them via the conan build requires
feature can
offer a lot of advantages and make it a
lot easier to get consistency and
guarantee
and also to move things forward at the
same time
so there’s much more to say about build
requires but we’ll use this one
in our exercises today and we’ll talk a
little more about it when we get there
so the other major innovation of conan
which we want to mention
here is the abstraction layer that it
provides over build systems
when developing applications in a team
it’s typical
a typical requirement that each
developer should be able
to build the entire application from
source including all of its dependencies
like libraries and other things
along the way many cnc plus libraries
and applications
have multiple open source dependencies
so unfortunately
open source cnc plus plus dependencies
often have unique and complex build
systems and scripts related to those
build systems
and so learning all of that is a major
burden for cnc
plus plus developers who want to focus
on the code
not on arcane build scripts and build
systems
and all the complexity that goes with
that
so conan aims to reduce this burden on
the developer
and that’s a big goal of using a package
manager
as an abstraction in general once an
open source project has been packaged
with conan
the users of conan no longer need to
interact with that build system directly
just to use it just to consume it
instead users can just install such
projects with the conan install
command and conan will handle all of the
necessary interactions with the build
system in the background behind the
scenes
this is a great utility of conan and
it’s a huge part of the value
proposition
okay so earlier i said we can start the
exercises now we can really start the
interactive exercises
so for our first exercise we’re going to
demonstrate a seemingly simple goal
which is to get an open source
dependency from conan center and use it
in our cmake project
it’s worth noting that the open source
dependency happens to be boost regex
this is relevant because getting and
using boost libraries has historically
been
a non-trivial endeavor for most
developers
so here we’ll show how conan makes that
process easier
and the key command that we’re going to
take away from this exercise
is conan install that is the fundamental
feature that we’re going to use
here are the relevant files for the
exercise just to put everything out in
the open
we’ve got a single source file named
regex dot cpp
which includes boost regex hpp
so there’s the dependency at the source
level
now uh in this case we declared that we
need boost version 1.74.0
and that we’re using the c make build
system so you can see that here
in the conan file.txt box so conan
file.txt is where we specify things
related to conan
and our dependencies and here we see see
make fine package
there we are specifically saying that we
want to provide the dependency to our
project
in a way that’s compatible with cmake’s
find package feature
and over here then we see the cmake list
file which tells cmake how to build our
source
and we can see that we’re indeed using
cmake’s find package
feature to locate boost and boost regex
specifically
with the required keyword
so this demonstrates that well notice
first that this
cmakelist file does not contain anything
specific or proprietary to conan
it is a vanilla cmake list file
this demonstrates that any existing
cmake projects using find package
can now use dependencies from conan
without needing to make any
modifications to the cmakelist file to
do so
this is noteworthy because it hasn’t
always been this way
the cmake team has had to work very hard
for several years with the community
to achieve this capability of complete
transparency
with cmake’s find package feature
so here are the commands that i’m about
to run
again these commands are available in
the slides so once you have the slides
you can run them yourself
the operative command again is conan
install
so the connect install command here will
download all of the dependencies listed
in the conan file that we just showed
and it will also download all of the
transitive dependencies of boost
regex so anything boost regex requires
and anything that those requirements
require will all be downloaded
automatically with the conan
install command after downloading
all the dependencies conan will generate
a number of files
to be used in the upcoming cmake build
one of those files is activate.sh which
we use in the next command source
activate.sh which you can see
here the fourth line
remember that we talked about build
requires in cmake so the activate.sh
script
is exposes the cmake executable from the
conan package where it exists
to our current shell it pre-pens the
bin directory from the package where the
executable is
to our path environment variable and
this is how we make use of cmake from
our conan package
so conan generates shell scripts which
allow you to use the contents of those
packages
at your shell so again
there’s a lot to say about build
requires and these script generators
so we explain a lot more about those and
it’s called the
virtual environment feature in the
courses in the jfrog academy
so if you’d like to learn more about
this really cool part of conan um
definitely go check out the courses in
the academy
but for now we’re just going to use it
so after that we’ll just run our normal
build system commands
and my build system will automatically
find and use the dependencies provided
by conan
again based on the generated files that
conan produced
in this case we’ll run cmake.dot to
configure our project
a very traditional workflow with cmake
and then cmake dash dash build
to execute the build so that’s going to
create
an executable called regex underscore
exe
and when we run it with a with the input
string here
we will get an output which has been
transformed by the boost regex
library so in this case just uh
the little demo we’re doing is that
we’re taking
email subject strings that start with
the word subject colon
and re colon and
it parses out using boost regex the the
prefix part
and it spits out a string that says
regarding conan so it just transforms
your question yes go ahead shaw uh thank
you
uh first of all i i work with conan or
at least
studied it and i work it with my own
projects for a while and i think it’s
great
i i want to make sure something that to
to know if i understand it correctly
in the third line you actually install
with the profile that used
gcc7 but implicitly you you assume that
gcc 7 is in your environment
like in this scenario as as you put it
you conan does not uh guarantees that
actually gcc 7 will run is that’s true
that is true and also a great point um
it’s a very confusing thing for me like
so i wanted to uh
to to check it okay yes worth pointing
out
as we said we provide cmake to the build
process using the build require
we do not do the same thing in this
example with gcc the compiler or any of
the related
tools the linker the archiver things
like that it is possible to do that with
build requires and we do know of
enterprise teams doing
that especially with cross compilers
people have packaged
their compilers into conan packages and
use them in this way
however a lot of other organizations
they
tend to use compilers that come with the
operating systems
um you know especially with regard to
gcc but on the windows side
you know it’s a different story as well
um but it’s a great point and
so yes we’re effectively just labeling
now and saying
that this is going to be a gcc seven
build we’re instructing conan that this
will be gc7
and it is up to us who manages the
environment in our case a docker
container
that gcc7 is in fact what is installed
in the environment it is up to us so
that part is out of band
very good point thank you
all right so now let us bring our shell
up once again
and let us just run the command so i
already narrated them once
hopefully
we it makes sense as i go through them
so we are going to make a directory
called build linux and cd into it
because as i said before we are
generating files with the conan install
command
and we want those you know with a
specific profile and this will be a
folder for a specific build
configuration
and then we are going to do the same
thing for windows so we don’t want we
want those two folders to be separate
so here i run the conan install and
there are quite a few things to point
out during this process
i will pause and say that
here you can see that it is looking up
boost then it is looking up
zlib and bzip2 and lib icon v
and then cmake so while we only listed
boost as the dependency
and cmake as a build require conan takes
care of the difficult process
of transitive dependency management so
it’s a really important part
of what a package manager provides in
general
another thing to point out here is that
we are downloading boost 174
as we see and the key file here is
called conan package.tgz
and if you look closely you can see that
the size is 20 megabytes
that is a testament to obviously we all
know most of us know
that boost traditionally is much bigger
than 20 megabytes
so what it’s actually downloading here
is the very specific single build
of boost that is a static library built
with gcc 7
on linux release mode uh for 64-bit
so that one single binary and the
headers is the only things contained in
this
and so that’s how conan provides a much
more efficient experience
than perhaps other approaches to
obtaining and
using boost which often downloads huge
files or the entire source
and re-ban you know building it multiple
times so
very efficient and it made the download
very quick so there we have it
the output here shows all of the things
that conan generated
if you’re familiar with cmake and the
find package
system you know that it is based on each
library and each component
having a find dot cmake file
and so we produce one for each
dependency in the tree
and those we could we could look at but
we won’t today
but they’re there in our current
directory
so this was all produced by a c make a
generator what’s called a conan
generator
named c make find package and the
scripts as i mentioned before the shell
scripts of activate.sh and deactivate.sh
those are produced by a generator called
virtual environment
and so conan can use any number can
generate any number of files for any
number of build systems in one shot
simply by adding the list of generators
it’s a very
extensible feature and a very important
part of conan
so we are going to leverage the first
generated script called activate.sh
before i do that i’m going to type which
cmake
and show that i do have cmake installed
on this system under user bencemake
and if i do cmake version
it is 3.9.1 however after i activate
and i type which cmake you can see now
that cmake is located
in a deep nested subdirectory of the
conan cache
so this simply proves you don’t need to
understand the entire conan cache
this simply demonstrates that we are
using now cmake from
that location and when i type c make
dash dash version
here we have not 3.9 but 3.19
so a very novel part there
so now with our new cmake we will run
the configuration step
we um do specify two arguments there
i will tap the up arrow uh the
this final argument is worth noting
this tells cmake to look in the current
directory
for additional cmake modules such as the
ones generated by conan
and this is how we get conan or cmake to
transparently find conan’s features it’s
not entirely transparent in that we pass
an extra command line argument
it’s transparent in that we don’t have
to modify any cmake files to do so
okay great now let’s build
our project with our cmake file
great so it built a target named regex
exe
and let me just check the chat to make
sure that i’m not missing anything
great in order to run that on linux we
do
excuse me we do
we run regex dot slash regex exe and
then we pass it this string
and it’s going to transform that string
if i don’t uh
and there we get the output that i said
we would get regarding conan
so it parses out the information
so that was pretty quick in terms of
using boost to produce a little
executable
you know from a package manager is a
pretty simple demo pretty effective
we also want to run a command called
sourcedeactivate.sh
and what that does is if i type which
path again
as you can imagine sorry which cmake
you can see that we’ve now reverted the
current shell back to using
the cmake in the system so these virtual
environments
similar to python virtual environments
if you’re familiar with those but
it allows you to add and remove things
from conan packages from your shell
okay so now i will cd back to
the source repository
now we will go back briefly to the
slides and a very timely question just
popped up in the chat
about how easy or hard is it to use
conan with visual studio on windows
let’s look so we have here now
the exact same exercise that we just did
on linux
this time it’s on windows and this time
it’s for visual studio
still using cmake as the build system oh
okay there we go star without cemic well
that’s another question entirely
um but it is possible and it the process
is
in spirit identical so but that’s a good
question
i’ve highlighted in green the parts of
these commands which are different to
account for windows
but everything else if it’s not
highlighted in green is identical
so i will breeze through this and say
that um
let me see here one moment
right so the directory name is different
the profile name
is different again it’s up to us to
guarantee that we have visual studio 16
installed instead of
using source we use the call command and
instead of a dot sh script it’s a dot
bat script
also we have instead of pwd as the thing
we have this
cd environment variable
to tell cmic to look in the current
directory
and then additionally when you run cmake
for a windows visual studio project you
have to specify a configuration so we
specify the release config which is a
little different
and when we execute our binary the
syntax is slightly different we have to
run it
but the program is the same we still
pass the same string we still get the
same
output and we still deactivate our shell
so let’s do it
all right so we will now bring up our
shell once again
i will leave this linux shell in place
i will open a new shell for
this part cd
into our repository we will cd
into the exercise directory and we will
make directory
for build windows and cd into that
now we’ll do the conan install command
again this is the magic command we’re
focused on here is conan install
giving it a specific profile
so that went a little faster because i
had downloaded previously on my machine
this is a good opportunity to show that
this tells us that boost
bzip lib icon v this uh this output were
all found in the cache they did not need
to be downloaded so we didn’t have to
wait for that
so this goes to the point i made earlier
about reusing binaries
once you’ve used them for one project
and they’re in your cache that
development machine can reuse those
binaries for any number of projects
as long as it’s using the same config it
still generated
all of the files that were needed so the
find package
files.cmake and the scripts
to the scripts to add cmake to the path
excuse me copy paste is the hard thing
so we call the activate script we type
where
on linux or on windows we type where cma
and here you can see that it is
preferred it was prepended
to the conan directory but
it also shows that another version of
cmake exists at a different path but
we will now do the cmake configuration
you can see it’s using visual studio 16
as i indicated by the profile name
i will run the build command and then i
will execute
the binary that was produced so here you
can see the output if i zoom out a
little bit
regex underscore exe.exe i will run that
in the release directory and we get our
output we give it the string and we get
our output
okay now normally i will stop and take
questions here in between
um exercises but i know we have a little
bit of uh
of a compressed time scale we have so
there’s some things we want to go
uh adi mentioned at the end we want to
make time for so
i will i will take questions at the end
so please do write them down if i don’t
um
if you don’t get them answered in the
chat
all right so we did that it worked no
technical difficulties yet that is a win
um so now we are going to move uh
well we’re going to quickly summarize
what we just learned the process of
consuming a conan package
is one use case for conan if you don’t
want to use conan for anything else
you can use it to obtain open source
third-party dependencies
and use them in your build system
without
coupling your development process any
further to conan
the command to do so is conan install
this obtains the dependencies and
generates files for build systems
um consuming oss packages can be simple
as we’ve just shown uh and in c plus
plus that is
has relatively uh has not been the case
uh up until relatively recently um
conan can provide dependencies and the
information for those
to any build system so this includes
cmake which has a fairly sophisticated
dependency management feature called
find package
but it also includes custom build
systems and
for which you can generate a text file
with the dependency
information in it as long as you know
the format that your build system needs
you can create a generator to produce
the information
for that build system custom generators
is one of the most frequently used
extensibility features of conan
and almost all enterprise organizations
that we work with and talk to on a
regular basis have written at least one
custom generator for their different use
cases so
it is very easy to get into easy to do
and uh something that um
we’re really excited about finally conan
center provides many packages for
popular cnc plus plus projects
which include pre-compiled binaries for
many uh different platforms and
configurations
but not all so if you’re if you’re doing
development for an arduino or for a
raspberry pi
we’re not going to have the binaries you
need but if you’re using a mainstream
compiler and operating system like
windows and visual studio 2019
or linux with gcc 89 or you know newer
we often will have a good configurations
pre-built for you to try
however if we don’t have the binaries
built that you need
it’s really important that everyone
understand that there’s an additional
flag to conan install and that flag is
dash dash
build the dash dash build flag
lets users choose to build all or some
of the dependencies from source on the
fly so
and even if you just simply don’t trust
the binaries in conan center or don’t
want to use pre-compiled binaries
it’s another reason to just pass the
dash-build flag and you can tell conan
while downloading also build from source
the process will take much longer
but all the binaries you end up with
will be simply built with your compiler
and your tool set
on your machine um so there are many
situations when you might want to use
this
your organization may have a policy
against pre-compiled binaries
and so forth so you know understanding
conan install
we really also want you to understand
the power
of the dash dash build flag
okay now i would like to talk about
conan recipes
i want to explain what a recipe is and
then show some examples of them
but in simple terms a conan recipe is a
python class which conan will read
and execute in order to create a conan
package
obviously we’ve used some conan packages
so that means there are recipes for
those packages somewhere that someone
has written
which takes care of all the difficulty
of interacting with the build system so
let’s see what that looks like
so here’s an example of a conan recipe
as we said it’s a python class
and it features a number of standard
method names
these method names all correspond to
traditional steps in the process of
building a c
project for example
in the requirements method we declare
our project dependencies
that’s perhaps less common in the
traditional build process
and but in the export sources method we
define which sources conan will need
to capture in order to do to build the
project so maybe you don’t need to
capture the readme
or the you know jenkins file or
something like this you just need to
capture the c files and the header files
and that’s what you need to build so
that’s what export sources does
now in the generate method we produce
the files for the build system
which contain all the variables we need
so if you recall we we generated a bunch
of find
package cmake scripts that will take
place inside the generate method
moving forward in the build method we
invoke the build system
which we’ll you know call cmake or call
ms build or
call make in order to build the sources
into the binaries that we want
then finally in the package method we
capture those binaries which were built
and we copied them to a package
directory this includes
shared libraries static libraries header
files anything at all that was produced
and that we need to transmit to our
consumers
then i guess i should say finally now in
the package info method
we declare the contents of the package
directory
that we just captured and populated for
the consumers
so for example if we captured some
static libraries the package info method
will list
what are the names of those static
libraries so for the
linkers of the consumers and then what
are the directory names if they’re in
subdirectories
where we can find those libraries to
link with also we
it defines the directory for the headers
and also other variables like see
the preprocessor directives and things
like this environment variables can be
defined here
so again the package info method
describes
the contents of the package as needed by
the consumers
so conan will call each of these methods
in the appropriate order
when you run the the creation process
now here’s another example of a recipe
and this one has the methods actually
defined
so it’s a little less abstract it’s also
um
we simplified things a little bit of
note the project being packaged with
this recipe here is a cmake project
so it takes advantage of several cmake
specific helper classes
that the conan team provides which makes
the recipes a little easier to read and
write
and we’ll show an alternative to that
next
so first in the requirements method as i
said we declare dependencies
and this is what the syntax looks like
for that you call self.requires
and you list your dependencies um
next in the esports sources method it
declares that in this case we’re going
to capture all files in the working
directory
as potential sources for the build as i
said you could filter out files that
don’t matter
but this star is easy for demonstration
next the generate method uses two helper
classes
which produce dot c make files from
conan variables
this includes all the information cmake
will need for the upcoming build
as we said it creates the find
boost.cmake
it also creates a tool chain file which
has other variables related to
the settings that you pass to conan in
the profile so we won’t go into detail
about those but
generate this is where it produces these
files
and in the build step we will consume
those files
so the build method uses another helper
called the cmake
helper class it’s just called cmake
and this will when we run
cmake.configure and cmake.build methods
in the conan recipe conan will invoke
the cmake executable at the shelf for us
these helpers just make it a lot easier
to get the command line syntax right get
spaces and quotes right
avoid common errors but you could just
do self.run and
write your command line string raw
exactly how you want it to go to cmake
and not use the helper and we’ll see
that in the next slide
but ultimately it translates all the
arguments correctly and it gets
it’s helpful to use the helper
particularly in lee in cmix case
finally the package method also uses the
cmake helper
to do the install method and this
actually runs the cmake install target
so a lot of cmake
projects define what’s called an install
target which defines kind of the run
time
layout that the produced artifacts
should go to
and conan allows us to make use of that
target
by doing this giving this cmake install
function
and in the notes here in the teal you
can see that
it’s it sets the cmake install prefix
variable to the appropriate directory in
the conan cache
to make use of it so it’s a really neat
trick and it’s very helpful
finally as i said package info we need
to declare where we put files in the
package folder
and what files we put so we have an
include directory
us and we list that as the set of
include errors this will be translated
into
the dash i compiler flags that we would
see later in consumer projects
the list of library directories will
translate to traditionally for gcc
minus capital l as a lib directory
and so we put all the libs in the lib
directory and then finally
the libs is the list of libraries that
we produced
up here we can see the name of our
package is called my lib
and it’s going to produce a static
library called my lib
um that’s not shown here but we can
imagine that that’s the case
and so down here when we say libs equals
my lib we’re saying the linker needs to
link with my lib
that’s that’s what this package produces
is a library named
my libmylib.a or on windows it would be
my lib.lib
and so this is the abstract declaration
of what’s in this package
and we the the generators are what
converts this to gener
uh to build system specific files
some great interaction going on in the
chat great questions great answers
thanks everyone for participating there
all right here’s the third example of a
recipe and then we’ll we’ll run and
create the package out of this recipe
this one is very similar to the previous
example which you see make however we
want to demonstrate
what a recipe looks like when you’re
using a custom build system
for which conan provides no helper
classes which is very very common if
you’re going to use conan
professionally your recipes might look a
little more like this if you have
non-cmake projects so first
the requirements and exports statements
those are the same that’s great
for the generate method you’ll need to
write your own function
if you’re using a custom build system
which takes the variables from conan
and writes them to text files in some
format your build system could read
maybe your build system reads yaml files
or
text files or dot make files
for dot you know for make or make
derivative
again it’s trivial to write a custom
generator uh or you can just
write the function here that takes
information take we pass
you variables in your generate function
and you can convert those to
text files and you can write them to
disk so that is the purpose of the
generate command
we don’t show a complete generator but
we do show that you’ll need one
then in the build method you will need
to call your custom build system
and any number of other cli applications
that you might want to run
many build methods are more than just a
single command
they’re 10 20 30 commands long to
instrument and orchestrate a custom
build system
but you could do whatever you want in
the build system you have all in the
build
method you have all the power of python
at your disposal and a lot of helper
methods from conan
for running and for reading files and
writing files
so very very flexible way to
define a build process next in the
package method if we don’t have an
install target like cmake provides
we need to declare and specify what is
this
build system producing and how do we
want to store
the resultant files in our package
folder
we can take static libraries shared
libraries and we can put them in
whatever folder layout we want
in the package folder so we do that here
the self.copy function is a helper
function that you will see
all the time in conan it has very
flexible optional
parameters for source and destination
directories
and it’s pattern based so you can just
say find all the headers
in the in the resulting directory and
throw them all
into in include directory or you can
find only the ones that exist in a
source directory
if there’s header files laying in the
root those would be skipped
so we use the same pattern for dll files
lib files dilip files so files
we put those in well dlls go into a bin
directory traditionally
the rest go into a lib directory
traditionally so this
package method using patterns
and a sort of generic folder structure
for the destination
is a way to provide a cross-platform
method that will capture
all of the outputs of this build no
matter what platform it’s run on
so this is sort of a you know take a
messy output layout
on any number of operating systems
and or organize it into a package folder
of
of a generic structure so that’s what we
do here
and then again it doesn’t actually
matter what folder structure we want to
use
to store those files because we can
describe it
in the package info method however we
want so in this case we
simulated the cmake install by also just
using the same structure
the headers went into an include
directory the lib directory
all the libs went into a lib directory
and the libs
was named there was a single library and
it’s named my lib but we could have
defined any folder structure we wanted
in the package method
as long as we properly described it down
here in the package info method
that maybe is abstract and it takes a
little practice
once you’re writing recipes to see how
that works out
but hopefully now you can understand
sort of the process of
what the recipe does in what order and
the ultimate the final way that
conan captures artifacts and provides
them to
consumers okay that is
a lot about conan recipes but it is a
core you know fundamental piece of conan
so we really want people to understand
when they look in conan center for
example when what they’re seeing
and give them some context when they see
those recipes to understand what’s
happening
all right now we’re going to create a
conan package and of note it is the
same project as the previous example
we’re going to replace conan file.txt
with conan file.py
and we’re going to define all of the
required methods in the conan
dot py then we’re going to create a
package from that recipe
and the key command we want to take away
here is conan create that is conan
create a package
as a reminder our source file was named
regex.cpp
and it depended on boost regex uh we say
that the coin and file.py will show that
next
but we just want to show the cmake list
is identical to the one we did
we used in the first exercise so here is
our
actual conan recipe that we’re going to
use
and as you can see when you the the
former ones were a generic case that
showed all of the methods we needed
when we have a real simple package the
recipe gets a little simpler
and it’s easier to look at we um only
use four of the methods those are the
only four required for this particular
package
we named it appropriately so the package
will be named regex 0.1.0
we’re not going to talk about the
settings line today but this simply
uh defines some of the things that go
into the package id calculation
and finally we say that we want to
generate the cmakefind package generator
and the virtual env files just like we
did
in the consumption exercise but we are
saying here that we need it for the
build method
okay
as we mentioned earlier we’ll use the
conan create command which will
use our recipe to build our package and
create the package from it
after we create the package we do want
some way to show that it works
to be honest the exercise is a little
boring if all we do is run
the conan create command it’s a single
command that’s not really what we want
to show
it’s not really enough to show i should
say after we run
conan create the rest of these commands
are
a way to demonstrate the executable
works
from the conan package so just like we
ran cmake executable from a conan cache
we’re going to test our executable our
regex executable
from its package location after it’s
been created
so that’s the goal here and that’s why
we run conan create on the second line
and then we make a directory called run
linux and we cd into it
and then we behave like
we did in the previous example where we
are now consuming so we run conan
install
of our package reference of regex we
generate the virtual ian
run env generator we pass it the same
profile because we say we want to use
the one that we built with linux gcc7
release mode
and then we use the same source activate
trick to
get that executable on our path then we
run it and we get the same output
again going a little quickly for the
purposes of time
so i apologize for that we will take
questions at the end
okay so where’s our session here
so we will go back to our linux shell on
the left
and we will cd to
um
the examples directory once again
this is the create package example and
we will run
conan create command as we did before
sorry as we did before passing in the
profile the same profile we’ve been
using
and this will then
do everything that we said in the recipe
gather the sources it will download the
requirements
it will run the build and then it will
package the results of the build
and put them in our local cache
and we see here you can see hopefully
the the cmake was executed
here this is cmake logs
and then down here at the very end we
see
package created and we packaged one file
regex exe underscore exe
so we just want to run now that
executable and make sure that
it does what we expect it to do
so we run conan install and now we give
the reference because
in this log it prints out the full
reference
the of the package that was created so
it combined the name and the version
and also the two arguments we passed at
the command line
demo slash demo so now when we run our
install
we pass that same reference so we’re
saying we want to install that
with the virtual run envy generator
into the current directory so we will
try that
and we can see here that we have
generated the virtual
environment scripts once again and if i
activate
and i type which oh
i should note um
well i’ll just simply run it with the
which command
if i do which regex exe
just like the cmake example we did
before our path now finds that
executable
in the conan cache
so we can run it without having to pass
that whole path right
there we go so we passed the string and
we got the
expected result so that was fantastic
we will once again then deactivate that
environment
and cd back
okay i will now go to the next slide
and we’ll do the same for windows again
the again part of the
comp complexity of this demo is to show
the cross-platform nature and power of
conan
and just how similar the workflows are
on windows and linux and mac os
is the same again we just have the green
highlights that are different we have
the different profile the different
folders
slightly different semantics and syntax
for calling
the the binaries but more or less the
process is the same in spirit
so let’s do that very quickly i’m going
to leave the
linux shell and now enter the windows
shell again
remember the right commands
okay so we are now under windows
the conan command is the same the conan
create command has no differences on
windows and linux so that part’s good
we’ll go quickly because at the end we
really want to show
that final thing the thing we talked
about very early on
which is how the binaries live side by
side
under a single package reference
okay we did it if we look down here
package was created
and we packaged one exe file with this
file name
so it invoked cmake everything’s the
same
it packaged the one exe file and now we
want to use that
i will check the chat real quick if
there’s any
important things that i’m missing
[Music]
i do see yes it’s uh it’s good to note
that there’s a
there are actually more than one virtual
environment generator and we have used
two they’re slightly different
um and i did breeze over that so
glad to see that that was asked and
being addressed in the chat
so we want to run our executable in our
run windows directory
to do that we just need to produce the
activate run script
and we do that it’s also worth noting
after we run
the activate scripts it always puts a
little
prefix on our shell that tells us that
we have actively loaded
a conan environment failed to mention
that earlier but there it is
okay we got the same result on windows
let’s deactivate which as you can see
now after i ran the deactivate
this conan run env prefix is now gone
and we have run the same thing on
windows we ran our executable
we got the same result okay so we are
cross
platform packaging and building now so
this is exciting
to quickly summarize what we did
we showed the recipe format what it
looks like how it works
we walked through each of the standard
methods
for creating a package you have
requirements export sources
which gather sources and then you build
and package
the artifacts and then you describe what
you built and packaged with the package
info
there are a handful of additional
methods which are optional
which you can read about in the docs and
see in other training sessions
this these are some sort of the most
fundamental
and the basic requirements for most
recipes
and finally yes conan calls these
methods in a particular order
there are other workflows as well you
can just call the build method sometimes
you can just call the source method
for for different workflows but this is
the core process for creating
a package from scratch
so our final exercise we’re almost done
is now to upload
the package we just created to a remote
repository
that includes uploading from the linux
build and uploading from the windows
build environments
so we’re going to upload effectively two
different binaries for the same package
and we’re going to see what happens
so first we’re going to review the list
of remote repositories
then we’re going to add a new repository
which is in our docker environment
um the repository
has just been created automatically for
us when we started up that docker
environment so we’re not going to go
through the process of creating
a new repository on artifactory there
are other courses for that
um but of no artifactory ce
was a great fit for this demo because
it’s free to use
and it’s specifically designed for
hosting conan repositories and making it
easy
in a local small environment like this
so on that server
we’ve got a repository named conan local
and we’re going to upload our package to
it so let’s do that
as i said we’ll start by listing the
existing repositories
and we should see conan center and then
we’ll add our new repository with the
name of artifactory that’s what we’re
going to call it here
and then we’re going to give the url of
our demo environment
since this is a private remote
repository and we want to upload
packages to it
we have to provide credentials in order
to do that
so the conan user command is how you
provide credentials
for a remote repository to give yourself
write access
obviously you can’t upload to conan
center
you won’t have credentials to do it even
if you did so on accident
it wouldn’t let you um so uh next we
should be able to upload our hello
our regex package to the repository
and um then finally at the end we will
use the conan search command
to search our remote and see what the
results look like on the remote server
so that’s that’s sort of the the last
moment the uh
the payoff is that we’ll get to see that
we’ve built these packages and now
they’re
coexisting together in a single place on
a remote server shared with the rest of
our team
okay let’s do that
i will bring up the shell
we will jump back over to the linux
environment
full screen it excuse me
clear
so if i do conan remote list we should
see conan center as i said
conan center comes by default when you
install conan it is the only official
repository that comes installed by
default
we will then use the kona remote add
command we will add
the name artifactory and with the url
that
corresponds to my docker container your
docker container
if you run it locally should be the same
url with the same ports
assuming you don’t change anything
before you start it up
okay let’s run conan remote list once
more
do we have it we do
artifactory is listed here the second
one now we can specify a user and a
password for artifactory
admin and password are the default
credentials for artifactory ce
now we should try to upload so conan
upload our regex package
to artifactory and upload all binaries
so here we see uploading regex
it has uploaded the reference of regex
0.1.0 at demo demo and a single binary
one binary package from the linux
machine
to that remote repository okay
so let us quickly do conan search
for that on artifactory and we should
see one binary
yes we see we have an existing recipe
with a single package id listed and here
the search command is a very nice output
can also come in json with settings
for the architecture the build type the
compiler the version
the c plus plus standard library we used
and the list of dependencies as well
great great feedback
great piece of output okay it’s
important to say that uh
for industrial use i also
every specific flag you use to create
your own package will
also be there yes yes
it is very um it takes into a lot of
information
and it produces a lot of information um
that output gets pretty long there are
ways to filter that output as well
that’s worth noting
now over on our windows machine let’s
make sure
we still only have one binary the whole
point
apologies we’re going to get a clean
start here
all right again our goal
is to upload in in a traditional
environment if you have multiple
platforms you have ci with build
machines that build on different
platforms
you will end up uploading the same
package but different binaries from many
different machines from from your build
infrastructure usually so we’re
simulating that here
and i will add artifactory once again
i will provide the credentials once
again
and then we will upload the regex
package
now here we see that it uploaded
the regex recipe but also it uploaded
one binary package this by this package
id
it is different from the one that we
built on the linux machine
so now when we do the search if we cross
our fingers we should
two see yes
so
there is a problem with uh which i
apologize for
all right so there we now see that if we
do the search
conan search for regex we have package
id
this was the windows build and here we
see it is visual studio
windows visual studio 16 release the md
build
and here were the dependencies on
windows and we see a separate package id
from the linux build which was the one
we had before so
in many environments you will have 4 8
16
32 different binaries for a package
such as regex depending on how many
build targets and how many
configurations
you need to build and test for your team
so again this drives home the original
point in the original innovation of
conan
which is a single recipe reference can
contain any number of binaries and those
binaries are managed separately they’re
downloaded and uploaded separately
it’s a very efficient and intelligent
process
if we have time which i think we may
um adi do you have a time limit
um are we approaching uh
how do you feel about time
i can jump over and show how it looks in
artifactory well
yeah you can take a few more minutes
okay actually so i’ll just jump to
i just want to finish out the slides and
then we can do that if we have time at
the very end
i don’t want to we do have a couple
slides here so um
so this exercise we uploaded a conan
package
very particularly we uploaded the
package multiple times
from different build machines and they
all showed up under the same recipe
reference
we showed some easy commands for remote
management
we showed that conan center always comes
installed
by default and um
we did show that the local conan cache
which exists under conan data can be
shared by any number of local projects
and builds we actually showed that a bit
earlier
but the idea of the local and remote
repository strategy
is very similar to other package
managers and we feel that that’s very
important
because this provides a practical and
familiar developer experience
for not just the developers but also
build engineers and
people who whose job it is to set up
automated build pipelines
oftentimes they’re working with multiple
package managers in multiple languages
and conan fitting in to that process in
a way that they’re
used to is very helpful um
and it makes the tooling better it makes
things like
ci servers and other tools interact a
little more
sensibly than say if we did if we didn’t
have
this these abstractions so finally we
demonstrated how artifactory ce provides
free local hosting of conan repositories
and how fast and easy it is to set up an
instance and start uploading packages to
it
when we started this i truly did not
have artifactory installed
i started artifactory in a container
with a docker command
and i was able to immediately start
uploading and downloading to it so
it can be that easy for you and for your
dev teams as well
so that’s all the exercises i have for
today which is a shame because there are
so many different features that we
didn’t even get to show
but with that said here are the places
you can go to learn more about those
features
on the home page at conan dot io you can
find links
to each of the following resources each
is valuable in its own way
and depending on your environment and
your interests and needs we hope you’ll
find them helpful
also if you want to just chat with me
the conan team
or other conan users you can find us all
on slack
in the cpp lang community there is a
channel
dedicated to conan hashtag conan so
you can sign up there and you know ask
questions and just
direct a message any of us if you need
in closing i really want to thank um
adi and everyone here for attending
for allowing us to present and i hope
the session was enjoyable and
informative
i hope you are able to download and run
the
demonstrations for yourself locally and
i do want to hear from anyone who has
any trouble
or success if in any case if you feel
free to mention us on twitter or
linkedin let us know what you thought
but at this point i’ll be happy to take
any questions and comments or any
feedback at all