Enhanced C/C++ Package Management Made Easy with Conan 2.0

Christopher McArthur:
So welcome, everybody. I’m super excited to get to kick off the Conan 2.0. For those of you who haven’t been following, it was officially released last week and we are super, super excited. Huge shout out to the team, little round of applause. As much as Diego is the hero, he’s not the only one. We got Carlos, William, and David who are online with us and I’m sure there’s a few others from the team that are listening, so.
I’m going to be presenting. Diego will be joining me today. And brief little round of introductions for those of you who might be joining us for the first time. My name is Christopher McArthur. I’m a former Conan user, longtime contributor. I was a member of the Tribe. And when I wrote these introductions, I decided to give myself a little quote. I’ve been opening enterprise issues for Conan for over three years now. I always been a thorn in Diego side. You can see me asking questions like backing up offline builds and sources of recipes and packages because that’s the type of thing you need to do as a enterprise developer. And now that I’m the Conan developer advocate, I will promise all of you to continue being a thorn in their side with, much affection, of course. So Diego, I’ll let you do your own introduction.
Diego Rodriguez-Losada Gonzalez:
Yeah. I just copied yours and added a little bit. I basically been resolving enterprise issues Conan for… Well, actually a bit more than six years. I think Conan is now like seven years. I’ve been six years at JFrog, leading the Conan team and trying to help the community with this tool.
Christopher McArthur:
So what is Conan? If you’re new and you’ve heard of Conan, maybe you’re not too sure with the familiarities of what it’s supposed to do. So Conan is a C and C++ package manager, and its role is to make it easy to install dependencies. So Conan install requires speed log and you should be able to get up and running. How is Conan different? It enables you to build and distribute binaries, and that’s one of the key focuses that it has that delineates it from the other ones.
We offer JFrog’s ConanCenter. There are over 1,500 open source projects with over a hundred configurations, different operating systems, compilers, architectures, and they’re all being created to help accelerate open source. So Conan is MIT license. It’s distributed or decentralized, depending how you look at it. There’s one client to many servers. That’s the architecture of it. It’s scalable, flexible. Really, with your leaky water back there, man. Ugh, the dog today. And it supports all kinds of different platforms, CMake, Meson, Autotools. And it’s any platform. So Windows, Linux, Mac, but you can also do Android iOS, WebOS if you’re interested in that. And it’s more than just CMake. So it really gives you a framework to build your CICD around.
So a Conan file. And this is what we call a recipe. You can see our cute little froggy in the bottom right-hand corner. So here we have a small example of a project math. So it has a class, a name aversion. There’s a source method, that’s how you get the sources, a build and a package. So if you do GIT clone, you go into your math directory and you do Conan create. You can take that recipe and you can create math binaries.
So with Conan 2.0’s launch, we have completely revamped the documentation. And if you want a hands-on tutorial for how to do this yourself, the best place to start is going to be this section here. So if you go to docs.conan.io, you can go on the tutorial section and you’ll be able to follow along with us.
Okay. We’re good. So why would you pick Conan? So the key difference between Conan and the other package managers is that focus on binaries that I mentioned earlier. So being able to create a package that can be reused across multiple teams throughout your organization. And the whole point of doing this is to enable flexibility and scalability, right? So it’s a framework for doing DevOps and package management in a very enterprise-ready manner. So with Conan, you can model platform configurations and the linkage between your libraries, and this will allow you to know which branch of your graph you need to rebuild, and it can save you so much time and money being able to do this.
So one of the common problems that I see all the time with C++ developers, most of us are behind on the latest standard. And as nerds, we always want to have the latest and greatest tools in technology. So let’s say you have a code base with a build environment validation and you produce an app, you ship a product to your customers and that’s how you get paid every day, so that build environment usually has a configuration, right? In this case, we’re doing a GCC 7 with C++11. And what we really want to get to is being able to have GCC 7 with C++11 and GCC 12 with C++20, right? And going through the same pipeline, we’re able to build our app for our LTS, which is supporting our customers, but also current. So the current future of development that we’re trying to aim for and trying to support.
So with that out of the way, what’s new in Conan 2.0? I think that’s what you’re all here for and that’s what we’re most excited for. So everything is new. There’s been five years without a breaking change. Stability has been amazing on this with the release format that we’ve been doing, and we’re very excited to introduce some of these new changes. So 60% of the code base is new and about 20% is backports. And the reason for all of this, is so there’s a compatible syntax between 1.X and 2.0. So you’ll be able to upgrade from the latest and greatest of 1.X to 2.0 with some small changes to your workflow alongside of it, but all of your rest… No, Google. Not too many. So the recipes, which are the most important part of all of this, those have that compatible syntax. So you’ll be able to use those same recipes for both 1.X and 2.0 as you move along.
Diego Rodriguez-Losada Gonzalez:
And same server. You can put binaries for both one and 2.0 in the same server at the same time.
Christopher McArthur:
Perfect. Perfect. So this is a page that I’ve been working on, what to expect? So on the left-hand side are some of the breaking changes that have gone through with Conan 2.0. We’re not going to look at those too much. Today we’re going to be focusing on the right-hand side and we got a few things in bold. We got graphs, extensions, and lock files, just to name a few. The key thing to highlight here is, everything is on the documentation website for the most part. So if you want to learn more information about all these things, the docs are going to be your best friend.
So why have we done this? That’s one of the big questions, is why would we make all these changes. And it’s because we’ve been listening. So usage and feedback is one of the key drivers to everything that’s been going on. We usually get about around 800,000 downloads every single month from PyPi. And Conan is designated as one of the most critical projects. That’s the top 1% of downloaded projects in Python. So the C++ package manager is top 1% in Python. I always find that amusing. And when I snagged the screenshot for the PyPi stats, you can see there was 840. So that’s an exceptional number of people decided to jump on and try out Conan 2.0 for the first time. We also have Slack, which we’re the second most active channel, and of course being open source, we’re available on GitHub. And you can see the number of contributions in terms of poll requests, have been growing year over year, and we hit over 5,000 last year, which is absolutely amazing.
Now this is where the team gets the most round of applause. The support that we do. So we get around 2,000 issues every single year on GitHub. We do over a hundred hours of voice and video calls and support for customers, users, and we really try to help them as much as possible. And there’s direct support as well, which is harder to track. So Slack, we get messages almost daily asking questions, and you can see that from our Slack community’s activity as well.
And last but not least, there are a ton of other enterprise users. We’ve launched the Tribe to help us define Conan 2.0. So you can see here we got some amazing companies like Bose, TomTom, Continental, just to name a few. And they’ve been helping us finalize some of the designs for 2.0. Anything you’d like to add about the introduction so far, Diego?
Diego Rodriguez-Losada Gonzalez:
No. There are a few questions on the chat. I’m trying to respond them regarding the compatibility of the Conan’s [inaudible 00:09:40] but I think it’s good, so you can keep moving.
Christopher McArthur:
So we are going to be going over some of those feature highlights today. So we have a binary and graph. We’re going to be going over some of the extensions and custom commands, and we’re going to be doing reproducibility with lock files. These are just some of the big features that are coming out, but by no means all of them.
So our first lesson is going to be learning to fly and C and C++ are challenging. So Conan was first launched in 2016, that’s over seven years ago now. And since there, Conan has seen significant growth. So with 2.0, the new documentation has been completely rewritten. So there’s a hands-on tutorial, there is better, well-organized API references and there’s the newly structured examples which are absolutely breathtaking. As a Conan user, I am super excited to actually use these. They’re so helpful. Oh. You were going to say something, Diego?
Diego Rodriguez-Losada Gonzalez:
Oh, yeah. And it’s still ongoing work. So we will be focusing on improving the documentation. There are still missing parts and we know it, but the documentation, in the following months, we will keep adding new things, new sections. We will try to advertise them, answer them with you. But there are important things like the continuous integration and other parts that are missing and we will be adding them soon. So keep tuned.
Christopher McArthur:
Awesome. So one of the things that we had to do for C and C++ is modeling build-time only dependencies. So when you have build-time dependencies… These are things like tooling requirements, test requirements. They don’t necessarily want to impact the final binaries themselves. So for this, we’re introducing, inside of our requires, the notion of traits. So there’s going to be requirement traits and package types that are going to get added. And this is going to help you define the interactions between your libraries so that Conan can more intuitively and more [inaudible 00:11:49] make decisions around them.
And so, one of these things is supporting direct and transitive dependencies. So you can see here we have a cute little example. We have a game that requires an engine that requires math. So the game has a direct requirement, so it does self requires engine. And then we have our engine recipe that has a requirement to math. When Conan builds the graph for games specifically, it’ll include that transitive requirement to math. And now, depending if these are shared, static, [inaudible 00:12:23] only, it may decide to include math or it’ll exclude math depending on what’s needed to build game. And all of this is configured with the requirement traits and the package types.
So the dependency graph can now allow you to do correct linkage requirements, correct header visibility, private and hidden dependencies and so much more. Diego gave an amazing talk at ACCU last year where he went over this in depth for a whole hour, and that should really give you a good starter pack. If this is the piece of information that you need more on, it’s going to be in that talk. So one of the highlights is, it works between different build systems as well and there’s a compatible require syntax between 1.X and 2.0. So again, you can take advantage of that now as you build out your 2.0 pipelines.
So all of this is to make Conan 2.0 way more efficient. So fetching only the binaries that is needed to build and run. So there’s a new mode for package IDs, and these are going to change some of the defaults and they do not collide with the 1.X. So that’s how your binaries can live in the same server, which is fantastic. So being able to skip parts of the graph to do downloads is absolutely amazing, and it’ll allow you to better manage those dependencies. So the new graph model, requirement traits and package types, along with package ID modes, are going to really change the game for you guys. Is there anything you wanted to add for that section, Diego?
Diego Rodriguez-Losada Gonzalez:
No. I’m just typing also answers on the chat.
Christopher McArthur:
Perfect. Thank you so much for following that one.
Diego Rodriguez-Losada Gonzalez:
Yeah. I think it’s very important here in the graph model in 2.0. The graph will always be complete. It will always include all the [inaudible 00:14:24] dependencies, including tool requires, test requires, because this was a very special pain point in Conan one. For example, for lock files, your computer [inaudible 00:14:35] lock file and the lock file was not locking everything because the dependencies were not there because they were skipped. So Conan 2.0 is building the whole graph, only the recipes. So the binaries, which is the heavy part, can be skipped so it’s still efficient, but the information about the dependency is always complete, and this is super helpful to for things like providing continuous integration and these things.
Christopher McArthur:
Perfect. Thank you very much for filling that in. So our second lesson today will be repeating yourself. So C++ and DevOps at scale. So with this, we’ve streamlined more powerful lock files, but most importantly, we’ve improved the CI oriented tools. So we’re going to be introducing extensions as well as custom commands and a stable Python API that’s currently released in preview.
So the plugins are definitely something we’re super excited for. This is going to empower users to do a lot of the things themselves. If you watch Diego’s CPPCon talk, this is going to be the Death by a Thousand Bites segment that he did. Absolutely give good giggles if you need one. So the whole goal here is to provide a framework for users to build the solutions tailored to their own needs. So some of these plugins are going to include the profile checker, the command wrapper and package signing, but there’s much more. There’s some of the old ones like hooks, which you’re probably already familiar with, but there’s other new ones. So I had put a poll out on Twitter. I said, “Which ones are you most interested in trying?” I put deployers, custom commands, profile checkers, and a lot of people picked other and replied it to us.
So the deployers, which everybody was super excited about, is a flexible way to extract artifacts from the cache. So this will allow you to automate post-Conan tasks. This is things like making an installer, extracting executables, making docker images. That will be much easier doing it through deployers. So this is no longer in recipes as it was previously done with import syntax. So it’s much more scalable. So you can just do Conan config install and you can install all your own deployers and extensions for them.
So I’ll give you an example of what this looks like. We have the compatibility plugin here for instance. So you can see it’s installed under extensions, plugins, compatibility. And it’s a simple Python file with one method compatibility and it’s very easy, just a few arguments to configure. So this one, for instance, comes with a built-in for different CPPSTDs. So if you’re trying to do binary compatibility between standards, it’s available through this means. And you can tailor this. It’s completely opt-in, so if you didn’t want this, you can easily replace it and change it for your own logic.
Diego Rodriguez-Losada Gonzalez:
The compatibility thing is, for example, this allowed us to address a longstanding problem. For example, the binary compatibility with CPPSTD. The C++ standard, when you activate or deactivated it, it creates different batteries but they are binary compatible in most cases. So default implementation of the compatibility plugin assumes that the different binaries created with different CPPSTD values are binary compatible. And you have the rules there. So that means, if for some reason your binary is compiled with CPPSTD 20 are binary compatible with C++ STD 17, you have control of that. You can go to the compatibility plugin and say, “Hey. My binary here is not compatible.” So it will keep both the efficiency of having to build only one binary with the CPPSTD that you want, and also to control the cases that there might be other incompatibilities and you want to control that. So this is one of the different of the use case. But the typical very requested use case, “Hey. We have some libraries that are binary compatible between [inaudible 00:18:41] versions.” You can put exactly that in your compatibility [inaudible 00:18:44]. So these longstanding questions about the binary compatibility, they are addressed with a compatibility plugin.
Christopher McArthur:
Perfect. Thank you so much for filling that in, Diego. So along with giving you the tools and choices to help and tailor things to your exact use case, we’ve also introduced a new Python API. And as I mentioned, it’s released as a preview, so the plan is to have no breaking changes in the major version, but we do plan on fine-tuning it in the meantime before we get there. The custom commands are built on top of this Python API and they allow you to do anything you want. We also have the JSON outputs, so there is new standardized serialized formats. So the output from the command line can change, but the goal is to provide a stable output from the JSON format so you can build your tooling around Conan.
So for the Python API, I decided to snag this graph from one of Diego’s past presentations to give you an idea of what they’re like. So in 1.X, the Python API was trying to recreate the command line, but this had terrible limitations where you were still locked into the command line which you were trying to get away from. And for 2.0, we decided to give you one level deeper. So you can do remote search, profile, graph, upload, and you can do things like get the profiles, compute the graph, check the integrity of the binaries before you upload them. And this will allow you to tailor things exactly to your use case and your CICD. I used this to do the proof of concept for Conan 2.0 pipeline. If you wanted to have a list of recipes that you wanted to build and you just wanted to read them and build all of them, it’s very easy to do it nowadays, which previously it was a lot of work to code that on a shell script or Python script.
The custom commands, just as friendly, hopefully. You can import the command and add a markup for the methods you want to add. And this is a simple hello world where you get to just print hello world. If you want to learn more about this, Diego’s CPPCon talk, I believe he went over this as well. So that’s definitely a great place for it. Is there anything you’d like to add, Diego, before we move on to our third lesson?
Diego Rodriguez-Losada Gonzalez:
Oh, yeah. Yeah. Custom commands is one feature that we are most excited about it, not only because it will help you a lot to be able to automate, construct high level task with Conan, so Conan becomes a framework that you can build on top of it, but also because it allows us to do more for you. For example, we had recently some questions about “Hey. We are migrating and we need to upgrade our conan.txt files to conanfile.py. It will be nice to have this automated somehow.” I say, “Hey. This is a custom command. You can do a custom command for this.” So actually what we did is, “Okay. Let’s do it. Let’s create a custom command.” So we created this repo here that I’m sharing with you. Probably the team will kill me because of sharing this too early, but this is a proof of concept and we put there a custom command that takes a conanfile.txt and output a conanfile.py. Okay. This is a command that will never land in the Conan code base. It cannot be built in. It’s super very specific for the migration.
And the same will happen for things, things, for example, that are related only to Artifactory or to continuous integration. Something that is not a Conan command but is something that Conan users might find very useful. This would be the perfect framework to build these things and to provide these things. So keep an eye on this repo because they will be adding more functionality. Yeah. And keep an eye on custom commands because this is one of things that can improve your workflows a lot.
Christopher McArthur:
Now I have to ask, can the community open poll requests and contribute their own custom commands?
Diego Rodriguez-Losada Gonzalez:
Yeah. I suggest open an issue first and [inaudible 00:23:14] this is something. The rule will be asking for it. This is something that is useful for the majority of users or for a large part of the users, we will be gladly accepting it. Of course, it is always a trade-off because then we have to maintain it and there will be more questions. So it depends on the complexity and the value for the users. But we are definitely looking forward. So we will be providing a bunch of them ourselves first. And probably with those commands, you will be able to see more or less what is the expectation for this repo, and then we can start to build together on top of that.
But the cool thing is that these repos shouldn’t be like a central sort of truth because custom command is something that you can manage. You can take my custom command, my custom converter, the conanfile.txt to .py converter and you can tune it, you can change it, you can provide more arguments, you can provide more use cases, whatever, and you can manage your command for you very easily. Just forking it, adapting and you install the commands with Conan config install. So this doesn’t aim to be another current extension, this is the base for you to create your own Conan commands and your Conan extensions. So definitely we will accept to request, but this is not idea that it will be centralized exclusively and everything should be there, but users should use it as an inspiration, as a base for building their workflows and their commands.
Christopher McArthur:
It’s a good frame of reference for building your own, which is fantastic. Well, thank you so much for sharing that. So our third lesson is building a dam. C++ are trusted by enterprises worldwide. So there are thousands of teams using Conan and C++ in their workflows, from every size to every industry. And we’ve made every effort to keep that stability and we’re going to continue doing the same thing we did with 1.X with 2.0. So you should have the same thing. I left a comment on my slide that I forgot to remove apparently. So toss it over to Diego with a heart. Is there anything you’d like to add on this one, Diego?
Diego Rodriguez-Losada Gonzalez:
Not specific. I’m one of the that does most customer’s and more user’s talks, video calls and engagements, and I’ve seen things there that you wouldn’t believe. Probably user suffer, some of them at home, but if you don’t… Many, many, many of your colleagues in other companies, they have very, very extraordinary, exceptional, weird requirements, processes, policies and everything. And if you see Chris’ presentation, 2.0 is very focused on accessibility, on trying to provide mechanisms for you to do the things that you want. And this one that is going to present Chris now, is and yet another learning from our users in these enterprises having their own requirements and stuff, that we want to provide a tool for them to be able to integrate the way they want.
Christopher McArthur:
Perfect. Reproducible dependencies and lock files. So this is actually something that I’ve struggled with myself as a Conan user, and the 2.0 lock files are huge, huge game changer for being able to do CICD. So in this small little example I’m going to lay out for you, we’re going to have our game with an engine and our math, and you can see here our requirements are going to be using version ranges. So our game requires engine between one and two, anything between those, and the same is true for our engine. So if we were to now do with Conan 2.0, a Conan install lockout game, we would get our game lock file with math and engine for 1.0. Now, we all have that pesky guy on the math team that we just absolutely delightfully love, who goes and publishes a math 1.1 in the midst of our poll request while we’re waiting for our CICD. And what we want is to make sure our development is not broken or impeded by somebody else’s changes or other improvements that are made in our code base. So if we were to rerun our Conan install for our game with our game lock file now, Conan will correctly resolve the requirements to engine and math, where this previously was not possible with Conan 1.0.
So in Conan 1.X, the lock files were the entire graph, and if the graph didn’t exactly match, this wouldn’t be possible. So with 2.0, this is going to be an absolute game changer for building out CICD and having reproducible builds with version ranges. So you give your engineers the flexibility to pick the best libraries you have, all the while making sure your delivery to customers is unimpeded, which is going to be fantastic. This is the feature I’m most excited to try out personally.
So Conan 2.0 lock files. It’s one lock file for all your configurations. They’re easily mutable, they’re easy to read and understand. It’s just a list of versions to pick from for the graph. There are fully restricted and partial restricted modes. They can be merged together. There are going to be manual commands to override and modify the lock files. And it’s possible to use the same lock file across multiple projects. The code base for lock files is 10 times shorter, which is just absolutely unbelievable that a smaller code base is going to give you this much more flexibility and control. It’s a game changer for CI at scale.
Diego Rodriguez-Losada Gonzalez:
I think I talk for probably most of the audience today here, I think one of the things that make us happier is to remove code from project and this one makes me especially happy. The lock files in Conan one is such a big pain. I won’t [inaudible 00:29:40] that. It is a huge pain, huge pain. The code base is so complicated, so many Conan cases difficult to manage. At the user level, it is challenging and at the development level in the Conan [inaudible 00:29:54] is a nightmare. It’s a nightmare. It was complicated due to historical reasons, is difficult. Necessary learning, it was a very necessary learning but it’s complicated. So I’m super happy that we were able to reduce the code base while improving a lot the functionality of the lock files. So for me, this is one of the thing that will be really a game changer for doing continuous integration.
And many of the users that have been struggling with that, we have test in our code base in the Conan test route now that they implement one of the most challenging problems that you have a graph, you change one package in the middle of the graph and you want to test how this change integrate in the whole graph with lock files, making sure that you lock the rest of the graph except that middle node in the middle of the graph. We have tests for that now in [inaudible 00:30:45]. So it’s not necessarily a full training like we’ve tried in the past with Conan one. So it is now a matter of documenting it, making the examples and writing everything. Probably there will be more webinars or something, but this is super exciting to have it built in and so simple that it’s manageable. Now it’s manageable.
Christopher McArthur:
The lock files that you’re talking about, that example in the use case is actually a demo you did at CPPCon and I did at Core C++. So if you’re interested, you can go check those other videos out on YouTube. So welcoming enterprise DevOps for C and C++. So enterprises can scale, can be ridiculously high, and the domain-specific requirements are exceptionally challenging, right? So continuous integration at scale is critical. And so, this goes beyond thinking about package and dependency management. So you usually say programming over time is software engineering, and that’s a quote we have from Winters, but we’re going to go and say dependency management over time is DevOps, and that’s what this whole buzzword is being about. It’s being able to pick and choose which packages you’re building over time and managing their lifecycle throughout your product’s development.
So conclusion. We have a new graph, new plugins and extensions, new deployers, new binary compatibility. We have a multi-revision cache, package IDs, lock files, new configuration environments, package immutability, and so much more. On the docs, there is a what’s new section. If you want, you can check that out. There is a very long list of features that have been added, changes that have been made, and it’s quite a [inaudible 00:32:35].
So more resources. We mentioned ACCU talk from Diego. I did not add to this slide, but I have to, is our CPPCon talk that we did as well. We have a blog post. If you’re looking for a technical walkthrough of a lot of the features and some more details over what we did today, blog.conan.io is going to be your good first place to go. If you want to get more hands-on, the tutorial section is going to be absolutely fantastic for you.
And of course, if you have questions, you can open them up on GitHub on the main Conan repo, you can ask your questions there as well. That’s the best place if you want official answers. If you have small questions, technical questions, you want feedback on something you’re working on, I highly recommend the Slack channel. There are a ton of active users and they’re always helping each other, and it’s amazing to see them. And if you want to keep up to date, we’ve been making an effort to post more on Twitter. So if you want, you can get more news there as well. So thank you very much. And you can go right now and do pip install Conan and it’ll be version two. Absolutely amazing. Is there anything you would like to add, Diego?
Diego Rodriguez-Losada Gonzalez:
I’m responding to questions here on the chat. Yeah. I don’t know if we want to give the users the word so they can do questions. What is done now, Chris?
Christopher McArthur:
The Q&A, the Q&A. I see there’s one in the Q&A. So if I click the answer live button, it should share the question with you guys. So will there be a recording of the webinar available? Yes. We plan to post this on YouTube. We haven’t decided where exactly. It’ll either be on JFrog YouTube channel or perhaps somewhere else. But when it does get uploaded, we will make sure to share it out on social media, on Slack as well.
Speaker 3:
Yeah. And sorry, to all the attendees, you will receive the recording in 24 hours.
Christopher McArthur:
Ah. Perfect. Thank you. Even better, even easier.
Diego Rodriguez-Losada Gonzalez:
There are a couple of questions here in the chat, I think, that are interesting. For example, regarding the CMake Conan integration, we didn’t upgrade it yet but it is in our backlog. So now we are kind of finalizing the stabilization of Conan, trying to finalize the stabilization of ConanCenter, the migration of the recipes of ConanCenter to be ready for Conan 2.0, to polish some UX things. But the CMake Conan integration is something that will be next. It doesn’t mean that it will be just identical to what it is today in Conan 1.X, but we know that this is use case and you want to have some CMake integration like [inaudible 00:35:34] Conan from CMake. So we’ll be investigating this, proposing solutions, trying to come up with something that is useful and allows to basically solve the same problem. Might not be exactly the same solution but the idea that it solves also the same problem.
Also the question from Jerome about concurrent access to the Conan cache. The Conan cache got several major improvements in Conan 2.0, specifically the ability to host multiple revisions in the cache. Okay. Giving it concurrent access is also on the roadmap. It will probably not be as prioritized as the CMake Conan integration, for example, because it’s performance, it’s a non-functional requirement. It’s basically about performance and resources. So it might wait a little bit. If you want to do multiple parallel jobs in CI, you can use parallel, different caches, and you might rely on the [inaudible 00:36:34] cache to have it bit more performance, especially on the server, and we will eventually… It’s what we call the 2.X roadmap. So making the Conan cache concurrent is in the 2.X roadmap as well.
[inaudible 00:36:51] is asking about, sorry, the CMake [inaudible 00:36:56]. The CMake tool chain is already creating CMake presence. And Conan tool, in our documentation in the tutorial, if you’re using CMake 323 or 322 at least, you can use preset. You type Conan install and Conan will generate preset files for you. You can call them CMake presets and the name of the preset and it’ll work. It’s in the tutorial. So as Chris said, we strongly recommend following the tutorial, checking that, because the presets is already there. You can do it right now. There are questions, Chris, about the [inaudible 00:37:43]. Maybe you want to address those? One is going to be summarized for request, the maintenance, for ConanCenter.
Christopher McArthur:
Yes. So couple of our core contributors to ConanCenter have been asking about ConanCenter. So for those of you who aren’t aware, that’s our central repository for all the packages, as I mentioned earlier. That has been under maintenance mode. So as William mentioned in the chat, there are packages that were broken with V2 and we’re running internal jobs trying to make sure those are running, and we’re hoping to open it back up by the end of the week. So fingers crossed. All of the hardship should be behind us and it’s just the final tweaking to make sure everything’s ready. As you know, building C++ takes time and the number of configurations we do just multiplies that. So it’s slowly unrolling.
We do have some [inaudible 00:38:36] from Q&A. So from Ravi we have, “Is the Conan 2.0 tutorial documentation already up?” Yes. It is. So if you go to docs.conan.io, you’ll be taken to that 2.0. Hopefully you can see my screen, and it’s up here, and you can use that and rely on that as well. We have another one. For example, on lock files, is there going to be more content added? Yes, yes and yes. The documentation is one of those places where we want to put a lot of time and effort. And describing CICD with Conan is going to be one of the most important things, and it’s definitely on the plan in the short term. We have another one.
Diego Rodriguez-Losada Gonzalez:
Yeah.
Christopher McArthur:
Oh.
Diego Rodriguez-Losada Gonzalez:
Yeah. Sorry. I was going to address the question by Timor about, “Is necessary to upgrade server Artifactory for Conan 2.0?” And the answer is no. You can use Conan 2.0 with previous server versions of the server Artifactory, [inaudible 00:39:48] minor but related to the Conan list. When you are listing binaries in the server, there was 500 [inaudible 00:39:58]. That has been fixed in Artifactory 752. Okay. But for typical upload, install and normal operation with Conan, you can use previous versions of Artifactory with Conan 2.0 without any problem.
And also a question that is related also is the recipes. The same recipe might be exactly the same for Conan one and Conan 2.0, but the binaries are different for Conan one and Conan 2.0 because the [inaudible 00:40:30] of the package ID has changed. This is also managed. So when you are listing the binaries in Conan one, it will list you only the binaries built and compatible with Conan one. And when you do the Conan list binaries with Conan 2.0 client, it will only give you the binaries for Conan 2.0. So even if they’re hosted in the same server, in the same repo. Okay. So this is already prepared. So you can create binaries with Conan 2.0 and they will not conflict, they will not interfere in any way with binaries with the Conan one, even if they are in the same repo, exactly in the same place.
Christopher McArthur:
So looking at my Slack conversations, we were running 743 for ConanCenter while doing a lot of the development for Conan 2.0. So almost all of the functionality even in that previous version was available. Just to give you a frame of reference. We have more questions. So from an anonymous attendee. Ooh, spooky. Is there a video that demonstrates how to use our framework? I’m not too sure what you’re looking for specifically in terms of framework. For custom commands, Diego’s talk at CPPCon 2022 is going to be a good reference, he did go over the custom commands and that’ll be an easy one to watch. It should be at the end of the video, if I’m not mistaken. Will the training in the JFrog Academy be updated? I’m not sure. Maybe. Probably not.
Diego Rodriguez-Losada Gonzalez:
Not at the moment, first of all. Creating these videos takes a huge effort and they also become outdated. So we did that for Conan one, now those videos are outdated. So our focus here, what we did is, we converted those training into the tutorial in the docs. So if you go to the docs, the 2.0 docs, the tutorial there is hands-on. It’s exactly the same experience that the academy was provided. Is exercises with code that you can execute. You will see the exact commands that you need to type, the output that you will get, incremental lessons teaching you the concepts, why things are done, how to do things. And the whole tutorial section, which is large, you can run it from the beginning to the end as you could run the academy. So we will first build the knowledge and build the hands-on tutorial in the docs.
And maybe by the end of the year, something like that, if things are way stable and there is still enough demand for video format, then we might consider to go and modernize the videos for the academy. But it will be probably just a parallel video format of the tutorial in the docs.
Christopher McArthur:
Perfect. I’m sorry to Herald, your question is very long, so I’m avoiding it at the moment, but if we have time, I will get to it. Renee asked, “Will there be a Conan one Python API to Conan 2.0 migration/translation reference?” So if you go to the Conan one docs, there is a Conan migration guide to 2.0, and this should have a pretty detailed, in-depth version of everything you need to know to make that migration, especially in terms of the recipes. So I hope this is going to be a good reference. There are definitely pieces of this that are missing.
My gripe, when I was learning it, is the command lines. Not all the command line changes are documented, but if you see anything missing, feel free to open up a poll request, open up an issue, and we’ll try to add it in as much as possible. Again, 80% of the code base is different, so having the documentation for all of that is going to be something that comes over a little bit of time. There’s five years of Conan one docs that need to be migrated, and that’s not going to happen overnight unfortunately.
Diego Rodriguez-Losada Gonzalez:
Yeah. I think I might give some hints to Herald regarding the migration. Definitely, when you are doing a major upgrade of something, it takes some effort. So by recommendation, to be as prepared as possible, little by little, step by step in Conan one. Do not even propose, “Hey. Let’s upgrade to 2.0.” No. Don’t do it. There will be time. Okay. There will be time because community needs to migrate, we need to finalize immigration of ConanCenter as well. So the idea is to become as ready as possible in Conan one by doing incremental changes. For example, A, let’s make sure that we are using CMakeDeps depth instead of the legacy CMake find package generator, and that’s [inaudible 00:45:40] that can go incrementally from the top dependencies to the bottom one with small batches of the port. It doesn’t need to be a huge investment in, “Hey. Let’s stop everything and for the next two months just do this.” And being prepared with the recipes and having the recipes ready for 2.0 is probably, I don’t know, 80% of the effort.
The common line has changed. Yes. It has changed. But at the end of the day [inaudible 00:46:10] Conan create. Conan create. still works. Conan install. still works. Conan lock create still works. So the format of the lock file might be different but the concepts are not that different. It’s install, create. The inspect is there, slightly changed. The search has been mapped to Conan list, but on the other hand, also the commands are way more powerful. So the way that the revisions are listed, the package IDs are listed, and the JSON output and everything, is much better compared. So the command might have changed, but changing that on a pipeline is not a huge effort, is not like having all the recipes prepared.
So my recommendation, clearly, is work in Conan one, make sure that the recipes are as updated as possible [inaudible 00:47:05] ready as possible with the migration guide and with the syntax, and everything. And then the last mile, at some point if you have, “Hey. We really want to use the lock files or these custom commands will be a frame whether we can leverage and that will be really worth,” then there is a value proposition for management. “Hey. We have this, we will be able to scale better, we will be able to build more efficiently, our products, to use less continuous integration power. We will save time and money if we migrate this because the lock file and the CI is more efficient in 2.0. And then we can use a few weeks to make sure that the pipeline and the commands and everything is migrated correctly.”
Christopher McArthur:
Perfect. I will probably cut that and post it on Twitter as a beautiful description. So we had a couple questions. I saw one about the Conan cheat sheet. There is a task in my calendar to work on it. It’s scheduled in two weeks from now because we’ve been exceptionally busy with the Conan 2.0 launch, but we are definitely planning on working on it. There have been questions about basal support. Julius answered those. Thank you. We do have a few more minutes. So I’ll try to answer one or two more questions. Is there an idea where to store new lock files? Will Artifactory be able to store the lock files together with the binaries? So usually these lock files are something you would check into in your source code. So these usually live alongside your project.
Diego Rodriguez-Losada Gonzalez:
But [inaudible 00:48:49] that, you have to be careful there because committing things to the source, it also creates new commits. If you get new commits, it might change also your SCM, it might change your recipe revision. If it is a consumer project, if it is your final application that you’re [inaudible 00:49:06] with Conan install and [inaudible 00:49:07] dependencies, you can perfectly put the lock file there. Okay. For someone in the middle of the graph, you need to check which approach you are using for revisions and you want to check that your lock file is not affecting that. Otherwise, probably lock files, the greatest thing about is that only with one file you can manage tons of things, all the configuration, everything. So managing that is way easier than in the past.
We have also started experimenting with putting the lock files inside the recipes in the server. Okay. Something similar to the string wrap in MPM. If anyone knows MPM, the string wrap JSON, which is the reflect of the lock file in MPM lock file, it’s possible to do the same. It’s an experiment and it is something that we will leave probably something for the next year. In the meantime, probably having [inaudible 00:50:11] repo, that you put the lock files there in a branch that follows the convention of the poll request or of the flow that you are processing at the moment, probably is the best approach. It’s a little bit easier to manage that for developers that put in the lock files in a server that you need to download and upload.
Christopher McArthur:
In GitHub Actions, you can upload artifacts and then it stays across the build. That’s been my solution for 1.X, but-
Diego Rodriguez-Losada Gonzalez:
Oh, yeah. Exactly. You can also leverage the CI mechanisms but if you want to, let’s say, store it somewhere that is permanent there outside of the CI, probably a GIT repo is a bit more convenient than a server repo.
Christopher McArthur:
Very good call. So you have two options depending on your different workflows.
Diego Rodriguez-Losada Gonzalez:
Jackie is asking about if there are features that are not ported to 2.0. And, yes, I think they are reflected there in the migration guide. Something like the imports. Imports doesn’t exist anywhere in 2.0, but you can already migrate to something that makes sense in Conan one. That is the generate. So whatever imports is executing, you can write that in a generate method with more control of the imports. Okay. Imports was very, let’s say, restricted. So whenever you wanted to do something outside of the [inaudible 00:51:39] with the imports, it was not possible. While in the generate method, you can do whatever you want. The copy that you can do or the gathering of information that you can do is way more. So you can already write alternatives in Conan one that will replace things like the imports, and those things will move into 2.0 without problems. And also in 2.0, they have replacements or better alternate like deployers. So deployers is a better substitute for the imports in Conan one. But Conan 2.0 doesn’t have imports anymore. All the legacy generators like CMake find package or [inaudible 00:52:21], they have been removed in Conan 2.0 as well.
Christopher McArthur:
Almost all of the features I talked about today are not ported to 1.X. So all the new features are exclusives of 2.0. The compatibility is really between the actual recipes themselves. So there’s a syntax that you can use for the same recipes in both, but the features that are added in 2.0 are going to be exclusively there, and some of them are not possible to backport. I don’t think any of them would be. So changes to the graph, for instance, can’t be backported. So hopefully that answers your question.
Diego Rodriguez-Losada Gonzalez:
Alessandra’s asking about another cache feature, which is the retention, and this we provided in the Conan cache database, we have [inaudible 00:53:10]. There is a few that is the least recently used timestamp. Okay. So if there is some infrastructure already provided there to be able to implement the cache retention. This is also a Conan 2.X roadmap feature. Being able today, Conan cache clean everything that hasn’t been used in the last four weeks, for example. Yeah. Now we have the right infrastructure to develop that and it’s just a matter of [inaudible 00:53:42]. So I think that we will see this feature landing in the probably… I don’t know. In this next six months or something like that. Not already now ’cause this is not a critical [inaudible 00:53:51] performance thing, but it is there, it’s in the backlog.
Christopher McArthur:
There are a lot of things on the 2.X roadmap. And the team is in the process of trying to figure out a priority list. So one of the things that I always ask people to be mindful of is, the team is limited in size and there’s only so much we can do at once. There’s like a million and one things we want to do. I have my own list that I’m pushing for for myself because I know other people like me could benefit from them, but we try to make the tough decisions, unfortunately, for what’s best for everybody. And if we don’t get to the features you’re asking for right away, just be mindful that we’re listening to a lot of people, right? There’s tens of thousands of users using Conan every single day. And making the right decision for everybody is what we try to do. So just be mindful, be patient with us. We’re going to get there, but it’s going to take some time.
Diego Rodriguez-Losada Gonzalez:
We totally feel the pain when you ask CMake Conan, something, we know it. And we know that you think for you it’s kind of critical, but maybe this is critical for 10% of the user base and there is something more critical for 20% of the user base that is maybe not a thing for you but might be a thing for them. And sometimes your requirements are not even less prioritized than others. They are contradicting other user requirements. So say, “Hey, Conan team. Your package ID is too restrictive. It’s asking [inaudible 00:55:27] all the time.” And other thing, “Hey. Your package ID is too relaxed. It should be more restrictive.” Okay. So at the end of the day we try to provide means so you are able to configure that.
Yeah. Someone is asking, “You need to check in the 2.0 roadmap?” You can check the 2.X milestone in GitHub. This is where we put the things. So if you go to milestones in GitHub, then you see there how we prioritize. So the 2.X roadmap is basically the backlog, the value of relatively important features for the following month. But you can also see what is the next minor or the next patch that we already prepared there. You can see there are already 34 issues closed there. So this will be probably this week or beginning of the next week. You can see the next minor 2.1 and you can see the midterm roadmap like 2.X.
Christopher McArthur:
Get back to work. Thank you, Paul. You’re amazing.
Diego Rodriguez-Losada Gonzalez:
Yeah. Yes, Paul. Right now we will close the call and we will get back to those three tickets in 2.0.1.
Christopher McArthur:
Yeah. You guys aren’t allowed to go to sleep. For those of you who are joining us from Australia that I can see that are online, thank you so much for staying up late. Diego is up late as well, working late, and I woke up early. So we have the four corners of the earth covered, which is fantastic. So with that, I know we didn’t get into all the questions and I’m sorry, we’re going to wrap up the call here. Thank you guys so much for joining and asking your questions. This has been super helpful for everybody, I’m sure. If you have any more questions, Twitter, GitHub, Slack are great places. So I really hope you guys appreciate this and you join us for the next one.
Diego Rodriguez-Losada Gonzalez:
Yeah. Thank you, everyone. It was a pleasure to have you all here today.
Christopher McArthur:
Goodbye, everyone. Have a good afternoon, evening. Thank you for joining us.
Diego Rodriguez-Losada Gonzalez:
Cheers.
Speaker 3:
Bye.