Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added support for Conan as a dependency manager #1479

Closed
wants to merge 28 commits into from
Closed

Conversation

jellespijker
Copy link
Member

@jellespijker jellespijker commented Aug 8, 2021

This PR is part of the following PRs:

The purpose of these changes is to set up a dependency manager for Cura and her repositories. Cura uses both third-party and Ultimaker maintained dependencies, written in both Python and C++ (or mixtures of both). Not all of these dependencies can be downloaded with the help of a dependency manager such as pip. This makes setting up Cura from source, a pain in the $%^#$$%^. See the graph below for the current dependencies. Adding to the complexity is the way how we're currently consuming third-party dependencies; Some have to be present on the system/provided by the user, some are shipped within the repo, while others are downloaded by CMake.

dep graph

All of the above-mentioned PRs and this one, add a conanfile.py to the root of this project. This is a recipe written in Python which instructs Conan (https://docs.conan.io/en/latest/) how to build and package the repository in such a way that it can be reused by other dependencies. If a required dependency has no binary for your OS and compiler it will build that dependency from scratch and store it in the cache. Making installing Cura from source as simple as:

conan install Cura/4.10.0@ultimaker/testing --build=missing

For a more detailed description see the README.md in this repository https://github.com/jellespijker/conan-um

For testing purposes, I have set up a small home server that can be used by Ultimaker employees. Other developers can test this by cloning the above-mentioned repositories and performing a conan export . ultimaker/testing in each root. That only leaves the SIP package, if you execute a conan export . riverbankingcomputing/testing in this folder https://github.com/jellespijker/conan-um/tree/main/recipes/sip it creates a Conan package for SIP 4.19.25

You can use your own profiles for this, but I have personally tested and developed them with my own jinja template profiles on Linux Manjaro with a GCC compiler, Mac OS Big Sur with a Clang compiler and Windows 11 with a Visual Studio 2019 compiler.
These profiles can be installed with the conan config install https://github.com/jellespijker/conan-config. Make sure you add -pr:b cura_release.jinja -pr:h cura_release.jinja to your install instructions

Conan allows for multiple ways of working. Either the exiting package can be used from the cache, or if you want to work on multiple repositories you can put that repo in editable mode such that the xxx-config.cmake in the project that is depending on the other, will point to the paths of your repo, see https://docs.conan.io/en/latest/developing_packages/editable_packages.html

Because the best practice method to use Conan, which is also the preferred way in Conan 2.0, is to use ( https://docs.conan.io/en/latest/reference/conanfile/tools/cmake.html ) With the tools CMakeDeps, CMakeToolchain, and CMake. The CMakeDeps class will generate xxx-config.cmake files per dependency, while the CMakeToolchain will generate a toolchain to be passed to CMake -DCMAKE_TOOLCHAIN_FILE=conan_toolchain.cmake These two prepare the way for CMake to actually build the project such that it won't need to change the CMakeLists.txt. Allowing Conan to be optional and not mandatory. Unfortunately, our repositories have organically grown CMakeLists.txt mixing old and modern syntax and a build system that uses CMake 3.13.

I decided to harmonize our CMakeLists.txt because often they were inconsistent, for instance how we set the fPIC or MD/MDd and MT/MTd flags. Often I had to changes these methods in one repo, while it already worked in another repo. Getting stuff to work on three different OSs in three different languages is a bit time-consuming, to say the least. These uniform methods of preparing CMake instructions can be found in cmake/StandardProjectSettings.cmake. These are tried and tested methods that we already use in our Spatial plugin and are used by Json Turner in this https://github.com/lefticus/cpp_starter_project and are based on the best practices described here: https://github.com/lefticus/cppbestpractices

The changes in CMake basically boil down to:

  • StandardProjectsSettings.cmake
    • Set all compiler warnings known to mankind.
    • Prefer BUILD_SHARED_LIBS over BUILD_STATIC but no change in the interface so no change is needed on
      the build servers.
    • Set build_type to Release if not specified
    • Export compile commands to a json for easier debugging
    • Use position-independent code if applicable (-fPIC)
    • Switch between MT/MTd of MD/MDd for Visual Studio using text generators
    • Set C++17 standard
    • Set stdlib=libc++ for clang-apple
    • Add option ENABLE_BUILD_WITH_TIME_TRACE for generating time tracing json on Clang compilers
    • Created a uniform method of using threading which prefers pthread if the platform supports it and has multiple
      threading libraries installed.
    • ENABLE_CPPCHECK "Enable static analysis with cppcheck" default OFF
    • ENABLE_CLANG_TIDY "Enable static analysis with clang-tidy" default OFF
    • ENABLE_INCLUDE_WHAT_YOU_USE "Enable static analysis with include-what-you-use" default OFF
    • ENABLE_COVERAGE "Enable coverage reporting for gcc/clang" default OFF
    • ENABLE_SANITIZER_ADDRESS "Enable address sanitizer" default OFF
    • ENABLE_SANITIZER_LEAK "Enable leak sanitizer" default OFF
    • ENABLE_SANITIZER_UNDEFINED_BEHAVIOR "Enable undefined behavior sanitizer" default OFF
    • ENABLE_SANITIZER_THREAD "Enable thread sanitizer" default OFF
    • ENABLE_SANITIZER_MEMORY "Enable memory sanitizer" default OFF
    • Disallow in-source builds by default but added the option ALLOW_IN_SOURCE_BUILD, some build systems such as
      Conan copies the source and builds from there.
    • Enable IPO/LTO https://en.wikipedia.org/wiki/Interprocedural_optimization
  • Use targets in CMake consistently
  • Added targets to the shipped Find-modules CMake
  • Moved shipped Clipper and RapidJSON a directory up so they include behave the same as when they are imported from
    the system.
  • Created a static library from the shipped Clipper which links to _CuraEngine and the relevant tests.
  • updated .gitignore with cmake-build-* and tmp
  • Added clang-tidy and clang-format (which can be used by CMake or most IDE's. Same ones that are used for the
    SpatialBackend these are probably still a bit incomplete at the moment. But if we update them after each comment
    during a review, it will get more and more helpful.
  • Moved CMake testing logic to it's own folder

The above-mentioned CMake changes should keep the build interface the same for existing environments (only add extra options) Now you guys might get a bit discouraged when you see all of those warnings scroll up when building CuraEngine. We can opt to disable it. But I personally am strongly in favor of keeping these visible as a motivation to clean up our codebase each time we're working in a certain section.

I already noticed the added value of these warnings yesterday when it tripped over new code in the AngleDegree class. The old operator+ only allowed for int additions. the Wpedantic warning flag throws an error over this. Added a template type deduced operator+/- that performs a static_cast to a double. This should enable the use of all arithmetic types with AngleDegrees for the + and - operator.

Still, WIP at the moment, since I'm finalizing some last changes across all repos

- Added a conanfile.py for Conan dependency managing
- Use StandardProjectSettings (also used in libArcus)
  - Set all compiler warnings known to mankind.
  - Prefer BUILD_SHARED_LIBS over BUILD_STATIC but
    no change in interface so no change needed on
    the build servers.
  - Set build_type to Release if not specified
  - export compile commands to a json for easier debugging
  - Use position independent code if applicable (-fPIC)
  - Switch between MT/MTd of MD/MDd for Visual Studio using
    text generators
  - Set C++17 standard
  - Set stdlib=libc++ for clang-apple
  - Add option ENABLE_BUILD_WITH_TIME_TRACE for generating
    time tracing json on Clang compilers
  - Created an uniform method of using threading which prefer
    pthread if the platform supports it and has multiple
    threading libraries installed.
- Use targets in CMake
- Added targets to the shipped Find-modules CMake
- Moved shipped Clipper and RapidJSON so the includes behave
  the same as when they are imported from system.
- updated .gitignore with cmake-build-* and tmp
Making the main CMakeLists.txt more readable
The old operator+ only allowed for int additions.
the `Wpedantic` warning flag throws an error over
this. Added a template type deduced operator+/-
that performs a `static_cast` to a `double`.
This should enable the use of all arithmetic types
with AngleDegrees for the + and - operator.
The MT/MD flags were also set for non MSVC compilers
Bad practice don't do that!
You can now set the following extra options (default is OFF)
- ENABLE_CPPCHECK "Enable static analysis with cppcheck"
- ENABLE_CLANG_TIDY "Enable static analysis with clang-tidy"
- ENABLE_INCLUDE_WHAT_YOU_USE "Enable static analysis with include-what-you-use"
- ENABLE_COVERAGE "Enable coverage reporting for gcc/clang"
- ENABLE_SANITIZER_ADDRESS "Enable address sanitizer"
- ENABLE_SANITIZER_LEAK "Enable leak sanitizer"
- ENABLE_SANITIZER_UNDEFINED_BEHAVIOR "Enable undefined behavior sanitizer"
- ENABLE_SANITIZER_THREAD "Enable thread sanitizer"
- ENABLE_SANITIZER_MEMORY "Enable memory sanitizer"
Same ones that are used for the SpatialBackend
These are probably still a bit incomplete at the
moment. But if we update them after each comment
during a review it will get more and more helpful.
I hate failures on the build server! Only way
to see if things workout is to commit.
Created a library for Clipper not just
including the directory
The test is probably failing on this.
Which isn't caught localy since Conan
adds this flag automatically when you
use Arcus as a dependency.
Strongly discouraged but some automated systems
use this method. This can be enabled by setting
ALLOW_IN_SOURCE_BUILD ON
jellespijker and others added 10 commits August 8, 2021 17:36
We currently don't support openMP on Mac. However with
Conan we could use the llvm-openmp package to add support
for this. Out of scope for now.
This allows it to be used by Cura
Somehow it passed the function instead of the value
when called upon
Not relevant for CuraEngine, but I want to keep
the StandardProjectSettings.cmake the same across
repo's
# Conflicts:
#	src/raft.cpp
#	src/utils/Coord_t.h
#	src/utils/IntPoint.h
#	src/utils/polygon.h
@jellespijker jellespijker deleted the conan branch November 12, 2021 14:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant