Add flake.nix to SRI #1344
Replies: 7 comments 10 replies
-
|
as a nix user, I find this idea quite interesting, but I feel this requires a bit more brainstorming first and foremost, we cannot force any SRI contributors to start using nix... it has a steep learning curve and it can actually slow down productivity for those who are not used to it therefore, it's very important to have the following restrictions in mind:
second, the original proposal feels very broad and generic, and a few questions come to mind:
again, as a nix user I like the idea and I'm not opposing it but overall if we were to go in this direction, it's important to have a solid rationale! |
Beta Was this translation helpful? Give feedback.
-
CI pipelines is one example of something from @Shourya742 original proposal that I feel would not be a good idea as much as I agree it would be extremely powerful to build a nix-based CI, that would become a blocker for non-nix-using contributors to maintain the CI as stated above, we cannot force everyone to become nix users, and therefore the CI must remain as un-opinionated as possible |
Beta Was this translation helpful? Give feedback.
-
another example of something that would not be desirable we must not use nix as an assumption or pre-requisite for onboarding |
Beta Was this translation helpful? Give feedback.
-
|
@Georges760 I see you liked this issue, and you also mentioned you use a can you share your why are you using it? which problems are you solving? |
Beta Was this translation helpful? Give feedback.
-
because I switched to NixOS
not having to install system-wide any toolchain/tools specific version for a specific project that may conflict with other toolchain/tools version for other project, it is very very efficient on that regard |
Beta Was this translation helpful? Give feedback.
-
Of course, here is my shell.nix : { pkgs ? import <nixpkgs> {} }:
let
overrides = (builtins.fromTOML (builtins.readFile ./rust-toolchain.toml));
libPath = with pkgs; lib.makeLibraryPath [
# load external libraries that you need in your rust project here
];
in
pkgs.mkShell rec {
buildInputs = with pkgs; [
clang
cmake
openssl
# Replace llvmPackages with llvmPackages_X, where X is the latest LLVM version (at the time of writing, 16)
llvmPackages.bintools
pkg-config
rustup
];
RUSTC_VERSION = overrides.toolchain.channel;
# https://github.com/rust-lang/rust-bindgen#environment-variables
LIBCLANG_PATH = pkgs.lib.makeLibraryPath [ pkgs.llvmPackages_latest.libclang.lib ];
shellHook = ''
export PATH=$PATH:''${CARGO_HOME:-~/.cargo}/bin
export PATH=$PATH:''${RUSTUP_HOME:-~/.rustup}/toolchains/$RUSTC_VERSION-x86_64-unknown-linux-gnu/bin/
'';
# Add precompiled library to rustc search path
RUSTFLAGS = (builtins.map (a: ''-L ${a}/lib'') [
# add libraries here (e.g. pkgs.libvmi)
]);
PKG_CONFIG_PATH = "${pkgs.openssl.dev}/lib/pkgconfig";
LD_LIBRARY_PATH = libPath;
# Add glibc, clang, glib, and other headers to bindgen search path
BINDGEN_EXTRA_CLANG_ARGS =
# Includes normal include path
(builtins.map (a: ''-I"${a}/include"'') [
# add dev libraries here (e.g. pkgs.libvmi.dev)
pkgs.glibc.dev
])
# Includes with special directory paths
++ [
''-I"${pkgs.llvmPackages_latest.libclang.lib}/lib/clang/${pkgs.llvmPackages_latest.libclang.version}/include"''
''-I"${pkgs.glib.dev}/include/glib-2.0"''
''-I${pkgs.glib.out}/lib/glib-2.0/include/''
];
}It is a little bit messy, as I am on NixOS for few month only, and still learning it. |
Beta Was this translation helpful? Give feedback.
-
|
thanks for sharing @Georges760, it's interesting to get some insight into how some contributors are leveraging nix for their workflows I assume your I don't have any guesses for what which brings me to another question: are you using this file for other development workflows, or exclusively for SRI? in case we decided to move forward with adding some (but this is just a general comment, I see you mentioned the |
Beta Was this translation helpful? Give feedback.
-
|
that's true, I kind of grown my shell.nix file while using it on different Rust crate dev. the cmake/clang/bintools/openssl where added when I had to deal with some crates that either do some FFI/bindgen stuff or depend on FFI/bindgen crates, so not strictly during my contribution on SRI/no_std effort on protocols/sv2 because none of them does or depend on FFI/bindgen. But there are many other crates in SRI thay may do or depend on FFI/bindgen, so we may need them at some point. FYI, the base instruction I used for my shell.nix was there : https://nixos.wiki/wiki/Rust And up to now, I only used shell.nix, but I would love to switch to flakes soon. |
Beta Was this translation helpful? Give feedback.
-
|
may I ask why are you using nix? what kind of problems it solved for you while working on SRI? |
Beta Was this translation helpful? Give feedback.
-
|
I'm going to answer your question with a broader context by sharing my nix journey. And then I'll add another comment about SRI specifically. Before starting blockchain development, I used to do system integration for embedded systems, where highly deterministic and reproductible builds were a very important factor. In that universe, we start from source code and cross-compile package distributions for multiple target platforms, which sometimes have exotic architectures and toolchain management can be a pain in the ass if you're not using the right tools. So in 2021, while in a previous shitcoin job, I went down the nix rabbit hole. The project I was working on had a nix culture, which motivated me to start learning. There was nothing too powerful though, it mostly revolved around using some basic I briefly attempted to switch over to nixOS but the learning curve was bringing my productivity down, so I switched back to ubuntu. Later in 2023 I started using a MacBook as my main workstation, which allowed me to give nixOS another try as my secondary workstation (which wouldn't block my productivity). That helped me slowly evolve into more average to advanced knowledge of nix. As I gave up on shitcoin life, the Bitcoin ecosystem further motivated me to continue exploring this universe. A few projects come to mind in this context:
I know that at some point in time @Fi3 used also Guix for some parts of his development workflow for SRI. If you Finally, in 2024 I started making some contributions to the nix/bitcoin ecosystem, namely:
|
Beta Was this translation helpful? Give feedback.
-
|
now, brainstorming a bit on how nix could be leveraged to boost productivity on SRI development: aside from the low-hanging-fruit of deterministic toolchain management, I can also imagine some interesting workflows for live testing setups for example, whenever I want to test SRI on a live setup (meaning: some TP connected to it's kind of a boring task where I manually open multiple terminals and start the different processes. this is also assuming I already have a built TP (which I usually do via a one could argue that this can also be automated by simple shell scripting, but that's the beauty of nix: all the dependency management (e.g.: cmake, pkg-config, rustup, cargo, etc) would be constrained to this nix environment, and it would deterministically work on macOS, linux, x86-64, aarch64 and whatever flavors we wanted to support obviously this would not be a replacement for our Integration Tests, which are crucial for CI automation of pure-Rust testing in regtest setups but sometimes live setups are still useful (and even important), for example if we want to make tests with beefy hashpower from real ASICs |
Beta Was this translation helpful? Give feedback.
-
I was bored to lose (again) my whole ubuntu system at a "dist-upgrade", so wanted to test NixOS. I kind of like it now, begining was difficult (like Rust) but once everything is running, it is very convenient to maintain. |
Beta Was this translation helpful? Give feedback.
-
|
another relevant point is that using nixOS is not a pre-requisite for using nix it can still be installed into regular linux distributions and macOS nixOS is simply an entire linux distribution based on this tool |
Beta Was this translation helpful? Give feedback.
-
|
very true |
Beta Was this translation helpful? Give feedback.
-
|
my rant above actually made me realize something that is kind of adjacent to this discussion: do we want to maintain the legacy Guix things on the SRI repo? here's a quick search for |
Beta Was this translation helpful? Give feedback.
-
|
@Fi3 any compelling reason to keep this around? |
Beta Was this translation helpful? Give feedback.
-
|
@Fi3 over Discord:
|
Beta Was this translation helpful? Give feedback.
-
Add a
flake.nixfile to the repository to standardize and simplify the development environment setup using Nix flakes. This will ensure consistent dependency management, reproducible builds, and alignment across different systems and contributors. By introducingflake.nix, we can provide a unified configuration for tools, Rust toolchains, and any project-specific dependencies, streamlining the onboarding process and minimizing environment-related issues. The flake should define the required Rust version (aligned withrust-toolchain.toml) and any additional tools or components needed for development, testing, and CI pipelines.Beta Was this translation helpful? Give feedback.
All reactions