Support for different distributions filesystems #12
Comments
TechSupportJosh
added
enhancement
|
Some ideas for multiple filesystem implementation File structure/etc/netkit
json files could be in some other format, but just something with filesystem / kernel name, version, url (from github release page) etc. For filesystems we could have something like 'compatible kernels' - not that its likely people will be swapping them around but it shouldnt be too difficult to add. The official filesystem and kernel files could be downloaded from github so that new filesystems are available before a new release of netkit core. just a quick example: $HOME/.netkit-jhfilesystemsThis could contain downloaded filesystems, custom filesystems and a json file for specifying custom filesystems. E.g. you've built your own for a particular purpose, or you want to define the path to filesystems build for dev purposes, so you can quickly test with kernelsThis could contain downloaded kernels, custom kernels and a json file for specifying custom kernels. The custom json files for filesystems and kernels could be similar to the official ones in /etc/netkit but allow filepaths in addition to urls. vstartWe would want vstart to be able to take options for --filesystem and --kernel. This would then need to look to see if the fs or kernel is downloaded to the $HOME/.netkit-jh directory and compare to the hash. If it is then all good, add those to $KERNELCMD, if not, attempt to download the filesystem (or exit and tell the user to download the fs?). lstartNeed to be able to take --filesystem and --kernel from lstart and pass that to vstart. Also lstart might be a good place to check if the official kernel and filesystem json files in /etc/netkit are up to date with the latest on github. lab.confWould be useful to have an option for new command for managing fs and kernelsnot sure what this one would be called but something where you can download and delete downloaded filesystems and kernels. This should also just have an easy update option for those who dont care about custom fs / kernels, to just delete the current stuff and download the latest of debian. Release VersionsThis could be done a bit like docker tags, where the default is CoreI'm not sure where is best to install to the core tools to, but probably somewhere that is in the $PATH by default - usually when you install something you dont have to update your bashrc / zshrc to reflect that. When you install / update netkit, you should be installing the core tools and configuration - and these should handle updates in filesystems and kernels, rather than the releases being baked together. This means when theres a new filesystem or kernel its immediately available, or when theres an update to core, people dont need to reinstall filesystems. These are just some ideas i've come up with that might be worth considering when thinking about multi-filesystem support. By not having large filesystems and kernels built into the package it might be possible to consider adding netkit to package repositories, e.g. the AUR. |
|
From conversating with @b177y and @AdamBromiley, our current plan of action is this:
For each kernel or filesystem, we will have a tag associated with it in this format: Therefore, if you wanted to download v3 of Ubuntu Focal, you can specify RELEASE_VERSION is where we release patches to the same FS or kernel version. For example, if we fix a bug in the 5.10.4 kernel, RELEASE_VERSION will increment. This is because we may fix multiple things at different times for the same kernel version, therefore relying on just the kernel version is not sufficient. Each entry within the releaseinfo looks like this (the format allows us to parse in bash easily) Description of each parameter:
Filesystem: Description of each parameter:
When files are downloaded (using the tools described later on), they will end up looking like this within the install directory: In order to manage the downloading of these filesystems and kernels, two new tools will be created.
Description of each of the commands:
|
|
|
|
Added metadata files to each release as well as symlinks to latest, which will be used when not specifying a specific version. The latest symlinks will be updated when installing a new fs/kernel. If the one referenced by latest is older than the one currently being installed, the latest symlink will be removed and recreated. |
|
Installed filesystems / kernels will be listed under installed-fs / installed-kernel. It will feature a somewhat similar format to releaseinfo: This then allows us to do uninstall commands, by deleting the DIRECTORY tag, possibly updating latest -> and delete the entry. This will also allow us to keep track of fs/kernels installed using local files, rather than retrieving them from our file server (e.g. for development, something like |
|
Now we're using python for netkit-fs/kernel, releaseinfo will look something like this: {
"arch": {
"rolling": {
"v1": {
"min_core": "1.2.0",
"max_core": "1.7.0",
"min_kernel": ["linux:5.10.4:v3", "linux-hardened:5.10.4:v6"],
"max_kernel": ["linux:5.11:v1", "linux-hardened:5.11:v1"],
"download_url": "https://meme.com/arch/rolling/v1/fs.tar.bz2",
"sha256_hash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
}
}
},
"debian": {
"bullseye": {
"v1": {
"min_core": "1.2.0",
"max_core": "1.7.0",
"min_kernel": ["linux:5.10.4:v3", "linux-hardened:5.10.4:v6"],
"max_kernel": ["linux:5.11:v1", "linux-hardened:5.11:v1"],
"download_url": "https://meme.com/debian/bullseye/v1/fs.tar.bz2",
"sha256_hash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
},
"v2": {
"min_core": "1.2.0",
"min_kernel": ["linux:5.10.4:v3", "linux-hardened:5.10.4:v6"],
"download_url": "https://meme.com/debian/bullseye/v2/fs.tar.bz2",
"sha256_hash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
}
},
"jessie": {
"v1": {
"min_core": "1.2.0",
"min_kernel": ["linux:5.10.4:v3", "linux-hardened:5.10.4:v6"],
"download_url": "https://meme.com/debian/jessie/v1/fs.tar.bz2",
"sha256_hash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
}
}
}
} |
Describe the solution you'd like
It would be great if we could use different distributions for the Netkit VMs rather than solely Debian. This could allow us to do something like this inside lab.conf files:
This would then mean machineA uses Arch for it's filesystem, machineB uses Ubuntu for it's filesystem and machineC uses the traditional Debian.
Our current build method uses debootstrap to build a Debian filesystem. debootstrap also has support for Ubuntu and therefore wouldn't be too much hassle changing. Arch uses pacstrap, which is very similar to debootstrap, making it a very viable option.
Implementation steps
We will also need to take care about not making updating Netkit really difficult. Wherever possible, we need to limit the number of changes and ensure they're well documented and therefore maintainable. This process will become easier when we have proper testing of Netkit releases.
The text was updated successfully, but these errors were encountered: