Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for JetPack 6.1 build #3211

Merged
merged 11 commits into from
Oct 17, 2024
Merged
Prev Previous commit
Next Next commit
add the documentation for jetpack6.1 build
  • Loading branch information
lanluo-nvidia committed Oct 5, 2024
commit 4fed4df46c33cefdc11652fec375fd57d0928783
61 changes: 0 additions & 61 deletions .github/scripts/build_jetson_6.1.sh

This file was deleted.

109 changes: 109 additions & 0 deletions docsrc/getting_started/jetpack.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
.. _Torch TensorRT in JetPack 6.1

Overview
##################

JetPack 6.1
---------------------
Nvida JetPack 6.1 is the latest production release ofJetPack 6.
With this release it incorporates:
CUDA 12.6
TensorRT 10.3
cuDNN 9.3
DLFW 24.0
lanluo-nvidia marked this conversation as resolved.
Show resolved Hide resolved

You can find more details for the JetPack 6.1:

* https://docs.nvidia.com/jetson/jetpack/release-notes/index.html
* https://docs.nvidia.com/deeplearning/frameworks/install-pytorch-jetson-platform/index.html


Prerequisites
~~~~~~~~~~~~~~


Ensure your jetson devleoper kit has been flashed with the latest JetPack 6.1
You can find more details on how to flash Jetson board via sdk-manager

* https://developer.nvidia.com/sdk-manager

.. code-block:: sh
# ensure the current jetpack version is 6.1
apt show nvidia-jetpack


Ensure you have installed JetPack Dev components
This step is required if you need to build on jetson board
You can only install the dev components that you require
ex: tensorrt-dev would be the meta-package for all TRT development
or install everthing

.. code-block:: sh
# install all the nvidia-jetpack dev components
sudo apt update
sudo apt install nvidia-jetpack


Ensure you have cuda 12.6 installed(this should be installed automatically from nvidia-jetpack)

.. code-block:: sh
# check the cuda version
nvcc --version
# if not installed or the version is not 12.6, install via the below cmd:
sudo apt update
sudo apt install cuda-toolkit-12-6


Ensure libcusparseLt.so exists at /usr/local/cuda/lib64/

.. code-block:: sh
# if not exist, download and copy to the directory
wget https://developer.download.nvidia.com/compute/cusparselt/redist/libcusparse_lt/linux-sbsa/libcusparse_lt-linux-sbsa-0.5.2.1-archive.tar.xz
lanluo-nvidia marked this conversation as resolved.
Show resolved Hide resolved
tar xf libcusparse_lt-linux-sbsa-0.5.2.1-archive.tar.xz
sudo cp -a libcusparse_lt-linux-sbsa-0.5.2.1-archive/include/* /usr/local/cuda/include/
sudo cp -a libcusparse_lt-linux-sbsa-0.5.2.1-archive/lib/* /usr/local/cuda/lib64/


Build torch_tensorrt
~~~~~~~~~~~~~~


Install bazel

.. code-block:: sh
wget -v https://github.com/bazelbuild/bazelisk/releases/download/v1.20.0/bazelisk-linux-arm64
sudo mv bazelisk-linux-arm64 /usr/bin/bazel
chmod +x /usr/bin/bazel

Install pip and required python packages

.. code-block:: sh
# install pip
sudp apt install python3-pip
lanluo-nvidia marked this conversation as resolved.
Show resolved Hide resolved

.. code-block:: sh
# install setuptools with the version less than 71.*.*
python -m pip install setuptools==70.2.0
lanluo-nvidia marked this conversation as resolved.
Show resolved Hide resolved

.. code-block:: sh
# install torch
wget https://developer.download.nvidia.cn/compute/redist/jp/v61/pytorch/torch-2.5.0a0+872d972e41.nv24.08.17622132-cp310-cp310-linux_aarch64.whl
lanluo-nvidia marked this conversation as resolved.
Show resolved Hide resolved
python -m pip install torch-2.5.0a0+872d972e41.nv24.08.17622132-cp310-cp310-linux_aarch64.whl
lanluo-nvidia marked this conversation as resolved.
Show resolved Hide resolved

# install torchvision
# currently it has not available yet for JetPack 6.1, it should be available in future


Build and Install torch_tensorrt wheel file

.. code-block:: sh
cuda_version=$(nvcc --version | grep Cuda | grep release | cut -d ',' -f 2 | sed -e 's/ release //g')
export TORCH_INSTALL_PATH=$(python -c "import torch, os; print(os.path.dirname(torch.__file__))")
export SITE_PACKAGE_PATH=${TORCH_INSTALL_PATH::-6}
export CUDA_HOME=/usr/local/cuda-${cuda_version}/
# replace the MODULE.bazel with the jetpack one
cat toolchains/jp_workspaces/MODULE.bazel.tmpl | envsubst > MODULE.bazel
# build and install torch_tensorrt wheel file
python setup.py --use-cxx11-abi install --user
lanluo-nvidia marked this conversation as resolved.
Show resolved Hide resolved


Loading