From ab47abea81631269b55f81ca8d67285c8f6ecc5b Mon Sep 17 00:00:00 2001 From: pranavm Date: Tue, 14 Jan 2025 13:32:15 -0800 Subject: [PATCH 1/3] Rewrites docs/README.md to be briefer --- tripy/docs/README.md | 218 +++++++++++++++++++++---------------------- 1 file changed, 108 insertions(+), 110 deletions(-) diff --git a/tripy/docs/README.md b/tripy/docs/README.md index 5c6d583a8..81605f9af 100644 --- a/tripy/docs/README.md +++ b/tripy/docs/README.md @@ -1,167 +1,165 @@ # Documentation -This directory includes all the source files for the public API documentation. +This directory contains **source** files for public guides and **configuration** +files for the public documentation. + ## Building Documentation Locally -You can build the documentation locally in the development container by running: +In the development container, run: + ```bash python3 docs/generate_rsts.py sphinx-build build/doc_sources build/docs -c docs/ -j 4 -W -n ``` -To view the documentation, you can open `build/docs/index.html` in a browser. - -## Adding Documentation - -### How It Works - -The `export.public_api()` decorator allows you to specify metadata for documentation -generation, such as where in the documentation hierarchy the API should be documented. - -The `generate_rsts.py` script uses this information to automatically generate a directory -structure and populate it with `.rst` files. -For more information, see the docstring for [`export.public_api()`](../nvtripy/export.py). +- To **view** the docs, launch an HTTP server *outside* the container. + From the [tripy root directory](../), run: -### Docstrings - -For all public facing docstrings, we have several requirements: + ```bash + python3 -m http.server 8001 --directory build/docs + ``` -- The function signature must have type annotations for all parameters and return type. + Then navigate to http://localhost:8001 in a browser. -- The docstring must explain what the operation is doing. -- The parameters must be documented in the `Args:` section of the docstring. +## How It Works -- The return value must be documented in the `Returns:` section of the docstring. +1. [`generate_rsts.py`](./generate_rsts.py) creates `.rst` files based on public APIs + and guides (details below). -- The docstring must include a code example (denoted by `.. code-block:: python`). +2. Sphinx generates HTML documentation based on the `.rst`s. -### Guides +### API Documentation -In addition to the API reference, we also include various guides in subdirectories -of [docs](.). +Public APIs are indicated by the [`export.public_api()`](../nvtripy/export.py) decorator, +which specifies doc metadata for each API (e.g. location). -Each such subdirectory must start with a `pre_` or `post_` prefix, which indicates -the ordering of each set of guides in the index/side bar. Specifically, `pre` indicates -that the guides in that directory should precede the API reference documentation, while -`post` indicates that they should follow it. The number indicates the relative ordering -with respect to other sets of guides. For example, if we have the following directories: +**Requirements:** -- `pre0_user_guides` -- `pre1_examples` -- `post0_developer_guides` +- Signature must have **type annotations**. -then the documentation will have the following ordering: +- All parameters and return values must be documented. -- User Guides -- Examples -- API Reference -- Developer Guides +- Docstring must include *at least* **one [code example](#code-examples)**. -The markdown files there are included in `.rst` files and parsed by the Myst parser. -This means we need to make some special considerations: +- If the function accepts `tp.Tensor`s, must indicate **data type constraints** + with the [`wrappers.interface`](../nvtripy/wrappers.py) decorator. -1. We cannot use the `[[TOC]]` directive to automatically generate tables of contents. - Our Sphinx theme will automatically generate tables of contents, so you can omit these entirely. +**Example:** -2. All links to files in the repository must be absolute and start with `source:` so that - Myst can replace them with URLs to our remote repository. Otherwise, the links will - cause the relevant file to be downloaded. For example: - ``` - [Fill operation](source:/nvtripy/frontend/trace/ops/fill.py) - ``` +```py +@export.public_api(document_under="operations/functions") +@wrappers.interface( + dtype_constraints={"input": "T1", wrappers.RETURN_VALUE: "T1"}, + dtype_variables={ + "T1": ["float32", "float16", "bfloat16", "int4", "int32", "int64", "bool", "int8"], + }, +) +def relu(input: "nvtripy.Tensor") -> "nvtripy.Tensor": + r""" + Applies Rectified Linear Unit (RELU) function + to each element of the input tensor: + + :math:`\text{relu}(x) = \max(0,x)` + + Args: + input: The input tensor. + + Returns: + A tensor of the same shape as the input. + + .. code-block:: python + :linenos: + + input = tp.Tensor([1., 2., 3., 4.], dtype=tp.float32) + output = tp.relu(input) + + t = torch.tensor([1, 2, 3, 4], dtype=torch.float32) # doc: omit + assert tp.allclose(output, tp.Tensor(torch.nn.functional.relu(t))) + """ +``` - Links to markdown files are an exception; if a markdown file is part of the *rendered* - documentation, it should be linked to using the `project:` tag instead. -3. For links to documentation for APIs, you can use the following syntax: +### Guides - ```md - {}`` - ``` +**Guides** are included in various subdirectories of [docs](.) for longer-form +content, e.g. **workflows** or **concepts**. - For example: +- The `pre_` or `post_` directory prefix indicates ordering of guide sets; + `pre` guides **precede** the API reference, `post` guides **follow** it. - ```md - {class}`nvtripy.Tensor` - ``` +Guides are **markdown** files parsed by the Myst parser, therefore: - `` can take on any value that is a valid role provided by - [Sphinx's Python domain](https://www.sphinx-doc.org/en/master/usage/domains/python.html). +- Do **not** use `[[TOC]]` or include a table of contents manually; one will be generated automatically. -Guides may use the markers specified in [tests/helper.py](../tests/helper.py) to customize -how the documentation is interpreted (see `AVAILABLE_MARKERS` in that file). +- Links to files must be **absolute** paths and use a `source:` tag. + - **Exception:** links to **rendered** markdown files (e.g. other guides) should use a `project:` tag. -### Code Examples + - **Example:** -Code examples in public facing docstrings and guides are preprocessed before -documentation is generated. Specifically: + ```md + [Fill operation](source:/nvtripy/frontend/trace/ops/fill.py) + ``` -- Any code examples are executed so that their output can be - displayed after the code block. Several modules, including `nvtripy` (as `tp`), - `numpy` (as `np`), `cupy` (as `cp`), and `torch` are automatically imported - and can be used in code examples. + - **Why:** Other links will cause the file to be downloaded instead of linking to the repository. -- The values of any `nvtripy` type local variables are appended to the output. - You can customize this behavior: +- Links to API documentation should use the syntax: - - To only display certain variables, add `# doc: print-locals` followed by a space - separated list of variable names. For example: `# doc: print-locals inp out`. + ```md + {}`` + ``` - - To only disable certain variables, add `# doc: no-print-locals` followed by a space - separated list of variable names. For example: `# doc: no-print-locals inp out`. + `` should be a role from + [Sphinx's Python domain](https://www.sphinx-doc.org/en/master/usage/domains/python.html). - - To disable it completely, add just `# doc: no-print-locals` without specifying any variables. + - **Example:** -- In docstrings, but not guides, any `assert` statements are stripped out. + ```md + {class}`nvtripy.Tensor` + ``` -- Any lines that end with `# doc: omit` are stripped out. +#### Tip: Dynamically Generating Content -- By default, documentation generation will fail if an exception is thrown by a code snippet. - In order to allow exceptions, add `# doc: allow-exception`. +You can embed code whose **output** is included but whose **content** is excluded from the guide +using the `DOC: OMIT` marker: -To avoid running code entirely, you can add `# doc: no-eval` in the docstring. Note that this will -not prevent the code block from being executed in the tests. +```md + + ```py + # doc: no-print-locals + print("This line should be rendered into the docs") + ``` + +``` -#### Code Example Format +- **See also:** `AVAILABLE_MARKERS` in [tests/helper.py](../tests/helper.py) for a complete list of markers. -The code block format for docstrings is: -```py -""" -.. code-block:: python - :linenos: - :caption: Descriptive Title - -""" -``` +### Code Examples -Captions will have a prefix of `Example: ` prepended to them. +Code blocks in docstrings/guides are **preprocessed**: -**NOTE: The docstrings must *not* import `nvtripy`, `numpy`, or `torch`. They will be imported** - **automatically as `tp`, `np`, and `torch` respectively. Any other modules will need to be imported.** +- Some modules are **automatically imported** - do **not** import these manually: + - `nvtripy` (as `tp`) + - `numpy` (as `np`) + - `cupy` (as `cp`) + - `torch` +- In docstrings, but **not** guides, `assert` statements are removed. +- Lines ending with `# doc: omit` are stripped out. -### Dynamically Generating Content In Guides +- Code is **executed** and any output is displayed in the docs. -In some cases, it's useful to run Python code and include the output in a guide *without* including -the Python code itself. To do so, you can use a trick like this: + If the code throws, doc generation will fail. Use `# doc: allow-exception` to allow exceptions. -```md - - ```py - # doc: no-print-locals - print("This line should be rendered into the docs") - ``` - -``` + - **Note:** `# doc: no-eval` disables execution but this means the code will be **untested**! -This works because `DOC: OMIT` removes the encapsulated text from the post-processed markdown file -but does *not* prevent the block from being evaluated (to do that, use `# doc: no-eval` as normal). +- Local variables are also displayed. You can customize this: -We include special logic that will omit the `Output:` heading when the output -is generated in this way. + - **Include** only specific variables: `# doc: print-locals ...` + - **Exclude** *specific* variables: `# doc: no-print-locals ...` + - **Exclude** *all* variables: `# doc: no-print-locals` (with no arguments). From 64a65b36d4a4da060a81352c6e2a3745a832e783 Mon Sep 17 00:00:00 2001 From: pranavm Date: Tue, 14 Jan 2025 13:32:43 -0800 Subject: [PATCH 2/3] Rewrites tests/README.md to be briefer --- tripy/tests/README.md | 137 ++++++++++++++++++++---------------------- 1 file changed, 64 insertions(+), 73 deletions(-) diff --git a/tripy/tests/README.md b/tripy/tests/README.md index dd739fb3d..6af3d39e4 100644 --- a/tripy/tests/README.md +++ b/tripy/tests/README.md @@ -1,129 +1,120 @@ # Tests -The tests directory includes both unit and integration tests. For the former, the file -structure is meant to exactly mirror the structure of the code. That means, for example -that `nvtripy/path/to/.py` will have all of its unit tests in `tests/path/to/test_.py`. -The `tests/integration` directory captures the latter group of tests. +## Test Types + +- **Unit tests** mirror the structure of the code. + + - **Example:** Tests for [`nvtripy/frontend/tensor.py`](../nvtripy/frontend/tensor.py) + are located in [`tests/frontend/test_tensor.py`](../tests/frontend/test_tensor.py). + +- **Integration tests** are under [`tests/integration`](../tests/integration/). + +- **Performance tests** are under [`tests/performance`](../tests/performance/). There are 3 kinds: + + - Regression tests, using + [`pytest-benchmark`](https://pytest-benchmark.readthedocs.io/en/latest/) + and the + [Continuous Benchmark GitHub Action](https://github.com/marketplace/actions/continuous-benchmark). + + - **Tip:** View historical perf charts by opening + [`index.html` from the `benchmarks` branch](https://github.com/NVIDIA/TensorRT-Incubator/blob/benchmarks/dev/bench/index.html) + in a browser. + + - Comparative tests between Tripy and `torch.compile`. + + - Overhead tests to measure runtime overheads vs. running the MLIR executable directly. ## Running Tests -You can run all tests locally in the development container by running: +To run all tests: + ```bash pytest tests/ -v ``` -You can also provide marker arguments to only run specific test cadences -(see [the test cadence section](#test-cadence) below). For example, to run only -L0 tests, use: +To run L0 tests: ```bash pytest tests/ -v -m "not l1 and not l1_release_package" -n 4 --dist worksteal --ignore tests/performance pytest tests/performance -v -m "not l1 and not l1_release_package" ``` -Note that the L0/L1 tests can be parallelized. In that case, performance tests -are run separately because they must run serially to ensure accurate measurements. +- **Note:** Performance tests are run separately because they must run serially. + + +## Test Cadence + +Use pytest markers to indicate which job a test should run in. + +- The default is **L0**, i.e. the pull request pipeline. + +- Markers will overwrite the implicit L0 marker but are otherwise additive. + + +- **Example:** Marking tests to run in **L1** (nightly): + + ```py + @pytest.mark.l1 + def test_really_slow_things(): + ... + ``` + + +- **See also:** [pyproject.toml](../pyproject.toml) for all supported markers. + ## Profiling -You can profile test runtimes in the development container using the -`--profile` option, which will generate `pstats` files for each test -in a `prof/` directory, along with a `combined.prof` file for all the -tests together. +The `--profile` option will generate profiling data under `prof/`: -For example, to profile L0 tests, run: +- `pstats` file for each test +- `combined.prof` for all tests together + +**Example:** To profile L0 functional tests: ```bash pytest tests/ -v -m "not l1 and not l1_release_package" --ignore tests/performance --profile ``` -You can visualize the results using `snakeviz`. +Visualize the results using `snakeviz`. -*NOTE: Ensure that you launched the development container with port forwarding,* -*i.e. the `-p 8080:8080` option.* +- **Note:** Ensure you launched the container with port forwarding (`-p 8080:8080`). -For example: +**Example:** ```bash snakeviz prof/combined.prof -s --hostname 0.0.0.0 ``` -Then, in a browser, navigate to: +Navigate to: http://localhost:8080/snakeviz/%2Ftripy%2Fprof%2Fcombined.prof - +in a browser. ## Coverage Reports -You can generate code coverage reports locally by running: +To generate code coverage reports: ```bash pytest --cov=nvtripy/ --cov-report=html --cov-config=.coveragerc tests/ -v ``` -To view the report, open the `htmlcov/index.html` file from the root directory in a browser. +You can view the report at `htmlcov/index.html` in a browser. ## Dead Code Detection -Our development container includes a static analysis tool called `vulture` that can -detect dead code. *This **will** include false positives for our code, so be careful!* +Use `vulture` to detect dead code. -You can run it with: +- **Note:** This **will** include false positives! ```bash vulture . --sort-by-size ``` -To exclude false positives, use: +Exclude false positives (but also some true positives) with: ```bash vulture . --sort-by-size --min-confidence=100 ``` - - -## Adding Tests - -When modifying or adding new files in `nvtripy`, make sure that you also modify or add the corresponding -unit test files under `tests`. For integration tests, you can find an appropriate file in -`tests/integration` or create a new one if none of the existing files fit your needs. - -### Test Cadence - -We don't necessarily want to run every test in every single pipeline. You can use special -pytest markers to indicate the cadence for a test. For example: - - - -```py -@pytest.mark.l1 -def test_really_slow_things(): - ... -``` - - - -Supported markers are documented in [pyproject.toml](../pyproject.toml). - -### Performance Tests - -In addition to functional tests, we also run performance tests of three kinds: - -1. Regression tests, which compare current Tripy performance to historical data - to ensure we don't regress. We use the - [`pytest-benchmark`](https://pytest-benchmark.readthedocs.io/en/latest/) - plugin to gather data and the - [Continuous Benchmark GitHub Action](https://github.com/marketplace/actions/continuous-benchmark) - for regression testing. - - You can view graphs and charts of the historical data by opening the - [`index.html` file from the `benchmarks` branch](https://github.com/NVIDIA/TensorRT-Incubator/blob/benchmarks/dev/bench/index.html) - in a browser. - -2. Comparative tests, which compare Tripy and `torch.compile`. - -3. Overhead tests, which check the overhead introduced by Tripy as compared - to running the underlying MLIR executable by itself. This is done by measuring - how long it takes to run an empty executable since in that case, all the time - is taken by the Tripy wrapper code. From 9557e912172f2007dfa37282a6bf228bf67d3171 Mon Sep 17 00:00:00 2001 From: pranavm Date: Tue, 14 Jan 2025 13:32:54 -0800 Subject: [PATCH 3/3] Various minor changes to public documentation --- tripy/README.md | 107 +++++++++--------- .../00-introduction-to-tripy.md | 2 +- tripy/nvtripy/types.py | 4 +- 3 files changed, 56 insertions(+), 57 deletions(-) diff --git a/tripy/README.md b/tripy/README.md index 1686c2d8c..129148525 100644 --- a/tripy/README.md +++ b/tripy/README.md @@ -1,8 +1,8 @@ # Tripy: A Python Programming Model For TensorRT -[**Installation**](#installation) -| [**Getting Started**](#getting-started) +[**Quick Start**](#quick-start) +| [**Installation**](#installation) | [**Examples**](https://github.com/NVIDIA/TensorRT-Incubator/tree/main/tripy/examples) | [**Notebooks**](https://github.com/NVIDIA/TensorRT-Incubator/tree/main/tripy/notebooks) | [**Contributing**](https://github.com/NVIDIA/TensorRT-Incubator/blob/main/tripy/CONTRIBUTING.md) @@ -17,53 +17,63 @@ a deep learning inference compiler. What you can expect: -- **High Performance:** Leveraging [TensorRT](https://developer.nvidia.com/tensorrt)'s optimization capabilties. +- **High performance** by leveraging [TensorRT](https://developer.nvidia.com/tensorrt)'s optimization capabilties. +- An **intuitive API** that follows conventions of the ecosystem. +- **Debuggability** with features like **eager mode** to interactively debug mistakes. +- **Excellent error messages** that are informative and actionable. +- **Friendly documentation** that is comprehensive but concise, with code examples. -- **Intuitive API:** A familiar API that follows conventions of the ecosystem. -- **Debuggability:** **Eager mode** to interactively debug mistakes. +## Quick Start -- **Excellent Error Messages**: Informative and actionable. +See the +[Introduction To Tripy](https://nvidia.github.io/TensorRT-Incubator/pre0_user_guides/00-introduction-to-tripy.html) +guide for details: -- **Friendly Documentation**: Comprehensive but concise, with code examples. + +- **Defining** a model: -Code is worth 1,000 words: + ```py + class Model(tp.Module): + def __init__(self): + self.conv = tp.Conv(in_channels=1, out_channels=1, kernel_dims=[3, 3]) - -```py -# Define our model: -class Model(tp.Module): - def __init__(self): - self.conv = tp.Conv(in_channels=1, out_channels=1, kernel_dims=[3, 3]) - - def __call__(self, x): - x = self.conv(x) - x = tp.relu(x) - return x - - -# Initialize the model and load weights: -model = Model() -model.load_state_dict( - { - "conv.weight": tp.ones((1, 1, 3, 3)), - "conv.bias": tp.ones((1,)), - } -) - -inp = tp.ones((1, 1, 4, 4)) - -# Eager mode: -eager_out = model(inp) - -# Compiled mode: -compiled_model = tp.compile( - model, - args=[tp.InputInfo(shape=(1, 1, 4, 4), dtype=tp.float32)], -) - -compiled_out = compiled_model(inp) -``` + def __call__(self, x): + x = self.conv(x) + x = tp.relu(x) + return x + ``` + +- **Initializing** it: + + ```py + model = Model() + model.load_state_dict( + { + "conv.weight": tp.ones((1, 1, 3, 3)), + "conv.bias": tp.ones((1,)), + } + ) + + dummy_input = tp.ones((1, 1, 4, 4)) + ``` + +- Executing in **eager mode**: + + ```py + eager_out = model(dummy_input) + ``` + +- **Compiling** and executing: + + ```py + compiled_model = tp.compile( + model, + args=[tp.InputInfo(shape=(1, 1, 4, 4), dtype=tp.float32)], + ) + + compiled_out = compiled_model(dummy_input) + ``` @@ -106,14 +116,3 @@ For the latest changes, build Tripy wheels from source: python3 -c "import nvtripy as tp; x = tp.ones((5,), dtype=tp.int32); assert x.tolist() == [1] * 5" ``` - - -## Getting Started - -- **Start with**: - [Introduction To Tripy](https://nvidia.github.io/TensorRT-Incubator/pre0_user_guides/00-introduction-to-tripy.html) - -Other guides: - -- [Compiling For Better Performance](https://nvidia.github.io/TensorRT-Incubator/pre0_user_guides/02-compiler.html) -- [Quantization](https://nvidia.github.io/TensorRT-Incubator/pre0_user_guides/01-quantization.html) diff --git a/tripy/docs/pre0_user_guides/00-introduction-to-tripy.md b/tripy/docs/pre0_user_guides/00-introduction-to-tripy.md index e0ca40786..edd5b9bf4 100644 --- a/tripy/docs/pre0_user_guides/00-introduction-to-tripy.md +++ b/tripy/docs/pre0_user_guides/00-introduction-to-tripy.md @@ -5,7 +5,7 @@ a deep learning inference compiler. ## API Semantics -Unlike TensorRT's graph-based semantics, Tripy uses a functional style: +Unlike TensorRT's graph-based semantics, Tripy uses a **functional** style: ```py # doc: no-print-locals diff --git a/tripy/nvtripy/types.py b/tripy/nvtripy/types.py index 8389fbc64..f34cacb99 100644 --- a/tripy/nvtripy/types.py +++ b/tripy/nvtripy/types.py @@ -33,7 +33,7 @@ module=sys.modules[__name__], symbol="TensorLike", doc=""" - A Tripy :class:`Tensor` or a Python number that can be automatically converted into one. + A :class:`nvtripy.Tensor` or a Python number that can be automatically converted into one. """, )(Union["nvtripy.Tensor", numbers.Number]) @@ -53,6 +53,6 @@ module=sys.modules[__name__], symbol="ShapeLike", doc=""" - A shape of a :class:`Tensor` . + A shape of a :class:`nvtripy.Tensor` . """, )(Sequence[IntLike])