diff --git a/CHANGELOG.md b/CHANGELOG.md index e0a5692943..3736cfc222 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,7 @@ Release report: TBD ### Added +- Add scans environment setup documentation. ([#4444](https://github.com/wazuh/wazuh-qa/pull/4444)) \- (Tests) - Add system test for global group hash ([#4015](https://github.com/wazuh/wazuh-qa/pull/4015)) \- (Tests) - Add tests for new FIM audit buffer option. ([#4485](https://githubhttps://github.com/wazuh/wazuh-qa/pull/4497#pullrequestreview-1654748331.com/wazuh/wazuh-qa/pull/4485)) \- (Framework + tests) - Add tests for merged.mg file generation. ([#4129](https://github.com/wazuh/wazuh-qa/pull/4129)) \- (Tests) diff --git a/deps/wazuh_testing/wazuh_testing/tools/scans/dependencies.py b/deps/wazuh_testing/wazuh_testing/tools/scans/dependencies.py index da953db623..d3f38a9d2c 100644 --- a/deps/wazuh_testing/wazuh_testing/tools/scans/dependencies.py +++ b/deps/wazuh_testing/wazuh_testing/tools/scans/dependencies.py @@ -63,6 +63,8 @@ def prepare_input(pip_mode, input_file_path): with open(input_file_path, mode='r') as input_file: lines = input_file.readlines() for line in lines: + if line.strip().startswith('#') or not line.strip(): + continue line = re.sub('[<>~]', '=', line) if ',' in line: package_version = max(re.findall(r'\d+\.+\d*\.*\d', line)) diff --git a/tests/scans/README.md b/tests/scans/README.md new file mode 100644 index 0000000000..95e37e04fc --- /dev/null +++ b/tests/scans/README.md @@ -0,0 +1,32 @@ +# Scans + +This directory contains scripts used to scan and verify Wazuh Python code and its dependencies. + +### Getting started + +In order to run the scripts, you should install the `wazuh_testing` package and the dependencies listed in the requirements file. + +These are the steps to do so: + +> Your terminal should be at the root of the `wazuh-qa` repository. + +1. Create and activate a virtual environment + +```console +python3 -m venv venv +source venv/bin/activate +``` + +2. Install the dependencies + +```console +pip install -r requirements.txt +``` + +3. Build the `wazuh_testing` package + +```console +pip install deps/wazuh_testing/ +``` + +That's all it takes, you can now run the [code analysis](./code_analysis/README.md) and [dependencies](./dependencies/README.md) vulnerability scanning tools. diff --git a/tests/scans/code_analysis/README.md b/tests/scans/code_analysis/README.md index 237e82815d..d6e0c6d1c9 100644 --- a/tests/scans/code_analysis/README.md +++ b/tests/scans/code_analysis/README.md @@ -1,64 +1,47 @@ - # Code Analysis -The `code_analysis` directory contains Python tests used to verify possible vulnerabilities in the Wazuh Python code. - -## Test Python Flaws - -### Description - -`test_python_flaws.py` is a Pytest test used to look for new possible vulnerabilities in directories containing Python code. - -The test uses `Bandit` to look for these possible flaws. - -In order to find new vulnerabilities, the test compares the Bandit output with vulnerabilities that we consider false positives or vulnerabilities to fix and that we save in three JSON files. By default, the directories we are checking are the `framework/`, `api/` and `wodles/` directories of the **Wazuh** repository. - -This test is located at `wazuh-qa/tests/scans/code_analysis`. -In this directory, we can find the test itself, called `test_python_flaws.py`, this `README.md`, a pytest configuration file (`conftest.py`); and a folder called `known_flaws`. - -- `known_flaws`: contains three JSON files. Each file contains a dictionary with two keys: false_positives and to_fix. The values are a list of vulnerabilities considered false positives and a list of vulnerabilities we must fix (with issues), respectively. -These files must be edited after analyzing new vulnerabilities when passing the test. - -- `conftest.py`: pytest configuration file. It adds the possibility to use specific parameters when passing the test. +The `code_analysis` directory contains Python tests to verify potential vulnerabilities in the Wazuh Python code. -- `test_python_flaws.py`: the test itself. This test will be passed using the same Python virtual environment used in the Wazuh framework and API unittests. -If the test fails, a new JSON file will be created in `wazuh-qa/tests/scans/code_analysis` showing information about the possible new vulnerabilities found. +## Description -### Usage +`test_python_flaws.py` is a Pytest test used to look for new possible vulnerabilities in directories containing Python code. It uses [Bandit](https://github.com/PyCQA/bandit) to search for these potential flaws. -The workaround for this test will be the following: +The test checks the `framework/`, `api/` and `wodles/` directories of the [Wazuh](https://github.com/wazuh/wazuh) repository by default, comparing the *Bandit* output with the vulnerabilities identified as false positives or vulnerabilities to fix. It saves the results in three JSON files (one JSON file for each module). -- Pass the test. +The contents of this directory are: +- `known_flaws`: The directory contains three JSON files, one for each module (`api`, `framework` and `wodles`). Each file has a dictionary with two keys: **false_positives** and **to_fix**. These values are the list of vulnerabilities considered false positives and the list of vulnerabilities you must fix (with issues), respectively. After running the test and analyzing the new vulnerabilities, you must edit these files. +- `conftest.py`: The Pytest configuration file. It adds the possibility to use specific parameters when running the test. +- `test_python_flaws.py`: The test itself. You should run this test using the same Python virtual environment used in the Wazuh framework and API unit tests. If the test fails, a new JSON file will be created inside this directory, showing information about the possible new vulnerabilities found. -- If the test passes, no actions are needed, everything is correct. +## Usage -- If the test fails, new code vulnerabilities will be found in `wazuh-qa/tests/scans/code_analysis/new_flaws_{module}.json`. - - We analyze the new vulnerabilities found in the module and report them in GitHub issues. - - We move the vulnerabilities to the `to_fix` key of the known flaws JSON file. - - If the new vulnerability is considered a false positive, we add it to the `false_positives` list of the dictionary in its respective `known_flaws` JSON file. - - If the new vulnerability is a real vulnerability, we solve the problem reported and remove the flaw from the known flaws file. +- Run the test: `pytest tests/scans/code_analysis/test_python_flaws.py` +- If the test passes without failures, everything is correct, and no action is needed. +- If the test fails, `wazuh-qa/tests/scans/code_analysis/new_flaws_{module}.json` file will report the new code vulnerabilities found. +You should analyze the new vulnerabilities found in the module and report them in GitHub issues. -The test also updates the known_flaws files automatically. If we have a look at a known_flaws file, we will see that each flaw dictionary contains information like the line number or range. This information is the one updated by the test. The test also removes flaws from the known_flaws file if they don't appear in the Bandit output. +If you need to fix a new vulnerability, add it to the **to_fix** key module's JSON file entry found in the **known_flaws** directory. +If the new vulnerability is a false positive, add it to the **false_positives** key module's JSON file entry found in the **known_flaws** directory. -#### Parameters +The test updates the files inside **known_flaws** automatically with information like the line number or range of the flaws in the **to_fix** dictionary. The test also removes flaws from the **known_flaws** files if Bandit did not report them. -As said in the description, the test uses `Bandit` to look for possible Python flaws. By default, the tests checks the framework, wodles and api directories in the Wazuh repository, in its master branch. +## Parameters -These directories, repository and branch can be passed to the test as parameters so it is possible to run the test in any directory containing Python code inside the Wazuh organization. +You can set the directories, repository, and branch parameters to test any directory containing Python code inside the Wazuh organization. +You can also use more parameters to customize the test functionality. The test will only succeed if you check different directories and repositories, as we don't have **known_flaws** files for non-default directories. -Apart from this parameters, there are more that can be used to customize the test functionality. Note that the test will fail if we check different directories and/or repository as we don't have known_flaws files for non-default directories. +> By default, the test checks the `framework`, `wodles` and `api` directories in the [wazuh/wazuh](https://github.com/wazuh/wazuh) repository's master branch. -- **--repo**: set the repository used. Default: `wazuh` -- **--reference**: set the repository branch or tag. Default: `master` -- **--check_directories**: set the directories to check, this must be a string with the directory name. -If more than one is indicated, they must be separated with comma. Default: `framework/,api/,wodles/`. -- **--exclude_directories**: set the directories to exclude, this must be a string with the directory name. -If more than one is indicated, they must be separated with comma. Default: `test/,tests/`. -- **--confidence**: set the minimum value of confidence of the Bandit scan. -This value must be 'UNDEFINED', 'LOW', 'MEDIUM' or 'HIGH'. Default: `MEDIUM` -- **--severity**: set the minimum value of severity of the Bandit scan. -This value must be 'UNDEFINED', 'LOW', 'MEDIUM' or 'HIGH'. Default: `LOW` +| Parameter | Description | Default Value | +|-----------------------|-------------------------------------------------------|-------------------| +| `--repo` | The repository to test. | `wazuh` | +| `--reference` | The repository branch. | `master` | +| `--check_directories` | The directories to check (comma-separated). | `framework/,api/,wodles/` | +| `--exclude_directories`| The directories to exclude (comma-separated). | `test/,tests/` | +| `--confidence` | Minimum confidence level for Bandit scan. | `MEDIUM` | +| `--severity` | Minimum severity level for Bandit scan. | `LOW` | +> The values accepted by the flags `--confidence` and `--security` are `UNDEFINED`, `LOW`, `MEDIUM` or `HIGH`. #### Example diff --git a/tests/scans/dependencies/README.md b/tests/scans/dependencies/README.md index 78d597a5f5..d2b338b277 100644 --- a/tests/scans/dependencies/README.md +++ b/tests/scans/dependencies/README.md @@ -1,22 +1,27 @@ # Dependencies Scanner ## Description -It's a tool used to scan for vulnerabilities in a requirements.txt file.\ -It can generate reports via console output or json file. Can be run with `pytest` and manage to handle remote files under github repositories. Requirements file can be specified with `repo`, `branch`, `requirements-path` parameters giving flexibility on file location. -Output file in which the report will be generated can be specified with `report-path` parameter. + +The Dependencies Scanner is a tool for scanning vulnerabilities in a *requirements.txt* file from different GitHub repositories. +It uses `pytest` to run and can generate reports via console output or JSON files. + +The requirements file to scan can be specified with the `repo`, `reference`, and `requirements-path` parameters. +Moreover, the path of the report file generated can be chosen with the `report-path` parameter. ## How to use - Pytest ``` Parameters: --repo: repository name. Default: 'wazuh'. - --branch: branch name of specified repository. Default: 'master'. - --commit: commit of specified repository. Default: None + --reference: branch name of specified repository. Default: 'master'. --requirements-path: requirements file path. Default: 'framework/requirements.txt'. --report-path: output file path. Default: 'dependencies/report_file.json'. ``` ### Scanning wazuh-qa requirements file: + +> The script does not support pyenv version changes because it doesn't expect system variables to be modified + ``` -↪ ~/git/wazuh-qa/tests/scans ⊶ feature/1612-package-vuln-scanner ⨘ python3 -m pytest -vv -x --disable-warnings dependencies/ --repo wazuh-qa --branch master --requirements-path requirements.txt +↪ ~/git/wazuh-qa/tests/scans ⊶ feature/1612-package-vuln-scanner ⨘ python3 -m pytest -vv -x --disable-warnings dependencies/ --repo wazuh-qa --reference master --requirements-path requirements.txt ==================================================================================== test session starts ===================================================================================== platform linux -- Python 3.9.5, pytest-6.2.3, py-1.10.0, pluggy-0.13.1 -- /home/kondent/pythonEnv/qa-env/bin/python3 cachedir: .pytest_cache @@ -75,7 +80,7 @@ FAILED dependencies/test_dependencies.py::test_python_dependencies_vuln_scan - A ### Scanning wazuh requirements file with a specific output path: ``` -↪ ~/git/wazuh-qa/tests/scans ⊶ feature/1612-package-vuln-scanner ⨘ python3 -m pytest -vv -x --disable-warnings dependencies/ --repo wazuh --branch master --requirements-path framework/requirements.txt --report-path ~/Desktop/report_file.json +↪ ~/git/wazuh-qa/tests/scans ⊶ feature/1612-package-vuln-scanner ⨘ python3 -m pytest -vv -x --disable-warnings dependencies/ --repo wazuh --reference master --requirements-path framework/requirements.txt --report-path ~/Desktop/report_file.json ==================================================================================== test session starts ===================================================================================== platform linux -- Python 3.9.5, pytest-6.2.3, py-1.10.0, pluggy-0.13.1 -- /home/kondent/pythonEnv/qa-env/bin/python3 cachedir: .pytest_cache @@ -93,4 +98,4 @@ dependencies/test_dependencies.py::test_python_dependencies_vuln_scan PASSED "vulnerabilities_found": 0, "packages": [] } -``` \ No newline at end of file +```