From 161dabc8378845bbdfe39d813a731f7d0617ea87 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Gast=C3=B3n=20Palomeque?= Date: Wed, 23 Aug 2023 15:15:31 -0300 Subject: [PATCH 1/7] fix(#4274): Create README with setup instructions --- tests/scans/README.md | 32 ++++++++++++++++++++++++++++++++ 1 file changed, 32 insertions(+) create mode 100644 tests/scans/README.md diff --git a/tests/scans/README.md b/tests/scans/README.md new file mode 100644 index 0000000000..95e37e04fc --- /dev/null +++ b/tests/scans/README.md @@ -0,0 +1,32 @@ +# Scans + +This directory contains scripts used to scan and verify Wazuh Python code and its dependencies. + +### Getting started + +In order to run the scripts, you should install the `wazuh_testing` package and the dependencies listed in the requirements file. + +These are the steps to do so: + +> Your terminal should be at the root of the `wazuh-qa` repository. + +1. Create and activate a virtual environment + +```console +python3 -m venv venv +source venv/bin/activate +``` + +2. Install the dependencies + +```console +pip install -r requirements.txt +``` + +3. Build the `wazuh_testing` package + +```console +pip install deps/wazuh_testing/ +``` + +That's all it takes, you can now run the [code analysis](./code_analysis/README.md) and [dependencies](./dependencies/README.md) vulnerability scanning tools. From 4c9026fe051b15ba29a5f0b029f6087c0ca1c276 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Gast=C3=B3n=20Palomeque?= Date: Wed, 23 Aug 2023 15:16:26 -0300 Subject: [PATCH 2/7] fix(#4274): Update parameters documentation --- tests/scans/code_analysis/README.md | 72 +++++++++++++---------------- tests/scans/dependencies/README.md | 18 ++++---- 2 files changed, 42 insertions(+), 48 deletions(-) diff --git a/tests/scans/code_analysis/README.md b/tests/scans/code_analysis/README.md index 237e82815d..729f7ef91c 100644 --- a/tests/scans/code_analysis/README.md +++ b/tests/scans/code_analysis/README.md @@ -1,63 +1,55 @@ - # Code Analysis -The `code_analysis` directory contains Python tests used to verify possible vulnerabilities in the Wazuh Python code. - -## Test Python Flaws - -### Description +The `code_analysis` directory contains Python tests to verify potential vulnerabilities in the Wazuh Python code. -`test_python_flaws.py` is a Pytest test used to look for new possible vulnerabilities in directories containing Python code. +## Description -The test uses `Bandit` to look for these possible flaws. +`test_python_flaws.py` is a Pytest test used to look for new possible vulnerabilities in directories containing Python code. It uses [Bandit](https://github.com/PyCQA/bandit) to search for these potential flaws. -In order to find new vulnerabilities, the test compares the Bandit output with vulnerabilities that we consider false positives or vulnerabilities to fix and that we save in three JSON files. By default, the directories we are checking are the `framework/`, `api/` and `wodles/` directories of the **Wazuh** repository. +The test checks the `framework/`, `api/` and `wodles/` directories of the [Wazuh](https://github.com/wazuh/wazuh) repository by default, comparing the *Bandit* output with the vulnerabilities identified as false positives or vulnerabilities to fix. It saves the results in three JSON files (one JSON file for each module). -This test is located at `wazuh-qa/tests/scans/code_analysis`. -In this directory, we can find the test itself, called `test_python_flaws.py`, this `README.md`, a pytest configuration file (`conftest.py`); and a folder called `known_flaws`. +The contents of this directory are: +- `known_flaws`: The directory contains three JSON files, one for each module (`api`, `framework` and `wodles`). Each file has a dictionary with two keys: **false_positives** and **to_fix**. These values are the list of vulnerabilities considered false positives and the list of vulnerabilities you must fix (with issues), respectively. After running the test and analyzing the new vulnerabilities, you must edit these files. +- `conftest.py`: The Pytest configuration file. It adds the possibility to use specific parameters when running the test. +- `test_python_flaws.py`: The test itself. You should run this test using the same Python virtual environment used in the Wazuh framework and API unit tests. If the test fails, a new JSON file will be created inside this directory, showing information about the possible new vulnerabilities found. -- `known_flaws`: contains three JSON files. Each file contains a dictionary with two keys: false_positives and to_fix. The values are a list of vulnerabilities considered false positives and a list of vulnerabilities we must fix (with issues), respectively. -These files must be edited after analyzing new vulnerabilities when passing the test. +## Usage -- `conftest.py`: pytest configuration file. It adds the possibility to use specific parameters when passing the test. +- Run the test. +- If the test passes without failures, everything is correct, and no action is needed. +- If the test fails, `wazuh-qa/tests/scans/code_analysis/new_flaws_{module}.json` file will report the new code vulnerabilities found. +You should analyze the new vulnerabilities found in the module and report them in GitHub issues. -- `test_python_flaws.py`: the test itself. This test will be passed using the same Python virtual environment used in the Wazuh framework and API unittests. -If the test fails, a new JSON file will be created in `wazuh-qa/tests/scans/code_analysis` showing information about the possible new vulnerabilities found. +If you need to fix a new vulnerability, add it to the **to_fix** key module's JSON file entry found in the **known_flaws** directory. +If the new vulnerability is a false positive, add it to the **false_positives** key module's JSON file entry found in the **known_flaws** directory. -### Usage +The test updates the files inside **known_flaws** automatically with information like the line number or range of the flaws in the **to_fix** dictionary. The test also removes flaws from the **known_flaws** files if Bandit did not report them. -The workaround for this test will be the following: +## Parameters -- Pass the test. +You can set the directories, repository, and branch parameters to test any directory containing Python code inside the Wazuh organization. +You can also use more parameters to customize the test functionality. The test will only succeed if you check different directories and repositories, as we don't have **known_flaws** files for non-default directories. -- If the test passes, no actions are needed, everything is correct. +> By default, the test checks the `framework`, `wodles` and `api` directories in the [wazuh/wazuh](https://github.com/wazuh/wazuh) repository's master branch. -- If the test fails, new code vulnerabilities will be found in `wazuh-qa/tests/scans/code_analysis/new_flaws_{module}.json`. - - We analyze the new vulnerabilities found in the module and report them in GitHub issues. - - We move the vulnerabilities to the `to_fix` key of the known flaws JSON file. - - If the new vulnerability is considered a false positive, we add it to the `false_positives` list of the dictionary in its respective `known_flaws` JSON file. - - If the new vulnerability is a real vulnerability, we solve the problem reported and remove the flaw from the known flaws file. +- `--repo`: the repository used. -The test also updates the known_flaws files automatically. If we have a look at a known_flaws file, we will see that each flaw dictionary contains information like the line number or range. This information is the one updated by the test. The test also removes flaws from the known_flaws file if they don't appear in the Bandit output. + Default: `wazuh` +- `--reference`: the repository branch. -#### Parameters + Default: `master` +- `--check_directories`: the directories names to check. This parameter accepts multiple directories separated by commas. -As said in the description, the test uses `Bandit` to look for possible Python flaws. By default, the tests checks the framework, wodles and api directories in the Wazuh repository, in its master branch. + Default: `framework/,api/,wodles/` +- `--exclude_directories`: the directories names to exclude. This parameter accepts multiple directories separated by commas. -These directories, repository and branch can be passed to the test as parameters so it is possible to run the test in any directory containing Python code inside the Wazuh organization. + Default: `test/,tests/` +- `--confidence`: the minimum value of confidence of the Bandit scan. This value must be 'UNDEFINED', 'LOW', 'MEDIUM' or 'HIGH'. -Apart from this parameters, there are more that can be used to customize the test functionality. Note that the test will fail if we check different directories and/or repository as we don't have known_flaws files for non-default directories. + Default: `MEDIUM` +- `--severity`: the minimum value of severity of the Bandit scan. This value must be 'UNDEFINED', 'LOW', 'MEDIUM' or 'HIGH'. -- **--repo**: set the repository used. Default: `wazuh` -- **--reference**: set the repository branch or tag. Default: `master` -- **--check_directories**: set the directories to check, this must be a string with the directory name. -If more than one is indicated, they must be separated with comma. Default: `framework/,api/,wodles/`. -- **--exclude_directories**: set the directories to exclude, this must be a string with the directory name. -If more than one is indicated, they must be separated with comma. Default: `test/,tests/`. -- **--confidence**: set the minimum value of confidence of the Bandit scan. -This value must be 'UNDEFINED', 'LOW', 'MEDIUM' or 'HIGH'. Default: `MEDIUM` -- **--severity**: set the minimum value of severity of the Bandit scan. -This value must be 'UNDEFINED', 'LOW', 'MEDIUM' or 'HIGH'. Default: `LOW` + Default: `LOW` #### Example diff --git a/tests/scans/dependencies/README.md b/tests/scans/dependencies/README.md index 78d597a5f5..5241116e52 100644 --- a/tests/scans/dependencies/README.md +++ b/tests/scans/dependencies/README.md @@ -1,22 +1,24 @@ # Dependencies Scanner ## Description -It's a tool used to scan for vulnerabilities in a requirements.txt file.\ -It can generate reports via console output or json file. Can be run with `pytest` and manage to handle remote files under github repositories. Requirements file can be specified with `repo`, `branch`, `requirements-path` parameters giving flexibility on file location. -Output file in which the report will be generated can be specified with `report-path` parameter. + +The Dependencies Scanner is a tool for scanning vulnerabilities in a *requirements.txt* file from different GitHub repositories. +It uses `pytest` to run and can generate reports via console output or JSON files. + +The requirements file to scan can be specified with the `repo`, `reference`, and `requirements-path` parameters. +Moreover, the path of the report file generated can be chosen with the `report-path` parameter. ## How to use - Pytest ``` Parameters: --repo: repository name. Default: 'wazuh'. - --branch: branch name of specified repository. Default: 'master'. - --commit: commit of specified repository. Default: None + --reference: branch name of specified repository. Default: 'master'. --requirements-path: requirements file path. Default: 'framework/requirements.txt'. --report-path: output file path. Default: 'dependencies/report_file.json'. ``` ### Scanning wazuh-qa requirements file: ``` -↪ ~/git/wazuh-qa/tests/scans ⊶ feature/1612-package-vuln-scanner ⨘ python3 -m pytest -vv -x --disable-warnings dependencies/ --repo wazuh-qa --branch master --requirements-path requirements.txt +↪ ~/git/wazuh-qa/tests/scans ⊶ feature/1612-package-vuln-scanner ⨘ python3 -m pytest -vv -x --disable-warnings dependencies/ --repo wazuh-qa --reference master --requirements-path requirements.txt ==================================================================================== test session starts ===================================================================================== platform linux -- Python 3.9.5, pytest-6.2.3, py-1.10.0, pluggy-0.13.1 -- /home/kondent/pythonEnv/qa-env/bin/python3 cachedir: .pytest_cache @@ -75,7 +77,7 @@ FAILED dependencies/test_dependencies.py::test_python_dependencies_vuln_scan - A ### Scanning wazuh requirements file with a specific output path: ``` -↪ ~/git/wazuh-qa/tests/scans ⊶ feature/1612-package-vuln-scanner ⨘ python3 -m pytest -vv -x --disable-warnings dependencies/ --repo wazuh --branch master --requirements-path framework/requirements.txt --report-path ~/Desktop/report_file.json +↪ ~/git/wazuh-qa/tests/scans ⊶ feature/1612-package-vuln-scanner ⨘ python3 -m pytest -vv -x --disable-warnings dependencies/ --repo wazuh --reference master --requirements-path framework/requirements.txt --report-path ~/Desktop/report_file.json ==================================================================================== test session starts ===================================================================================== platform linux -- Python 3.9.5, pytest-6.2.3, py-1.10.0, pluggy-0.13.1 -- /home/kondent/pythonEnv/qa-env/bin/python3 cachedir: .pytest_cache @@ -93,4 +95,4 @@ dependencies/test_dependencies.py::test_python_dependencies_vuln_scan PASSED "vulnerabilities_found": 0, "packages": [] } -``` \ No newline at end of file +``` From e105917bb9c8bc3113177f5cf77f2822c4cd69bc Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Gast=C3=B3n=20Palomeque?= Date: Wed, 23 Aug 2023 16:59:19 -0300 Subject: [PATCH 3/7] docs(#4274): Add changelog --- CHANGELOG.md | 1 + 1 file changed, 1 insertion(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index e0a5692943..3736cfc222 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,7 @@ Release report: TBD ### Added +- Add scans environment setup documentation. ([#4444](https://github.com/wazuh/wazuh-qa/pull/4444)) \- (Tests) - Add system test for global group hash ([#4015](https://github.com/wazuh/wazuh-qa/pull/4015)) \- (Tests) - Add tests for new FIM audit buffer option. ([#4485](https://githubhttps://github.com/wazuh/wazuh-qa/pull/4497#pullrequestreview-1654748331.com/wazuh/wazuh-qa/pull/4485)) \- (Framework + tests) - Add tests for merged.mg file generation. ([#4129](https://github.com/wazuh/wazuh-qa/pull/4129)) \- (Tests) From 6106477701afe92773429b4dffff600f9f80a310 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Gast=C3=B3n=20Palomeque?= Date: Tue, 19 Sep 2023 15:28:07 -0300 Subject: [PATCH 4/7] fix(#4274): Skip requirements file empty lines and comments --- deps/wazuh_testing/wazuh_testing/tools/scans/dependencies.py | 2 ++ 1 file changed, 2 insertions(+) diff --git a/deps/wazuh_testing/wazuh_testing/tools/scans/dependencies.py b/deps/wazuh_testing/wazuh_testing/tools/scans/dependencies.py index da953db623..cb5f989e85 100644 --- a/deps/wazuh_testing/wazuh_testing/tools/scans/dependencies.py +++ b/deps/wazuh_testing/wazuh_testing/tools/scans/dependencies.py @@ -63,6 +63,8 @@ def prepare_input(pip_mode, input_file_path): with open(input_file_path, mode='r') as input_file: lines = input_file.readlines() for line in lines: + if line.startswith('\n') or line.startswith('#'): + continue line = re.sub('[<>~]', '=', line) if ',' in line: package_version = max(re.findall(r'\d+\.+\d*\.*\d', line)) From 55d923350f91f40ed0efd2ce3ce45fa0d76fff33 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Gast=C3=B3n=20Palomeque?= Date: Tue, 19 Sep 2023 15:36:16 -0300 Subject: [PATCH 5/7] fix(#4274): Add pyenv note --- tests/scans/dependencies/README.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/tests/scans/dependencies/README.md b/tests/scans/dependencies/README.md index 5241116e52..d2b338b277 100644 --- a/tests/scans/dependencies/README.md +++ b/tests/scans/dependencies/README.md @@ -17,6 +17,9 @@ Parameters: --report-path: output file path. Default: 'dependencies/report_file.json'. ``` ### Scanning wazuh-qa requirements file: + +> The script does not support pyenv version changes because it doesn't expect system variables to be modified + ``` ↪ ~/git/wazuh-qa/tests/scans ⊶ feature/1612-package-vuln-scanner ⨘ python3 -m pytest -vv -x --disable-warnings dependencies/ --repo wazuh-qa --reference master --requirements-path requirements.txt ==================================================================================== test session starts ===================================================================================== From ebd26f4b765f169c34b4f5de38d42573b349e98d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Gast=C3=B3n=20Palomeque?= Date: Wed, 11 Oct 2023 09:29:47 -0300 Subject: [PATCH 6/7] fix(#4274): Style changes --- tests/scans/code_analysis/README.md | 29 ++++++++++------------------- 1 file changed, 10 insertions(+), 19 deletions(-) diff --git a/tests/scans/code_analysis/README.md b/tests/scans/code_analysis/README.md index 729f7ef91c..d6e0c6d1c9 100644 --- a/tests/scans/code_analysis/README.md +++ b/tests/scans/code_analysis/README.md @@ -15,7 +15,7 @@ The contents of this directory are: ## Usage -- Run the test. +- Run the test: `pytest tests/scans/code_analysis/test_python_flaws.py` - If the test passes without failures, everything is correct, and no action is needed. - If the test fails, `wazuh-qa/tests/scans/code_analysis/new_flaws_{module}.json` file will report the new code vulnerabilities found. You should analyze the new vulnerabilities found in the module and report them in GitHub issues. @@ -32,25 +32,16 @@ You can also use more parameters to customize the test functionality. The test w > By default, the test checks the `framework`, `wodles` and `api` directories in the [wazuh/wazuh](https://github.com/wazuh/wazuh) repository's master branch. -- `--repo`: the repository used. - - Default: `wazuh` -- `--reference`: the repository branch. - - Default: `master` -- `--check_directories`: the directories names to check. This parameter accepts multiple directories separated by commas. - - Default: `framework/,api/,wodles/` -- `--exclude_directories`: the directories names to exclude. This parameter accepts multiple directories separated by commas. - - Default: `test/,tests/` -- `--confidence`: the minimum value of confidence of the Bandit scan. This value must be 'UNDEFINED', 'LOW', 'MEDIUM' or 'HIGH'. - - Default: `MEDIUM` -- `--severity`: the minimum value of severity of the Bandit scan. This value must be 'UNDEFINED', 'LOW', 'MEDIUM' or 'HIGH'. - - Default: `LOW` +| Parameter | Description | Default Value | +|-----------------------|-------------------------------------------------------|-------------------| +| `--repo` | The repository to test. | `wazuh` | +| `--reference` | The repository branch. | `master` | +| `--check_directories` | The directories to check (comma-separated). | `framework/,api/,wodles/` | +| `--exclude_directories`| The directories to exclude (comma-separated). | `test/,tests/` | +| `--confidence` | Minimum confidence level for Bandit scan. | `MEDIUM` | +| `--severity` | Minimum severity level for Bandit scan. | `LOW` | +> The values accepted by the flags `--confidence` and `--security` are `UNDEFINED`, `LOW`, `MEDIUM` or `HIGH`. #### Example From 63c6ef3665604c26a47d23ae07b3bfbc56d4298d Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Gast=C3=B3n=20Palomeque?= Date: Wed, 11 Oct 2023 09:30:50 -0300 Subject: [PATCH 7/7] fix(#4274): Update empty lines and comments lookup --- deps/wazuh_testing/wazuh_testing/tools/scans/dependencies.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/deps/wazuh_testing/wazuh_testing/tools/scans/dependencies.py b/deps/wazuh_testing/wazuh_testing/tools/scans/dependencies.py index cb5f989e85..d3f38a9d2c 100644 --- a/deps/wazuh_testing/wazuh_testing/tools/scans/dependencies.py +++ b/deps/wazuh_testing/wazuh_testing/tools/scans/dependencies.py @@ -63,7 +63,7 @@ def prepare_input(pip_mode, input_file_path): with open(input_file_path, mode='r') as input_file: lines = input_file.readlines() for line in lines: - if line.startswith('\n') or line.startswith('#'): + if line.strip().startswith('#') or not line.strip(): continue line = re.sub('[<>~]', '=', line) if ',' in line: