Update scanning tools documentation #4444
Conversation
GGP1
force-pushed
the
update-scanning-tools-docs
branch
from
3b54a89 to
be4d4eb
Compare
There was a problem hiding this comment.
I think it is better to rewrite this sentence:
In order to run the scripts, you should install the wazuh_testing package and the dependencies listed in the requirements file.
To run the scripts, install the wazuh_testing package and the dependencies listed in the requirements file.
There was a problem hiding this comment.
Rewrite proposal to increase clarity in tests/dependencies/README.md, avoiding passive voice phrase and using a direct style.
It's a tool used to scan for vulnerabilities in a requirements.txt file.
It can generate reports via console output or json file. Can be run withpytestand manage to handle remote "requirements.txt" files under github repositories. Requirements file can be specified withrepo,branch,requirements-pathparameters giving flexibility on file location.
It can generate reports via console output or json file. Can be run withpytestand manage to handle remote files under github repositories. Requirements file can be specified withrepo,reference,requirements-pathparameters giving flexibility on file location.
Output file in which the report will be generated can be specified withreport-pathparameter.
The Dependencies Scanner is a tool for scanning vulnerabilities in a requirements.txt file. It can generate reports via console output or JSON file. The Dependencies Scanner uses pytest to run. The tool scans remote requirement files located in GitHub repositories. Requirements files can be specified with repo, reference, and requirements-path parameters, giving flexibility on a file location. The report-path parameter can specify the output file that the tool generates.
There was a problem hiding this comment.
I think that the whole code_analysis README.md file needs to be improved. This is my proposal, please review if the content is fine or needs to be corrected.
Proposed Text
Code Analysis
The code_analysis directory contains Python tests to verify potential vulnerabilities in the Wazuh Python code.
Test Python Flaws
Description
test_python_flaws.py is a Pytest test located at wazuh-qa/tests/scans/code_analysis directory. The test looks for vulnerabilities in Python code directories and uses Bandit to search for these potential flaws.
The test checks the framework/, api/ and wodles/ directories of the Wazuh repository by default, comparing the Bandit output with the vulnerabilities identified as false positives or vulnerabilities to fix. It saves the results in three JSON files (one JSON file for each module).
The contents of the code_analysis directory are:
README.mdfile: this file.known_flawsdirectory: The directory contains three JSON files, one for each module (api, framework and wodles). Each file has a dictionary with two keys: false_positives and to_fix. The values are the list of vulnerabilities considered false positives and the list of vulnerabilities you must fix (with issues). After running the test and analyzing the new vulnerabilities, you must edit these files.conftest.py: The Pytest configuration file. It adds the possibility to use specific parameters when passing the test.test_python_flaws.py: The test itself. You should run this test using the same Python virtual environment used in the Wazuh framework and API unit tests. If the test fails, a new JSON file will be created in wazuh-qa/tests/scans/code_analysis, showing information about the possible new vulnerabilities found.
Usage
The usage for this test is the following:
- Run the test.
- If the test passes without failures, everything is correct, and no action is needed.
- If the test fails, wazuh-qa/tests/scans/code_analysis/new_flaws_{module}.json file will report the new code vulnerabilities found.
You should analyze the new vulnerabilities found in the module and report them in GitHub issues.
If you need to fix a new vulnerability, add it to the to_fix key module's JSON file entry found in the known_flaws. If the new vulnerability is a false positive, add it to the false_positives key module's JSON file entry found in the known_flaws. The test updates the known_flaws files automatically with information like the line number or range of the flaws in the to_fix dictionary. The test also removes flaws from the known_flaws file if Bandit did not report them.
Parameters
As the description says, the test uses Bandit to look for possible Python flaws. By default, the test checks the framework, wodles and api directories in the Wazuh repository's master branch.
You can set the directories, repository, and branch parameters to test any directory containing Python code inside the Wazuh organization.
You can also use more parameters to customize the test functionality. The test will only succeed if you check different directories and repositories, as you don't have known_flaws files for non-default directories.
- --repo: the repository used. Default: wazuh
- --reference: the repository branch. Default: master
- --check_directories: the directory/ies name/s to check; This parameter accepts multiple directories separated by commas. Default: framework/,api/,wodles/.
- --exclude_directories: the directory/ies name/s to exclude; This parameter accepts multiple directories separated by commas. Default: test/,tests/.
- --confidence: the minimum value of confidence of the Bandit scan. This value must be 'UNDEFINED', 'LOW', 'MEDIUM' or 'HIGH'. Default: MEDIUM
- --severity: the minimum value of severity of the Bandit scan. This value must be 'UNDEFINED', 'LOW', 'MEDIUM' or 'HIGH'. Default: LOW
@mhamra I think this is out of the scope of the issue, but given that it's just documentation changes I'll go ahead and implement them |
GGP1
force-pushed
the
update-scanning-tools-docs
branch
3 times, most recently
from
50688f7 to
2c9e219
Compare
GGP1
force-pushed
the
update-scanning-tools-docs
branch
from
2c9e219 to
9b97442
Compare
GGP1
force-pushed
the
update-scanning-tools-docs
branch
from
a79b351 to
4517545
Compare
GGP1
force-pushed
the
update-scanning-tools-docs
branch
from
4517545 to
63c6ef3
Compare
There was a problem hiding this comment.
LGTM
Dependencies Scan 🟢
tests/scans/dependencies/test_dependencies.py::test_python_dependencies_vuln_scan FAILED [100%]
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> traceback >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
pytestconfig = <_pytest.config.Config object at 0x7fcb30adb340>
def test_python_dependencies_vuln_scan(pytestconfig):
"""Check that the specified dependencies do not have any known vulnerabilities.
Args:
pytestconfig (fixture): Fixture that returns the :class:`_pytest.config.Config` object.
"""
branch = pytestconfig.getoption('--reference')
repo = pytestconfig.getoption('--repo')
requirements_path = pytestconfig.getoption('--requirements-path')
report_path = pytestconfig.getoption('--report-path')
requirements_url = f"https://raw.githubusercontent.com/wazuh/{repo}/{branch}/{requirements_path}"
urlretrieve(requirements_url, REQUIREMENTS_TEMP_FILE.name)
result = report_for_pytest(REQUIREMENTS_TEMP_FILE.name,
os.path.join(os.path.dirname(os.path.abspath(__file__)), "known_flaws_deps.json"))
REQUIREMENTS_TEMP_FILE.close()
export_report(result, report_path)
> assert loads(result)['vulnerabilities_found'] == 0, f'Vulnerables packages were found, full report at: ' \
f"{report_path}"
E AssertionError: Vulnerables packages were found, full report at: test_file.json
E assert 25 == 0
tests/scans/dependencies/test_dependencies.py:30: AssertionError
Description
Adds a README file with instructions on how to setup the environment to run the vulnerability scanning tools and modifies the documentations from both scripts to reference to the correct parameters. It also adds a line to the dependencies script to skip empty lines and comments in requirements files.
Added
Updated
Executions
Run 1
Run 2
Run 3
Report
{ "report_date": "2023-09-19T15:23:42.782823", "vulnerabilities_found": 19, "packages": [ { "package_name": "python", "package_version": "3.9.16", "package_affected_version": "<3.11.3", "vuln_description": "The e-mail module of Python 0 - 2.7.18, 3.x - 3.11 incorrectly parses e-mail addresses which contain a special character. This vulnerability allows attackers to send messages from e-mail addresses that would otherwise be rejected.\r\nhttps://github.com/python/cpython/issues/102988\r\nhttps://python-security.readthedocs.io/vuln/email-parseaddr-realname.html", "safety_id": "55080" }, { "package_name": "python", "package_version": "3.9.16", "package_affected_version": ">0", "vuln_description": "In difflib module, table header in output of difflib.HtmlDiff.make_table is not escaped and can be rendered as code in the browser, leading potentially to XSS.\r\nhttps://bugs.python.org/issue35603\r\nhttps://github.com/python/cpython/commit/44e36e80456dabaeb59c6e2a93e0c1322bfeb179", "safety_id": "45612" }, { "package_name": "python", "package_version": "3.9.16", "package_affected_version": ">0", "vuln_description": "Lib/zipfile.py in Python allows remote attackers to cause a denial of service (resource consumption) via a ZIP bomb.\r\nhttps://bugs.python.org/issue36260", "safety_id": "45621" }, { "package_name": "python", "package_version": "3.9.16", "package_affected_version": ">=3.9.0a0,<3.9.17", "vuln_description": "An issue in the urllib.parse component of Python before 3.11.4 allows attackers to bypass blocklisting methods by supplying a URL that starts with blank characters.\r\nhttps://pointernull.com/security/python-url-parse-problem.html", "safety_id": "53376" }, { "package_name": "python", "package_version": "3.9.16", "package_affected_version": ">=3.9.0a1,<3.9.18", "vuln_description": "An issue was discovered in Python before 3.8.18, 3.9.x before 3.9.18, 3.10.x before 3.10.13, and 3.11.x before 3.11.5. It primarily affects servers (such as HTTP servers) that use TLS client authentication. If a TLS server-side socket is created, receives data into the socket buffer, and then is closed quickly, there is a brief window where the SSLSocket instance will detect the socket as \"not connected\" and won't initiate a handshake, but buffered data will still be readable from the socket buffer. This data will not be authenticated if the server-side TLS peer is expecting client certificate authentication, and is indistinguishable from valid TLS stream data. Data is limited in size to the amount that will fit in the buffer. (The TLS connection cannot directly be used for data exfiltration because the vulnerable code path requires that the connection be closed on initialization of the SSLSocket.)", "safety_id": "60680" }, { "package_name": "aiohttp", "package_version": "3.8.1", "package_affected_version": "<=3.8.4", "vuln_description": "Aiohttp 3.8.5 includes a fix for CVE-2023-37276: Sending a crafted HTTP request will cause the server to misinterpret one of the HTTP header values leading to HTTP request smuggling.\r\nhttps://github.com/aio-libs/aiohttp/commit/9337fb3f2ab2b5f38d7e98a194bde6f7e3d16c40\r\nhttps://github.com/aio-libs/aiohttp/security/advisories/GHSA-45c4-8wx5-qw6w", "safety_id": "59725" }, { "package_name": "certifi", "package_version": "2022.12.7", "package_affected_version": ">=2015.04.28,<2023.07.22", "vuln_description": "Certifi 2023.07.22 includes a fix for CVE-2023-37920: Certifi prior to version 2023.07.22 recognizes \"e-Tugra\" root certificates. e-Tugra's root certificates were subject to an investigation prompted by reporting of security issues in their systems. Certifi 2023.07.22 removes root certificates from \"e-Tugra\" from the root store.\r\nhttps://github.com/certifi/python-certifi/security/advisories/GHSA-xqr8-7jwr-rhp7", "safety_id": "59956" }, { "package_name": "cryptography", "package_version": "3.3.2", "package_affected_version": "<41.0.0", "vuln_description": "Cryptography 41.0.0 updates its dependency 'OpenSSL' to v3.1.1 to include a security fix.\r\nhttps://github.com/pyca/cryptography/commit/8708245ccdeaff21d65eea68a4f8d2a7c5949a22", "safety_id": "59062" }, { "package_name": "cryptography", "package_version": "3.3.2", "package_affected_version": "<41.0.2", "vuln_description": "The cryptography package before 41.0.2 for Python mishandles SSH certificates that have critical options.", "safety_id": "59473" }, { "package_name": "cryptography", "package_version": "3.3.2", "package_affected_version": ">=0.8, <41.0.3", "vuln_description": "Cryptography 41.0.3 includes a fix for an Insufficient Verification of Data Authenticity vulnerability. Applications that use the AES-SIV algorithm and want to authenticate empty data entries as associated data can be misled by removing adding or reordering such empty entries as these are ignored by the OpenSSL implementation. The AES-SIV algorithm allows for the authentication of multiple associated data entries along with encryption. To authenticate empty data the application has to call EVP_EncryptUpdate() or EVP_CipherUpdate() with a NULL pointer as the output buffer and 0 as the input buffer length. The AES-SIV implementation in OpenSSL just returns success for such a call instead of performing the associated data authentication operation. The empty data thus will not be authenticated.\r\nhttps://github.com/pyca/cryptography/commit/bfa4d95f0f356f2d535efd5c775e0fb3efe90ef2", "safety_id": "60224" }, { "package_name": "cryptography", "package_version": "3.3.2", "package_affected_version": ">=0.8, <41.0.3", "vuln_description": "Cryptography 41.0.3 includes a fix for a Denial of Service vulnerability. The function DH_check() performs various checks on DH parameters. One of those checks confirms that the modulus ('p' parameter) is not too large. Trying to use a very large modulus is slow and OpenSSL will not normally use a modulus that is over 10,000 bits in length. However, the DH_check() function checks numerous aspects of the key or parameters that have been supplied. Some of those checks use the supplied modulus value even if it has already been found to be too large. An application that calls DH_check() and supplies a key or parameters obtained from an untrusted source could be vulnerable to a Denial of Service attack. The function DH_check() is itself called by a number of other OpenSSL functions. The other functions affected by this are DH_check_ex() and EVP_PKEY_param_check(). Also vulnerable are the OpenSSL dhparam and pkeyparam command line applications when using the '-check' option.\r\nhttps://github.com/openssl/openssl/commit/1fa20cf2f506113c761777127a38bce5068740eb", "safety_id": "60225" }, { "package_name": "cryptography", "package_version": "3.3.2", "package_affected_version": ">=0.8,<41.0.3", "vuln_description": "Cryptography 41.0.3 includes a fix for a Denial of Service vulnerability. Checking excessively long DH keys or parameters may be very slow. The package that uses the functions DH_check(), DH_check_ex(), or EVP_PKEY_param_check() to check a DH key or DH parameters may experience long delays. Where the key or parameters that are being checked have been obtained from an untrusted source this may lead to a Denial of Service. The function DH_check() performs various checks on DH parameters. The function DH_check() is itself called by a number of other OpenSSL functions. The other functions affected by this are DH_check_ex() and EVP_PKEY_param_check(). Also vulnerable are the OpenSSL dhparam and pkeyparam command line applications when using the \"-check\" option.\r\nhttps://github.com/openssl/openssl/commit/1c16253f3c3a8d1e25918c3f404aae6a5b0893de\r\nhttps://github.com/openssl/openssl/commit/91ddeba0f2269b017dc06c46c993a788974b1aa5", "safety_id": "60223" }, { "package_name": "grpcio", "package_version": "1.38.1", "package_affected_version": "<1.53.0", "vuln_description": "Grpcio 1.53.0 includes a fix for a Reachable Assertion vulnerability. \r\nhttps://github.com/advisories/GHSA-6628-q6j9-w8vg", "safety_id": "59867" }, { "package_name": "grpcio", "package_version": "1.38.1", "package_affected_version": "<1.53.0", "vuln_description": "Grpcio 1.53.0 includes a fix for a Connection Confusion vulnerability. When gRPC HTTP2 stack raised a header size exceeded error, it skipped parsing the rest of the HPACK frame. This caused any HPACK table mutations to also be skipped, resulting in a desynchronization of HPACK tables between sender and receiver. If leveraged, say, between a proxy and a backend, this could lead to requests from the proxy being interpreted as containing headers from different proxy clients - leading to an information leak that can be used for privilege escalation or data exfiltration.\r\nhttps://github.com/advisories/GHSA-cfgp-2977-2fmm", "safety_id": "59869" }, { "package_name": "grpcio", "package_version": "1.38.1", "package_affected_version": "<1.53.0", "vuln_description": "Grpcio 1.53.0 includes a fix for a Connection Termination vulnerability. The prior versions contain a vulnerability whereby a client can cause a termination of connection between a HTTP2 proxy and a gRPC server: a base64 encoding error for -bin suffixed headers will result in a disconnection by the gRPC server, but is typically allowed by HTTP2 proxies.\r\nhttps://github.com/advisories/GHSA-9hxf-ppjv-w6rq", "safety_id": "59868" }, { "package_name": "numpy", "package_version": "1.21.6", "package_affected_version": "<1.22.0", "vuln_description": "Numpy 1.22.0 includes a fix for CVE-2021-34141: An incomplete string comparison in the numpy.core component in NumPy before 1.22.0 allows attackers to trigger slightly incorrect copying by constructing specific string objects. \r\nNOTE: the vendor states that this reported code behavior is \"completely harmless.\"\r\nhttps://github.com/numpy/numpy/issues/18993", "safety_id": "44717" }, { "package_name": "numpy", "package_version": "1.21.6", "package_affected_version": "<1.22.0", "vuln_description": "Numpy 1.22.0 includes a fix for CVE-2021-41496: Buffer overflow in the array_from_pyobj function of fortranobject.c, which allows attackers to conduct a Denial of Service attacks by carefully constructing an array with negative values. \r\nNOTE: The vendor does not agree this is a vulnerability; the negative dimensions can only be created by an already privileged user (or internally).\r\nhttps://github.com/numpy/numpy/issues/19000", "safety_id": "44716" }, { "package_name": "numpy", "package_version": "1.21.6", "package_affected_version": "<1.22.2", "vuln_description": "Numpy 1.22.2 includes a fix for CVE-2021-41495: Null Pointer Dereference vulnerability exists in numpy.sort in NumPy in the PyArray_DescrNew function due to missing return-value validation, which allows attackers to conduct DoS attacks by repetitively creating sort arrays. \r\nNOTE: While correct that validation is missing, an error can only occur due to an exhaustion of memory. If the user can exhaust memory, they are already privileged. Further, it should be practically impossible to construct an attack which can target the memory exhaustion to occur at exactly this place.\r\nhttps://github.com/numpy/numpy/issues/19038", "safety_id": "44715" }, { "package_name": "requests", "package_version": "2.25.1", "package_affected_version": ">=2.3.0,<2.31.0", "vuln_description": "Requests 2.31.0 includes a fix for CVE-2023-32681: Since Requests 2.3.0, Requests has been leaking Proxy-Authorization headers to destination servers when redirected to an HTTPS endpoint. This is a product of how we use 'rebuild_proxies' to reattach the 'Proxy-Authorization' header to requests. For HTTP connections sent through the tunnel, the proxy will identify the header in the request itself and remove it prior to forwarding to the destination server. However when sent over HTTPS, the 'Proxy-Authorization' header must be sent in the CONNECT request as the proxy has no visibility into the tunneled request. This results in Requests forwarding proxy credentials to the destination server unintentionally, allowing a malicious actor to potentially exfiltrate sensitive information.", "safety_id": "58755" } ] }