[ZAP] Ajax spider requires a lot of shared memory (#196)

* [ZAP] Ajax spider requires a lot of shared memory

The Selenium environment set up by ZAP for the Ajax Spider requires a
lot of shared memory (/dev/shm in Linux)

This commit does the following:
- Update the README troubleshooting section, for when the RapiDAST image
  is used
- In Podman mode: if Ajax is used, automatically ask podman to have 2GB
  of shared memory
- Added corresponding pytest
- Fixed `find_context()`, which broke when context was not found (that
  should happen only in pytest)

.pre-commit-config.yaml

@@ -41,8 +41,8 @@ repos:
   # R1710: Either all return statements in a function should return an expression, or none of them should. (inconsistent-return-statements)
 
   - repo: https://github.com/PyCQA/pylint
-    rev: v3.0.3
-    #rev: v2.17.4
+    #rev: v3.0.3
+    rev: v2.17.4
     hooks:
       - id: pylint
         exclude: ^tests/

README.md

@@ -567,6 +567,27 @@ Solutions:
 * If you are using a Swagger v2 definition, try converting it to v3 (OpenAPI)
 * Set a `maxYamlCodePoints` Java proprety with a big value, which can be passed using environment variables (via the `config.environ.envFile` config entry): `_JAVA_OPTIONS=-DmaxYamlCodePoints=99999999`
 
+### ZAP's Ajax Spider failing
+
+#### Insufficient shared memory
+
+ZAP's Ajax Spider makes heavy use of shared memory (`/dev/shm/`). When using the RapiDAST image or the ZAP image, the user needs to make sure that sufficient space is available in `/dev/shm/` (in podman, by default, its size is 64MB). A size of 2G would be the minimum recommended. In podman for example, the option would be `--shm-size=2g`.
+
+ZAP logs that would bring evidence of a lack of shared memory would look like the following:
+
+```
+2024-07-04 11:21:32,061 [ZAP-AjaxSpiderAuto] WARN  SpiderThread - Failed to start browser firefox-headless
+com.google.inject.ProvisionException: Unable to provision, see the following errors:
+
+1) [Guice/ErrorInCustomProvider]: SessionNotCreatedException: Could not start a new session. Response code 500. Message: Failed to decode response from marionette
+```
+
+Or the following:
+
+```
+2024-07-04 12:23:28,027 [ZAP-AjaxSpiderAuto] ERROR UncaughtExceptionLogger - Exception in thread "ZAP-AjaxSpiderAuto"
+java.lang.OutOfMemoryError: unable to create native thread: possibly out of memory or process/resource limits reached
+```
 
 ## Caveats
 

scanners/zap/zap.py

@@ -876,11 +876,11 @@ def ensure_list(entry):
     except:
         pass
     logging.warning(
-        f"No context matching {context} have ben found in the current Automation Framework configuration.",
+        f"No context matching {context} have ben found in the current Automation Framework configuration."
         "It may be missing from default. An empty context is created",
     )
     # something failed: create an empty one and return it
-    if not automation_config["env"]:
+    if not automation_config.get("env"):
         automation_config["env"] = {}
     if not automation_config["env"].get("contexts"):
         automation_config["env"]["contexts"] = []

scanners/zap/zap_podman.py

@@ -153,6 +153,22 @@ def cleanup(self):
         if not self.state == State.ERROR:
             self.state = State.CLEANEDUP
 
+    ###############################################################
+    # OVERLOADED METHODS                                          #
+    # Method overloading parent class                             #
+    ###############################################################
+
+    def _setup_ajax_spider(self):
+        """Ajax requires a lot of shared memory"""
+
+        if self.my_conf("spiderAjax", default=False) is False:
+            return
+
+        self.podman.add_option("--shm-size", "2g")
+
+        # Regular Ajax setup
+        super()._setup_ajax_spider()
+
     ###############################################################
     # PROTECTED METHODS                                           #
     # Accessed by Zap parent only                                 #

tests/scanners/zap/test_setup_podman.py

@@ -184,6 +184,18 @@ def test_setup_podman_pod_injection(test_config):
 # Misc tests
 
 
+def test_podman_handling_ajax(test_config):
+    test_config.set("scanners.zap.spiderAjax.url", "https://abcdef.jklm")
+    test_zap = ZapPodman(config=test_config)
+    # create a fake automation framework: just an empty `jobs` is sufficient
+    test_zap.automation_config = {"jobs": []}
+    test_zap._setup_ajax_spider()
+
+    cli = test_zap.podman.get_complete_cli()
+    i = cli.index("--shm-size")
+    assert cli[i + 1] == "2g"
+
+
 def test_podman_handling_plugins(test_config):
     test_config.set("scanners.zap.miscOptions.updateAddons", True)
     test_config.set("scanners.zap.miscOptions.additionalAddons", "pluginA,pluginB")