Add ctibulter api key (#105)

* add support for CTIBUTLER API Key #104 * Update .env.markdown * replace _HOST with _BASE_URL * fix typo * fix typo * fix typo * Update .env.markdown --------- Co-authored-by: Fadl <[email protected]>
muchdogesec · Jan 7, 2025 · 8e5db68 · 8e5db68
1 parent b8bf255
commit 8e5db68
Show file tree

Hide file tree

Showing 5 changed files with 44 additions and 38 deletions.
diff --git a/.env.example b/.env.example
@@ -7,6 +7,8 @@ TEMPERATURE=
 ## BIN LIST
 BIN_LIST_API_KEY=
 ## CTIBUTLER
-CTIBUTLER_HOST=
+CTIBUTLER_BASE_URL=
+CTIBUTLER_API_KEY=
 ## VULMATCH
-VULMATCH_HOST=
+VULMATCH_BASE_URL=
+VULMATCH_API_KEY=
diff --git a/.env.markdown b/.env.markdown
@@ -26,12 +26,16 @@ However, if you just want to experiment, set the following values
 
 Obstracts requires [ctibutler](https://github.com/muchdogesec/ctibutler) to lookup ATT&CK, CAPEC, CWE, ATLAS, and locations in blogs
 
-* `CTIBUTLER_HOST`: `'http://host.docker.internal:8006'`
-	* If you are running CTI Butler locally, be sure to set `'http://host.docker.internal:8006'` in the `.env` file otherwise you will run into networking errors.
+* `CTIBUTLER_BASE_URL`: `'http://api.ctibutler.com'` (recommended)
+	* If you are running CTI Butler locally, be sure to set `'http://host.docker.internal:8006/api/'` in the `.env` file otherwise you will run into networking errors.
+* `CTIBUTLER_API_KEY`:
+	* If using `'http://api.ctibutler.com'`, [get your API key here](http://app.ctibutler.com). Can be left blank if running locally.
 
 ## VULMATCH FOR CVE AND CPE LOOKUPS
 
 Obstracts requires [vulmatch](https://github.com/muchdogesec/vulmatch) to lookup CVEs and CPEs in blogs
 
-* `VULMATCH_HOST`: `'http://host.docker.internal:8005'`
-	* If you are running vulmatch locally, be sure to set `'http://host.docker.internal:8005'` in the `.env` file otherwise you will run into networking errors.
+* `VULMATCH_BASE_URL`: `'http://api.vulmatch.com'` (recommended)
+	* If you are running Vulmatch locally, be sure to set `'http://host.docker.internal:8005/api/'` in the `.env` file otherwise you will run into networking errors.
+* `VULMATCH_API_KEY`:
+	* If using `'http://api.vulmatch.com'`, [get your API key here](http://app.vulmatch.com). Can be left blank if running locally.
diff --git a/docs/stix-mapping.md b/docs/stix-mapping.md
@@ -1694,7 +1694,7 @@ Some objects created for extractions do not need to be generated by txt2stix, th
 Takes the extracted ID and passes it to
 
 ```shell
-GET CTIBUTLER_HOST/api/v1/attack-enterprise/objects/:attack_id/
+GET CTIBUTLER_BASE_URL/v1/attack-enterprise/objects/:attack_id/
 ```
 
 `CTIBUTLER_APIKEY` in request passed if set.
@@ -1711,23 +1711,23 @@ Relationship mode object generation behaviour:
 Takes the extracted name and passes it to;
 
 ```shell
-GET CTIBUTLER_HOST/api/v1/attack-enterprise/objects/?name=<VALUE>
+GET CTIBUTLER_BASE_URL/v1/attack-enterprise/objects/?name=<VALUE>
 ```
 
 ### stix-mapping: `ctibutler-mitre-attack-enterprise-alias`
 
 Takes the extracted alias and passes it to;
 
 ```shell
-GET CTIBUTLER_HOST/api/v1/attack-enterprise/objects/?alias=<VALUE>
+GET CTIBUTLER_BASE_URL/v1/attack-enterprise/objects/?alias=<VALUE>
 ```
 
 ### stix-mapping: `ctibutler-mitre-attack-mobile-id`
 
 Takes the extracted ID and passes it to
 
 ```shell
-GET CTIBUTLER_HOST/api/v1/attack-mobile/objects/:attack_id/
+GET CTIBUTLER_BASE_URL/v1/attack-mobile/objects/:attack_id/
 ```
 
 All the objects returned are imported.
@@ -1742,23 +1742,23 @@ Relationship mode object generation behaviour:
 Takes the extracted name and passes it to;
 
 ```shell
-GET CTIBUTLER_HOST/api/v1/attack-mobile/objects/?name=<VALUE>
+GET CTIBUTLER_BASE_URL/v1/attack-mobile/objects/?name=<VALUE>
 ```
 
 ### stix-mapping: `ctibutler-mitre-attack-mobile-alias`
 
 Takes the extracted alias and passes it to;
 
 ```shell
-GET CTIBUTLER_HOST/api/v1/attack-mobile/objects/?alias=<VALUE>
+GET CTIBUTLER_BASE_URL/v1/attack-mobile/objects/?alias=<VALUE>
 ```
 
 ### stix-mapping: `ctibutler-mitre-attack-ics-id`
 
 Takes the extracted ID and passes it to
 
 ```shell
-GET CTIBUTLER_HOST/api/v1/attack-ics/objects/:attack_id/
+GET CTIBUTLER_BASE_URL/v1/attack-ics/objects/:attack_id/
 ```
 
 `CTIBUTLER_APIKEY` in request passed if set.
@@ -1775,23 +1775,23 @@ Relationship mode object generation behaviour:
 Takes the extracted name and passes it to;
 
 ```shell
-GET CTIBUTLER_HOST/api/v1/attack-ics/objects/?name=<VALUE>
+GET CTIBUTLER_BASE_URL/v1/attack-ics/objects/?name=<VALUE>
 ```
 
 ### stix-mapping: `ctibutler-mitre-attack-ics-alias`
 
 Takes the extracted alias and passes it to;
 
 ```shell
-GET CTIBUTLER_HOST/api/v1/attack-ics/objects/?alias=<VALUE>
+GET CTIBUTLER_BASE_URL/v1/attack-ics/objects/?alias=<VALUE>
 ```
 
 ### stix-mapping: `ctibutler-mitre-capec-id`
 
 Takes the extracted ID and passes it to 
 
 ```shell
-GET CTIBUTLER_HOST/api/v1/capec/objects/:capec_id/
+GET CTIBUTLER_BASE_URL/v1/capec/objects/:capec_id/
 ```
 
 `CTIBUTLER_APIKEY` in request passed if set.
@@ -1808,15 +1808,15 @@ Relationship mode object generation behaviour:
 Takes the extracted name and passes it to;
 
 ```shell
-GET CTIBUTLER_HOST/api/v1/capec/objects/?name=<VALUE>
+GET CTIBUTLER_BASE_URL/v1/capec/objects/?name=<VALUE>
 ```
 
 ### stix-mapping: `ctibutler-mitre-cwe-id`
 
 Takes the extracted ID and passes it to 
 
 ```shell
-GET CTIBUTLER_HOST/api/v1/cwe/objects/:cwe_id/
+GET CTIBUTLER_BASE_URL/v1/cwe/objects/:cwe_id/
 ```
 
 `CTIBUTLER_APIKEY` in request passed if set.
@@ -1831,15 +1831,15 @@ Relationship mode object generation behaviour:
 Takes the extracted name and passes it to;
 
 ```shell
-GET CTIBUTLER_HOST/api/v1/cwe/objects/?name=<VALUE>
+GET CTIBUTLER_BASE_URL/v1/cwe/objects/?name=<VALUE>
 ```
 
 ### stix-mapping: `ctibutler-mitre-atlas-id`
 
 Takes the extracted ID and passes it to
 
 ```shell
-GET CTIBUTLER_HOST/api/v1/atlas/objects/:atlas_id/
+GET CTIBUTLER_BASE_URL/v1/atlas/objects/:atlas_id/
 ```
 
 `CTIBUTLER_APIKEY` in request passed if set.
@@ -1856,15 +1856,15 @@ Relationship mode object generation behaviour:
 Takes the extracted name and passes it to;
 
 ```shell
-GET CTIBUTLER_HOST/api/v1/atlas/objects/?name=<VALUE>
+GET CTIBUTLER_BASE_URL/v1/atlas/objects/?name=<VALUE>
 ```
 
 ### stix-mapping: `ctibutler-disarm-id`
 
 Takes the extracted ID and passes it to
 
 ```shell
-GET CTIBUTLER_HOST/api/v1/disarm/objects/:disarm_id/
+GET CTIBUTLER_BASE_URL/v1/disarm/objects/:disarm_id/
 ```
 
 `CTIBUTLER_APIKEY` in request passed if set.
@@ -1881,15 +1881,15 @@ Relationship mode object generation behaviour:
 Takes the extracted name and passes it to;
 
 ```shell
-GET CTIBUTLER_HOST/api/v1/atlas/objects/?name=<VALUE>
+GET CTIBUTLER_BASE_URL/v1/atlas/objects/?name=<VALUE>
 ```
 
 ### stix-mapping: `ctibutler-location`
 
 Takes the extracted ID and passes it to
 
 ```shell
-GET CTIBUTLER_HOST/api/v1/location/objects/?alpha2_code=ID
+GET CTIBUTLER_BASE_URL/v1/location/objects/?alpha2_code=ID
 ```
 
 `CTIBUTLER_APIKEY` in request passed if set.
@@ -1906,7 +1906,7 @@ Relationship mode object generation behaviour:
 Takes the extracted ID and passes it to 
 
 ```shell
-GET VULMATCH_HOST/api/v1/cve/objects/:cve_id/
+GET VULMATCH_BASE_URL/v1/cve/objects/:cve_id/
 ```
 
 `VULMATCH_APIKEY` in request passed if set.
@@ -1921,7 +1921,7 @@ Relationship mode object generation behaviour:
 Takes the extracted ID and passes it to 
 
 ```shell
-GET VULMATCH_HOST/api/v1/cpe/objects/:cpe_id/
+GET VULMATCH_BASE_URL/v1/cpe/objects/:cpe_id/
 ```
 
 `VULMATCH_APIKEY` in request passed if set.

diff --git a/txt2stix/retriever.py b/txt2stix/retriever.py
@@ -9,34 +9,34 @@
 class STIXObjectRetriever:
     def __init__(self, host="ctibutler") -> None:
         if host == "ctibutler":
-            self.api_root = os.environ['CTIBUTLER_HOST']
-            self.api_key = os.environ.get('CTIBUTLER_APIKEY')
+            self.api_root = os.environ['CTIBUTLER_BASE_URL'] + '/'
+            self.api_key = os.environ.get('CTIBUTLER_API_KEY')
         elif host == "vulmatch":
-            self.api_root = os.environ['VULMATCH_HOST']
-            self.api_key = os.environ.get('VULMATCH_APIKEY')
+            self.api_root = os.environ['VULMATCH_BASE_URL'] + '/'
+            self.api_key = os.environ.get('VULMATCH_API_KEY')
         else:
             raise NotImplementedError("The type `%s` is not supported", host)
 
     def get_attack_objects(self, matrix, attack_id):
-        endpoint = urljoin(self.api_root, f"/api/v1/attack-{matrix}/objects/{attack_id}/")
+        endpoint = urljoin(self.api_root, f"v1/attack-{matrix}/objects/{attack_id}/")
         return self._retrieve_objects(endpoint)
 
     def get_objects_by_id(self, id, type):
-        return self._retrieve_objects(urljoin(self.api_root, f"/api/v1/{type}/objects/{id}/"))
+        return self._retrieve_objects(urljoin(self.api_root, f"v1/{type}/objects/{id}/"))
 
     def get_location_objects(self, id):
-        return self._retrieve_objects(urljoin(self.api_root, f"/api/v1/location/objects/?alpha2_code={id}"))
+        return self._retrieve_objects(urljoin(self.api_root, f"v1/location/objects/?alpha2_code={id}"))
 
     def get_objects_by_name(self, name, type):
-        return self._retrieve_objects(urljoin(self.api_root, f"/api/v1/{type}/objects/?name={name}"))
+        return self._retrieve_objects(urljoin(self.api_root, f"v1/{type}/objects/?name={name}"))
 
     def get_objects_by_alias(self, alias, type):
-        return self._retrieve_objects(urljoin(self.api_root, f"/api/v1/{type}/objects/?alias={alias}"))
+        return self._retrieve_objects(urljoin(self.api_root, f"v1/{type}/objects/?alias={alias}"))
 
     def _retrieve_objects(self, endpoint, key='objects'):
         s = requests.Session()
         s.headers.update({
-            "Authority": f"Bearer {self.api_key}"
+            "API-KEY": self.api_key,
         })
         data = []
         page = 1

diff --git a/txt2stix/txt2stix.py b/txt2stix/txt2stix.py
@@ -160,8 +160,8 @@ def parse_args():
 
 REQUIRED_ENV_VARIABLES = [
     "INPUT_TOKEN_LIMIT",
-    "CTIBUTLER_HOST",
-    "VULMATCH_HOST",
+    "CTIBUTLER_BASE_URL",
+    "VULMATCH_BASE_URL",
 ]
 def load_env():
     dotenv.load_dotenv()