Add Game Boy Advance

- Add Game Boy Advance - Language parsing can now handle languages formatted like "En+De" (and test updated) - ROMParser will now filter out RetroAchievements subsets, since they're hacks - When checking for RAPatch matches, if the check is a list will simply check there's something in the list subset - RAPatch checks now includes modern/improved/demoted versions - ROMPatcher now supports RomPatcher.js - ROMCleaner clears patched files out of cache - ROMPatcher unquotes patch URL before downloading - Updated GUI for RomPatcher.js options - Added known issue for long path names
bbtufty · Dec 10, 2024 · 054f58c · 054f58c
1 parent 8b992c8
commit 054f58c
Show file tree

Hide file tree

Showing 21 changed files with 407 additions and 32 deletions.
diff --git a/CHANGES.rst b/CHANGES.rst
@@ -1,3 +1,38 @@
+0.1.1 (Unreleased)
+==================
+
+Features
+--------
+
+- Added Game Boy Advance
+- ROMPatcher now supports RomPatcher.js
+
+Fixes
+-----
+
+ROMCleaner
+~~~~~~~~~~
+
+- Ensure we clear patched files out of cache
+
+ROMParser
+~~~~~~~~~
+
+- ROMParser will now filter out RetroAchievements subsets, since they're hacks
+- When checking for RAPatch matches, if the check is a list will simply check there's something in the list subset
+
+ROMPatcher
+~~~~~~~~~~
+
+- Unquote patch URL before downloading
+
+General
+~~~~~~~
+
+- Added known issue for long filenames
+- RAPatch checks now includes modern/improved/demoted versions
+- Language parsing can now handle languages formatted like "En+De" (and test updated)
+
 0.1.0 (2024-12-04)
 ==================
 

diff --git a/README.md b/README.md
@@ -22,21 +22,25 @@ ROMSearch offers the ability to:
 * Moving files to a structured location, including potentially additional files that may be needed
 * Discord integration so users can see the results of runs in a simple, clean way
 
-To get started, see the [documentation](https://romsearch.readthedocs.io/en/latest/).
+To get started, see the [documentation](https://romsearch.readthedocs.io/en/latest/). For known issues and workarounds, 
+see the [known issues](https://romsearch.readthedocs.io/en/latest/known_issues.html).
 
 Currently, ROMSearch is in early development, and so many features may be added over time. At the moment, ROMSearch
 works for the following consoles:
 
-* Nintendo
+* Nintendo (Handheld)
    * Game Boy
    * Game Boy Color
+   * Game Boy Advance
+* Nintendo (Home)
    * Nintendo Entertainment System
    * Super Nintendo Entertainment System
    * GameCube
-* Sony
+* Sony (Handheld)
+   * PlayStation Portable
+* Sony (Home)
    * PlayStation
    * PlayStation 2
-   * PlayStation Portable
 
 but be aware there may be quirks that will only become apparent over time. We encourage users to open
 [issues](https://github.com/bbtufty/romsearch/issues) as and where they find them.
diff --git a/docs/configs/config.rst b/docs/configs/config.rst
@@ -85,8 +85,9 @@ Syntax: ::
                                         #           ['games', 'applications']. Defaults to 'all_but_games', which will
                                         #           remove everything except games
 
-    rompatcher:                         # ROMPatcher specific options
-      xdelta_path: [path_to_xdelta]     # OPTIONAL. This is where xdelta is located on your filesystem
+    rompatcher:                                # ROMPatcher specific options
+      xdelta_path: [path_to_xdelta]            # OPTIONAL. This is where xdelta is located on your filesystem
+      rompatcher_js_path: [path_to_rompatcher] # OPTIONAL. This is where RomPatcher.js is located on your filesystem
 
     discord:                            # OPTIONAL. If defined, supply a webhook URL so that ROMSearch can post Discord
       webhook_url: [webhook_url]        #           notifications

diff --git a/docs/configs/platforms.rst b/docs/configs/platforms.rst
@@ -15,11 +15,11 @@ Syntax: ::
     ra_id: [id]                       # OPTIONAL. The RetroAchievements console ID, from their API_GetConsoleIDs
     ra_hash_method: ["md5", "custom"] # OPTIONAL. The RetroAchievements hash method. Supports "md5" and "custom"
 
-    patch_method: ["xdelta"]          # OPTIONAL: Method for patching ROMs. Supports "xdelta", "rompatcher.js"
-    file_exts:                        # OPTIONAL: Potential file extensions. ROMPatcher uses this to figure
-       - [ext]                        #           out the file to patch
-    patch_file_exts:                  # OPTIONAL: Potential file extensions. ROMPatcher uses this to figure
-       - [ext]                        #           out the patch file
+    patch_method: ["xdelta", "rompatcher_js"]  # OPTIONAL: Method for patching ROMs. Supports "xdelta", "rompatcher_js"
+    file_exts:                                 # OPTIONAL: Potential file extensions. ROMPatcher uses this to figure
+       - [ext]                                 #           out the file to patch
+    patch_file_exts:                           # OPTIONAL: Potential file extensions. ROMPatcher uses this to figure
+       - [ext]                                 #           out the patch file
 
 Nintendo - Game Boy
 ===================
@@ -31,17 +31,37 @@ Nintendo - Game Boy Color
 
 .. literalinclude:: ../../romsearch/configs/platforms/Nintendo - Game Boy Color.yml
 
+Nintendo - Game Boy Advance
+===========================
+
+.. literalinclude:: ../../romsearch/configs/platforms/Nintendo - Game Boy Advance.yml
+
 Nintendo - GameCube
 ===================
 
 .. literalinclude:: ../../romsearch/configs/platforms/Nintendo - GameCube.yml
 
+Nintendo - Nintendo Entertainment System
+========================================
+
+.. literalinclude:: ../../romsearch/configs/platforms/Nintendo - Nintendo Entertainment System.yml
+
 Nintendo - Super Nintendo Entertainment System
 ==============================================
 
 .. literalinclude:: ../../romsearch/configs/platforms/Nintendo - Super Nintendo Entertainment System.yml
 
+Sony - PlayStation Portable
+===========================
+
+.. literalinclude:: ../../romsearch/configs/platforms/Sony - PlayStation Portable.yml
+
 Sony - PlayStation
 ==================
 
-.. literalinclude:: ../../romsearch/configs/platforms/Sony - PlayStation.yml
+.. literalinclude:: ../../romsearch/configs/platforms/Sony - PlayStation.yml
+
+Sony - PlayStation 2
+====================
+
+.. literalinclude:: ../../romsearch/configs/platforms/Sony - PlayStation 2.yml
diff --git a/docs/intro.rst b/docs/intro.rst
@@ -38,19 +38,26 @@ everything and then filter from the downloaded files. For more details, see the
 Currently, ROMSearch is in early development, and so many features may be added over time. At the moment, ROMSearch
 has the capability for:
 
-* Nintendo
+* Nintendo (Handheld)
 
   * Game Boy
   * Game Boy Color
+  * Game Boy Advance
+
+* Nintendo (Home)
+
   * GameCube
   * Nintendo - Nintendo Entertainment System
   * Nintendo - Super Nintendo Entertainment System
 
-* Sony
+* Sony (Handheld)
+
+  * PlayStation Portable
+
+* Sony (Home)
 
   * PlayStation
   * PlayStation 2
-  * PlayStation Portable
 
 but be aware there may be quirks that will only become apparent over time. We encourage users to open
 `issues <https://github.com/bbtufty/romsearch/issues>`_ as and where they find them. Known issues can be found at

diff --git a/docs/known_issues.rst b/docs/known_issues.rst
@@ -2,6 +2,11 @@
 Known Issues
 ############
 
+* For very long filenames in Windows, there may be an error in moving files due to the length of the filename.
+
+  * Enabling long paths will fix this. In the Registry Editor, set
+    ``HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem\LongPathsEnabled`` to 1.
+
 * In GameFinder, includes/excludes can cause some unexpected behavior since it will also search through duplicate files.
   For example, having an include of "Crash Bandicoot" for the PS1 will also grab "Crash Bash" and
   "CTR - Crash Team Racing" since at least one of their duplicates starts with "Crash Bandicoot".

diff --git a/docs/modules/rompatcher.rst b/docs/modules/rompatcher.rst
@@ -5,16 +5,29 @@ ROMPatcher
 If a patch file is found and the ROMPatcher module is selected, then this will download patch files and apply to ROMs.
 To highlight if a ROM has been patched, the final file will have a (ROMPatched) in the file name.
 
-Currently, ROMSearch only uses ``xdelta`` to patch ROMs.
+Currently, ROMSearch can use ``xdelta`` and ``RomPatcher.js`` to patch ROMs.
 
 For more details on the ROMPatcher arguments, see the :doc:`config file documentation <../configs/config>`.
 
 xdelta
 ======
 
 To use ``xdelta``, download the latest xdelta3 release from
-`here <https://github.com/jmacd/xdelta-gpl/releases/latest>`_. After unzipping the .exe file, add the path to this file
-into your config, under ``xdelta_path`` in the ``rompatcher`` config section.
+`https://github.com/jmacd/xdelta-gpl/releases/latest <https://github.com/jmacd/xdelta-gpl/releases/latest>`_.
+After unzipping the .exe file, add the path to this file into your config, under ``xdelta_path`` in the ``rompatcher``
+config section.
+
+RomPatcher.js
+=============
+
+You may need to download ``node.js`` to start with. You can get it at `https://nodejs.org/en <https://nodejs.org/en>`_.
+After that, clone the NodePatcher.js repository and install: ::
+
+   git clone https://github.com/marcrobledo/RomPatcher.js.git
+   cd RomPatcher.js
+   npm install
+
+After this, as the RomPatcher.js path in the config, put the path to the ``index.js`` file.
 
 API
 ===

diff --git a/romsearch/configs/clonelists/retool.yml b/romsearch/configs/clonelists/retool.yml
@@ -2,6 +2,7 @@ url: "https://raw.githubusercontent.com/unexpectedpanda/retool-clonelists-metada
 
 Nintendo - Game Boy: "Nintendo - Game Boy (No-Intro).json"
 Nintendo - Game Boy Color: "Nintendo - Game Boy Color (No-Intro).json"
+Nintendo - Game Boy Advance: "Nintendo - Game Boy Advance (No-Intro).json"
 Nintendo - GameCube: "Nintendo - GameCube (Redump).json"
 Nintendo - Nintendo Entertainment System: "Nintendo - Nintendo Entertainment System (No-Intro).json"
 Nintendo - Super Nintendo Entertainment System: "Nintendo - Super Nintendo Entertainment System (No-Intro).json"

diff --git a/romsearch/configs/dats/no-intro.yml b/romsearch/configs/dats/no-intro.yml
@@ -4,6 +4,9 @@ Nintendo - Game Boy:
 Nintendo - Game Boy Color:
   file_mapping: "Nintendo - Game Boy Color (*)"
 
+Nintendo - Game Boy Advance:
+  file_mapping: "Nintendo - Game Boy Advance (*)"
+
 Nintendo - Nintendo Entertainment System:
   file_mapping: "Nintendo - Nintendo Entertainment System (Headered) (*)"
 

diff --git a/romsearch/configs/defaults.yml b/romsearch/configs/defaults.yml
@@ -6,6 +6,7 @@ datetime_format: "%Y/%m/%d, %H:%M:%S"
 platforms:
   - "Nintendo - Game Boy"
   - "Nintendo - Game Boy Color"
+  - "Nintendo - Game Boy Advance"
   - "Nintendo - GameCube"
   - "Nintendo - Nintendo Entertainment System"
   - "Nintendo - Super Nintendo Entertainment System"
@@ -234,3 +235,6 @@ ra_patch_checks:
   - "multi_disc"
   - "demos"
   - "preproduction"
+  - "improved_version"
+  - "modern_version"
+  - "demoted_version"
diff --git a/romsearch/configs/platforms/Nintendo - Game Boy Advance.yml b/romsearch/configs/platforms/Nintendo - Game Boy Advance.yml
@@ -0,0 +1,13 @@
+group: "No-Intro"
+dir: "/No-Intro/Nintendo - Game Boy Advance/"
+unzip: false
+
+ra_id: 5
+ra_hash_method: "md5"
+
+# For the ROM patcher
+patch_method: "rompatcher.js"
+file_exts:
+  - ".gba"
+patch_file_exts:
+  - ".bps"
diff --git a/romsearch/configs/regex.yml b/romsearch/configs/regex.yml
@@ -4,7 +4,7 @@ regions:
   flags: "NOFLAG"
 
 languages:
-  pattern: "\\((([languages])(,\\s?)?)*\\)"
+  pattern: "\\((([languages])((,|\\+)\\s?)?)*\\)"
   type: "list"
   flags: "NOFLAG"
 
@@ -191,6 +191,10 @@ controller_set:
 culture_publishers:
   pattern: "\\(Culture Publishers\\)"
 
+dsi:
+  pattern: "\\(DSI\\)"
+  group: "improved_version"
+
 dx_pack:
   pattern: "\\(DX Pack\\)"
   group: "improved_version"
@@ -319,6 +323,10 @@ usb_mic_doukonban:
   pattern: "\\(USB Mic Doukonban\\)"
   group: "improved_version"
 
+vivendi:
+  pattern: "\\(Vivendi\\)"
+  group: "improved_version"
+
 # BUDGET EDITIONS
 
 artdink:
@@ -502,6 +510,10 @@ konami_collector_series:
   pattern: "\\(Konami Collector's Series\\)"
   group: "modern_version"
 
+mega_man_battle_network_legacy_collection:
+  pattern: "\\(Mega Man Battle Network Legacy Collection\\)"
+  group: "modern_version"
+
 mega_man_legacy_collection:
   pattern: "\\(Mega Man Legacy Collection\\)"
   group: "modern_version"
@@ -530,6 +542,10 @@ ninja_jajamaru_retro:
   pattern: "\\(Ninja JaJaMaru Retro Collection\\)"
   group: "modern_version"
 
+pokemon_box:
+  pattern: "\\(Pokemon Box\\)"
+  group: "modern_version"
+
 qubyte_classic:
   pattern: "\\(QUByte Classics\\)"
   group: "modern_version"

diff --git a/romsearch/configs/sample_config.yml b/romsearch/configs/sample_config.yml
@@ -82,6 +82,7 @@ rahasher:
 
 rompatcher:
   xdelta_path: F:/Emulation/xdelta3-3.1.0-x86_64.exe
+  rompatcher_js_path: F:\Emulation\RomPatcher.js\index.js
 
 discord:
   webhook_url: "https://discord.com/api/webhooks/discord_url"

diff --git a/romsearch/gui/gui_config.py b/romsearch/gui/gui_config.py
@@ -173,9 +173,11 @@ def __init__(
         # Do the same for the directories, but for files instead
         self.all_files = {
             "xdelta_path": self.ui.lineEditConfigRomPatcherxdeltaPath,
+            "rompatcher_js_path": self.ui.lineEditConfigRomPatcherRomPatcherjsPath,
         }
         self.all_files_buttons = {
             "xdelta_path": self.ui.pushButtonConfigRomPatcherxdeltaPath,
+            "rompatcher_js_path": self.ui.pushButtonConfigRomPatcherRomPatcherjsPath,
         }
         for b in self.all_files_buttons:
             self.all_files_buttons[b].clicked.connect(
@@ -194,6 +196,7 @@ def __init__(
 
         self.rompatcher_text_fields = {
             "xdelta_path": self.ui.lineEditConfigRomPatcherxdeltaPath,
+            "rompatcher_js_path": self.ui.lineEditConfigRomPatcherRomPatcherjsPath,
         }
 
         self.discord_text_fields = {

diff --git a/romsearch/gui/layout_about.py b/romsearch/gui/layout_about.py
@@ -3,7 +3,7 @@
 ################################################################################
 ## Form generated from reading UI file 'layout_about.ui'
 ##
-## Created by: Qt User Interface Compiler version 6.7.0
+## Created by: Qt User Interface Compiler version 6.7.3
 ##
 ## WARNING! All changes made in this file will be lost when recompiling UI file!
 ################################################################################

diff --git a/romsearch/gui/layout_romsearch.py b/romsearch/gui/layout_romsearch.py
@@ -3,7 +3,7 @@
 ################################################################################
 ## Form generated from reading UI file 'layout_romsearch.ui'
 ##
-## Created by: Qt User Interface Compiler version 6.7.0
+## Created by: Qt User Interface Compiler version 6.7.3
 ##
 ## WARNING! All changes made in this file will be lost when recompiling UI file!
 ################################################################################
@@ -1277,6 +1277,26 @@ def setupUi(self, RomSearch):
 
         self.verticalLayoutConfigRomPatcher.addLayout(self.horizontalLayoutConfigRomPatcherxdelta)
 
+        self.labelConfigRomPatcherRomPatcherjs = QLabel(self.tabConfigRomPatcher)
+        self.labelConfigRomPatcherRomPatcherjs.setObjectName(u"labelConfigRomPatcherRomPatcherjs")
+
+        self.verticalLayoutConfigRomPatcher.addWidget(self.labelConfigRomPatcherRomPatcherjs)
+
+        self.horizontalLayoutConfigRomPatcherRomPatcherjs = QHBoxLayout()
+        self.horizontalLayoutConfigRomPatcherRomPatcherjs.setObjectName(u"horizontalLayoutConfigRomPatcherRomPatcherjs")
+        self.lineEditConfigRomPatcherRomPatcherjsPath = QLineEdit(self.tabConfigRomPatcher)
+        self.lineEditConfigRomPatcherRomPatcherjsPath.setObjectName(u"lineEditConfigRomPatcherRomPatcherjsPath")
+
+        self.horizontalLayoutConfigRomPatcherRomPatcherjs.addWidget(self.lineEditConfigRomPatcherRomPatcherjsPath)
+
+        self.pushButtonConfigRomPatcherRomPatcherjsPath = QPushButton(self.tabConfigRomPatcher)
+        self.pushButtonConfigRomPatcherRomPatcherjsPath.setObjectName(u"pushButtonConfigRomPatcherRomPatcherjsPath")
+
+        self.horizontalLayoutConfigRomPatcherRomPatcherjs.addWidget(self.pushButtonConfigRomPatcherRomPatcherjsPath)
+
+
+        self.verticalLayoutConfigRomPatcher.addLayout(self.horizontalLayoutConfigRomPatcherRomPatcherjs)
+
         self.lineConfigRomPatcherBottom = QFrame(self.tabConfigRomPatcher)
         self.lineConfigRomPatcherBottom.setObjectName(u"lineConfigRomPatcherBottom")
         self.lineConfigRomPatcherBottom.setFrameShape(QFrame.Shape.HLine)
@@ -1808,6 +1828,9 @@ def retranslateUi(self, RomSearch):
         self.labelConfigRomPatcherxdelta.setText(QCoreApplication.translate("RomSearch", u"xdelta path", None))
         self.lineEditConfigRomPatcherxdeltaPath.setPlaceholderText(QCoreApplication.translate("RomSearch", u"xdelta.exe", None))
         self.pushButtonConfigRomPatcherxdeltaPath.setText(QCoreApplication.translate("RomSearch", u"Browse", None))
+        self.labelConfigRomPatcherRomPatcherjs.setText(QCoreApplication.translate("RomSearch", u"RomPatcher.js path", None))
+        self.lineEditConfigRomPatcherRomPatcherjsPath.setPlaceholderText(QCoreApplication.translate("RomSearch", u"index.js", None))
+        self.pushButtonConfigRomPatcherRomPatcherjsPath.setText(QCoreApplication.translate("RomSearch", u"Browse", None))
         self.tabWidgetConfig.setTabText(self.tabWidgetConfig.indexOf(self.tabConfigRomPatcher), QCoreApplication.translate("RomSearch", u"ROMPatcher", None))
         self.labelConfigDiscordWebhookUrlTitle.setText(QCoreApplication.translate("RomSearch", u"Webhook URL", None))
         self.labelConfigDiscordWebhookUrlDescription.setText(QCoreApplication.translate("RomSearch", u"URL for Discord webhooks. Must be set for notifications to be sent", None))