UW-515 File Copy/Link Tool (ufs-community#429)

docs/sections/user_guide/api/file.rst

@@ -0,0 +1,5 @@
+``uwtools.api.file``
+====================
+
+.. automodule:: uwtools.api.file
+    :members:

docs/sections/user_guide/api/index.rst

@@ -3,6 +3,7 @@ API
 
 .. toctree::
    config
+   file
    fv3
    logging
    rocoto

docs/sections/user_guide/cli/drivers/mode_fv3.rst → docs/sections/user_guide/cli/drivers/fv3.rst

@@ -1,5 +1,5 @@
-Mode ``fv3``
-============
+``fv3``
+=======
 
 The ``uw`` mode for configuring and running FV3.
 

docs/sections/user_guide/cli/drivers/index.rst

@@ -4,5 +4,5 @@ Drivers
 .. toctree::
    :maxdepth: 1
 
-   mode_fv3
-   mode_sfc_climo_gen
+   fv3
+   sfc_climo_gen

docs/sections/user_guide/cli/drivers/mode_sfc_climo_gen.rst → docs/sections/user_guide/cli/drivers/sfc_climo_gen.rst

@@ -1,5 +1,5 @@
-Mode ``sfc_climo_gen``
-======================
+``sfc_climo_gen``
+=================
 
 The ``uw`` mode for configuring and running the :sfc-climo-gen:`sfc_climo_gen<>` component.
 

docs/sections/user_guide/cli/tools/mode_config.rst → docs/sections/user_guide/cli/tools/config.rst

@@ -1,5 +1,5 @@
-Mode ``config``
-===============
+``config``
+==========
 
 The ``uw`` mode for handling configuration files (configs).
 
@@ -28,7 +28,7 @@ The ``uw`` mode for handling configuration files (configs).
 ``compare``
 -----------
 
-The ``compare`` mode lets users compare two config files.
+The ``compare`` action lets users compare two config files.
 
 .. code-block:: text
 
@@ -148,7 +148,7 @@ The examples that follow use namelist files ``values1.nml`` and ``values2.nml``,
 ``realize``
 -----------
 
-In ``uw`` terminology, to realize a configuration file is to transform it from its raw form into its final, usable state. The ``realize`` mode can build a complete config file from two or more separate files.
+In ``uw`` terminology, to realize a configuration file is to transform it from its raw form into its final, usable state. The ``realize`` action can build a complete config file from two or more separate files.
 
 .. code-block:: text
 
@@ -483,7 +483,7 @@ and an additional supplemental YAML file ``values2.yaml`` with the following con
 ``validate``
 ------------
 
-The ``validate`` mode ensures that a given config file is structured properly.
+The ``validate`` action ensures that a given config file is structured properly.
 
 .. code-block:: text
 

docs/sections/user_guide/cli/tools/file.rst

@@ -0,0 +1,174 @@
+``file``
+========
+
+The ``uw`` mode for handling filesystem files.
+
+.. code-block:: text
+
+   $ uw file --help
+   usage: uw file [-h] ACTION ...
+
+   Handle files
+
+   Optional arguments:
+     -h, --help
+         Show help and exit
+
+   Positional arguments:
+     ACTION
+       copy
+         Copy files
+       link
+         Link files
+
+.. _cli_file_copy_examples:
+
+``copy``
+--------
+
+The ``copy`` action stages files in a target directory by copying files. Any ``KEY`` positional arguments are used to navigate, in the order given, from the top of the config to the :ref:`file block <files_yaml>`.
+
+.. code-block:: text
+
+   % uw file copy --help
+   usage: uw file copy --target-dir PATH [-h] [--config-file PATH] [--dry-run] [--quiet] [--verbose]
+                       [KEY ...]
+
+   Copy files
+
+   Required arguments:
+     --target-dir PATH
+         Path to target directory
+
+   Optional arguments:
+     -h, --help
+         Show help and exit
+     --config-file PATH, -c PATH
+         Path to config file
+     --dry-run
+         Only log info, making no changes
+     --quiet, -q
+         Print no logging messages
+     --verbose, -v
+         Print all logging messages
+     KEY
+         YAML key leading to file dst/src block
+
+Examples
+^^^^^^^^
+
+Given ``config.yaml`` containing
+
+.. code-block:: yaml
+
+   config:
+     files:
+       foo: /path/to/foo
+       subdir/bar: /path/to/bar
+
+.. code-block:: text
+
+   $ uw file copy --target-dir /tmp/target --config-file config.yaml config files
+   [2024-03-14T19:00:02]     INFO Validating config against internal schema files-to-stage
+   [2024-03-14T19:00:02]     INFO 0 UW schema-validation errors found
+   [2024-03-14T19:00:02]     INFO File copies: Initial state: Pending
+   [2024-03-14T19:00:02]     INFO File copies: Checking requirements
+   [2024-03-14T19:00:02]     INFO Copy /tmp/source/foo -> /tmp/target/foo: Initial state: Pending
+   [2024-03-14T19:00:02]     INFO Copy /tmp/source/foo -> /tmp/target/foo: Checking requirements
+   [2024-03-14T19:00:02]     INFO Copy /tmp/source/foo -> /tmp/target/foo: Requirement(s) ready
+   [2024-03-14T19:00:02]     INFO Copy /tmp/source/foo -> /tmp/target/foo: Executing
+   [2024-03-14T19:00:02]     INFO Copy /tmp/source/foo -> /tmp/target/foo: Final state: Ready
+   [2024-03-14T19:00:02]     INFO Copy /tmp/source/bar -> /tmp/target/subdir/bar: Initial state: Pending
+   [2024-03-14T19:00:02]     INFO Copy /tmp/source/bar -> /tmp/target/subdir/bar: Checking requirements
+   [2024-03-14T19:00:02]     INFO Copy /tmp/source/bar -> /tmp/target/subdir/bar: Requirement(s) ready
+   [2024-03-14T19:00:02]     INFO Copy /tmp/source/bar -> /tmp/target/subdir/bar: Executing
+   [2024-03-14T19:00:02]     INFO Copy /tmp/source/bar -> /tmp/target/subdir/bar: Final state: Ready
+   [2024-03-14T19:00:02]     INFO File copies: Final state: Ready
+
+After executing this command:
+
+.. code-block:: text
+
+   $ tree /tmp/target
+   /tmp/target
+   ├── foo
+   └── subdir
+       └── bar
+
+Here, ``foo`` and ``bar`` are copies of their respective source files.
+
+.. _cli_file_link_examples:
+
+``link``
+--------
+
+The ``link`` action stages files in a target directory by linking files. Any ``KEY`` positional arguments are used to navigate, in the order given, from the top of the config to the :ref:`file block <files_yaml>`.
+
+.. code-block:: text
+
+   % uw file link --help
+   usage: uw file link --target-dir PATH [-h] [--config-file PATH] [--dry-run] [--quiet] [--verbose]
+                       [KEY ...]
+
+   Link files
+
+   Required arguments:
+     --target-dir PATH
+         Path to target directory
+
+   Optional arguments:
+     -h, --help
+         Show help and exit
+     --config-file PATH, -c PATH
+         Path to config file
+     --dry-run
+         Only log info, making no changes
+     --quiet, -q
+         Print no logging messages
+     --verbose, -v
+         Print all logging messages
+     KEY
+         YAML key leading to file dst/src block
+
+Examples
+^^^^^^^^
+
+Given ``config.yaml`` containing
+
+.. code-block:: yaml
+
+   config:
+     files:
+       foo: /path/to/foo
+       subdir/bar: /path/to/bar
+
+.. code-block:: text
+
+   $ uw file link --target-dir /tmp/target --config-file config.yaml config files
+   [2024-03-14T19:02:49]     INFO Validating config against internal schema files-to-stage
+   [2024-03-14T19:02:49]     INFO 0 UW schema-validation errors found
+   [2024-03-14T19:02:49]     INFO File links: Initial state: Pending
+   [2024-03-14T19:02:49]     INFO File links: Checking requirements
+   [2024-03-14T19:02:49]     INFO Link /tmp/target/foo -> /tmp/source/foo: Initial state: Pending
+   [2024-03-14T19:02:49]     INFO Link /tmp/target/foo -> /tmp/source/foo: Checking requirements
+   [2024-03-14T19:02:49]     INFO Link /tmp/target/foo -> /tmp/source/foo: Requirement(s) ready
+   [2024-03-14T19:02:49]     INFO Link /tmp/target/foo -> /tmp/source/foo: Executing
+   [2024-03-14T19:02:49]     INFO Link /tmp/target/foo -> /tmp/source/foo: Final state: Ready
+   [2024-03-14T19:02:49]     INFO Link /tmp/target/subdir/bar -> /tmp/source/bar: Initial state: Pending
+   [2024-03-14T19:02:49]     INFO Link /tmp/target/subdir/bar -> /tmp/source/bar: Checking requirements
+   [2024-03-14T19:02:49]     INFO Link /tmp/target/subdir/bar -> /tmp/source/bar: Requirement(s) ready
+   [2024-03-14T19:02:49]     INFO Link /tmp/target/subdir/bar -> /tmp/source/bar: Executing
+   [2024-03-14T19:02:49]     INFO Link /tmp/target/subdir/bar -> /tmp/source/bar: Final state: Ready
+   [2024-03-14T19:02:49]     INFO File links: Final state: Ready
+
+After executing this command:
+
+.. code-block:: text
+
+   $ tree /tmp/target
+   /tmp/target
+   ├── foo -> /tmp/source/foo
+   └── subdir
+       └── bar -> /tmp/source/bar
+
+Here, ``foo`` and ``bar`` are symbolic links.

docs/sections/user_guide/cli/tools/index.rst

@@ -4,6 +4,7 @@ Tools
 .. toctree::
    :maxdepth: 1
 
-   mode_config
-   mode_rocoto
-   mode_template
+   config
+   file
+   rocoto
+   template

docs/sections/user_guide/cli/tools/mode_rocoto.rst → docs/sections/user_guide/cli/tools/rocoto.rst

@@ -1,5 +1,5 @@
-Mode ``rocoto``
-===============
+``rocoto``
+==========
 
 The ``uw`` mode for realizing and validating Rocoto XML documents.
 

docs/sections/user_guide/cli/tools/mode_template.rst → docs/sections/user_guide/cli/tools/template.rst

@@ -1,5 +1,5 @@
-Mode ``template``
-=================
+``template``
+============
 
 The ``uw`` mode for handling :jinja2:`Jinja2 templates<templates>`.
 

docs/sections/user_guide/yaml/components/fv3.rst

@@ -84,7 +84,7 @@ Supports ``base_file:`` and ``update_values:`` blocks (see the :ref:`updating_va
 files_to_copy:
 ^^^^^^^^^^^^^^
 
-Defines files to be copied to the run directory. Keys in the ``files_to_copy:`` YAML map specify destination paths relative to the run directory, and values specify source paths. Both keys and values may contain Jinja2 variables/expressions using a ``cycle`` variable, which is a Python ``datetime`` object corresponding to the FV3 cycle being run. This supports specification of cycle-specific filenames/paths. For example, a key-value pair
+See :ref:`this page <files_yaml>` for details. For the ``fv3`` driver, both keys and values may contain Jinja2 variables/expressions using a ``cycle`` variable, which is a Python ``datetime`` object corresponding to the FV3 cycle being run. This supports specification of cycle-specific filenames/paths. For example, a key-value pair
 
 .. code-block:: yaml
 

docs/sections/user_guide/yaml/execution.rst

@@ -1,7 +1,7 @@
 .. _execution_yaml:
 
-The ``execution:`` Block:
-=========================
+The ``execution:`` Block
+========================
 
 Component drivers require an ``execution:`` block to provide details about how to configure a component's run-time environment, and how to run it.
 

docs/sections/user_guide/yaml/files.rst

@@ -0,0 +1,35 @@
+.. _files_yaml:
+
+File Blocks
+===========
+
+File blocks define files to be staged in a target directory as copies or symbolic links. Keys in such blocks specify destination paths relative to the target directory, and values specify source paths.
+
+Example block:
+
+.. code-block:: yaml
+
+   foo: /path/to/foo
+   subdir/bar: /path/to/bar
+
+* Result when copying:
+
+.. code-block:: text
+
+   target/
+   ├── foo
+   └── subdir
+       └── bar
+
+where ``foo`` and ``bar`` are copies of their respective source files.
+
+* Result when linking:
+
+.. code-block:: text
+
+   target
+   ├── foo -> /path/to/foo
+   └── subdir
+       └── bar -> /path/to/bar
+
+where ``foo`` and ``bar`` are symbolic links.

docs/sections/user_guide/yaml/index.rst

@@ -7,6 +7,7 @@ UW YAML
    components/index
    platform
    execution
+   files
    updating_values
    field_table
    rocoto

docs/sections/user_guide/yaml/platform.rst

@@ -1,7 +1,7 @@
 .. _platform_yaml:
 
-The ``platform:`` Block:
-========================
+The ``platform:`` Block
+=======================
 
 The top-level UW YAML ``platform:`` block, referenced by multiple component drivers, is described below. It is validated by JSON Schema.
 

format

@@ -7,7 +7,7 @@ echo "=> Running isort"
 isort -q src
 
 echo "=> Running docformatter"
-(cd src && docformatter . || test $$? -eq 3)
+(cd src && docformatter . || test $? -eq 3)
 
 echo "=> Running jq"
 for a in $(find src -type f -name "*.jsonschema"); do