Add user guide and docs for logging (NVIDIA#3134)

### Types of changes
<!--- Put an `x` in all the boxes that apply, and remove the not
applicable items -->
- [x] Non-breaking change (fix or new feature that would not break
existing functionality).
- [ ] Breaking change (fix or new feature that would cause existing
functionality to change).
- [ ] New tests added to cover the changes.
- [ ] Quick tests passed locally by running `./runtest.sh`.
- [ ] In-line docstrings updated.
- [ ] Documentation updated.

docs/examples/hello_cross_val.rst

@@ -52,7 +52,7 @@ Custom FL applications can contain the folders:
 
  #. **custom**: contains the custom components (``np_trainer.py``, ``np_model_persistor.py``, ``np_validator.py``, ``np_model_locator``, ``np_formatter``)
  #. **config**: contains client and server configurations (``config_fed_client.json``, ``config_fed_server.json``)
- #. **resources**: contains the logger config (``log.config``)
+ #. **resources**: contains the logger config (``log_config.json``)
 
 Let's get started. First clone the repo, if you haven't already:
 

docs/examples/hello_scatter_and_gather.rst

@@ -40,7 +40,7 @@ Custom FL applications can contain the folders:
 
  #. **custom**: contains any custom components (custom Python code)
  #. **config**: contains client and server configurations (``config_fed_client.json``, ``config_fed_server.json``)
- #. **resources**: contains the logger config (``log.config``)
+ #. **resources**: contains the logger config (``log_config.json``)
 
 Let's get started. First clone the repo, if you haven't already:
 

docs/quickstart.rst

@@ -247,12 +247,12 @@ Basic usage for the :ref:`FL Simulator <fl_simulator>` is available with ``nvfla
 .. code-block:: shell
 
   $ nvflare simulator -h
-  usage: nvflare simulator [-h] [-w WORKSPACE] [-n N_CLIENTS] [-c CLIENTS] [-t THREADS] [-gpu GPU] [-m MAX_CLIENTS] job_folder
-  
+  usage: nvflare simulator [-h] [-w WORKSPACE] [-n N_CLIENTS] [-c CLIENTS] [-t THREADS] [-gpu GPU] [-m MAX_CLIENTS] [--end_run_for_all] job_folder
+
   positional arguments:
     job_folder
-  
-  optional arguments:
+
+  options:
     -h, --help            show this help message and exit
     -w WORKSPACE, --workspace WORKSPACE
                           WORKSPACE folder
@@ -265,6 +265,7 @@ Basic usage for the :ref:`FL Simulator <fl_simulator>` is available with ``nvfla
     -gpu GPU, --gpu GPU   list of GPU Device Ids, comma separated
     -m MAX_CLIENTS, --max_clients MAX_CLIENTS
                           max number of clients
+    --end_run_for_all     flag to indicate if running END_RUN event for all clients
 
 
 Before we get into the Simulator, we'll walk through a few additional setup steps in the next section required
@@ -328,7 +329,7 @@ and codes, logs of the output, site and global models, cross-site validation res
   /tmp/nvflare/jobs/workdir
   ├── server
   │   ├── local
-  │   │   └── log.config
+  │   │   └── log_config.json
   │   ├── log.txt
   │   ├── pool_stats
   │   │   └── simulator_cell_stats.json
@@ -347,7 +348,7 @@ and codes, logs of the output, site and global models, cross-site validation res
   ├── site-1
   │   ├── cifar_net.pth
   │   ├── local
-  │   │   └── log.config
+  │   │   └── log_config.json
   │   ├── log.txt
   │   ├── simulate_job
   │   │   ├── app_site-1
@@ -358,7 +359,7 @@ and codes, logs of the output, site and global models, cross-site validation res
   ├── site-2
   │   ├── cifar_net.pth
   │   ├── local
-  │   │   └── log.config
+  │   │   └── log_config.json
   │   ├── log.txt
   │   ├── simulate_job
   │   │   ├── app_site-2

docs/real_world_fl/application.rst

@@ -18,7 +18,7 @@ The structure of the app folder needs to be::
             [another file with custom code].py
             ...
         resources/
-            log.config
+            log_config.json
 
 .. note::
 
@@ -121,9 +121,9 @@ to certain roles or even orgs or users. See :ref:`federated_authorization` for d
 Resources
 *********
 
-A ``log.config`` is needed inside the resources folder.
+A ``log_config.json`` is needed inside the resources folder.
 This file is for the Python logger to use.
-If you don't want to customize the log behavior, you can use the same ``log.config`` from one of
+If you don't want to customize the log behavior, you can use the same ``log_config.json`` from one of
 the example application folder.
 
-.. literalinclude:: ../resources/log.config
+.. literalinclude:: ../resources/log_config.json

docs/real_world_fl/operation.rst

@@ -19,7 +19,7 @@ commands shown as examples of how they may be run with a description.
 
 .. csv-table::
     :header: Command,Example,Description
-    :widths: 15, 10, 30
+    :widths: 15, 20, 30
 
     bye,``bye``,Exit from the client
     help,``help``,Get command help information
@@ -30,6 +30,8 @@ commands shown as examples of how they may be run with a description.
     ,``check_status client clientname``,"The name, token, and status of the specified client with *clientname* are displayed."
     submit_job,``submit_job job_folder_name``,Submits the job to the server.
     list_jobs,``list_jobs``,Lists the jobs on the server. (Options: [-n name_prefix] [-d] [job_id_prefix])
+    configure_job_log,``configure_job_log job_id server config``,"Configure the job log on the server. (*config* can be a path to a json config file, a levelname/levelnumber, or 'reload')"
+    ,``configure_job_log job_id client <client-name>... config``,Configure the job log on the target client(s).
     abort_job,``abort_job job_id``,Aborts the job of the specified job_id if it is running or dispatched
     clone_job,``clone_job job_id``,Creates a copy of the specified job with a new job_id
     abort,``abort job_id client``,Aborts the job for the specified job_id for all clients. Individual client jobs can be aborted by specifying *clientname*.
@@ -47,6 +49,8 @@ commands shown as examples of how they may be run with a description.
     ,``ls clientname -SR``,List files in workspace root directory (-S: sort by file size; -R: list subdirectories recursively)
     pwd,``pwd server``,Print the name of workspace root directory
     ,``pwd clientname``,Print the name of workspace root directory
+    configure_site_log,``configure_job_log server config``,"Configure the site log on the server. (*config* can be a path to a json config file, a levelname/levelnumber, or 'reload')"
+    ,``configure_site_log client <client-name>... config``,Configure the site log on the target client(s).
     sys_info,``sys_info server``,Get system information
     ,``sys_info client *clientname*``,Get system information. Individual clients can be shutdown by specifying *clientname*.
     remove_client,``remove_client clientname``,Issue command for server to release client before the 10 minute timeout to allow client to rejoin after manual restart.

docs/real_world_fl/workspace.rst

@@ -21,7 +21,7 @@ Server
         startup/
             authorization.json
             fed_server.json
-            log.config
+            log_config.json
             readme.txt
             rootCA.pem
             server_context.tenseal
@@ -89,7 +89,7 @@ Client
             client.crt
             client.key
             fed_client.json
-            log.config
+            log_config.json
             readme.txt
             rootCA.pem
             signature.pkl