Skip to content

Commit

Permalink
Merge branch 'main' into subprocess_launcher_not_quiting
Browse files Browse the repository at this point in the history
  • Loading branch information
nvidianz authored Jan 7, 2025
2 parents 59bc567 + 939735a commit 4e6d563
Show file tree
Hide file tree
Showing 61 changed files with 1,496 additions and 480 deletions.
23 changes: 23 additions & 0 deletions 3rdParty/tdigest.LICENSE.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
https://github.com/CamDavidsonPilon/tdigest/blob/master/LICENSE.txt

The MIT License (MIT)

Copyright (c) 2015 Cameron Davidson-Pilon

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
4 changes: 4 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,10 @@ To collaborate efficiently, please read through this section and follow them.
* [Building documentation](#building-the-documentation)
* [Signing your work](#signing-your-work)

> Note:
> some package dependencies requires python<version>-dev in local development such as
> python3.12-dev.
#### Checking the coding style
We check code style using flake8 and isort.
A bash script (`runtest.sh`) is provided to run all tests locally.
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

# NVIDIA FLARE

[Website](https://nvidia.github.io/NVFlare) | [Paper](https://arxiv.org/abs/2210.13291) | [Talks & Blogs](https://nvflare.readthedocs.io/en/main/publications_and_talks.html) | [Research](./research/README.md) | [Documentation](https://nvflare.readthedocs.io/en/main)
[Website](https://nvidia.github.io/NVFlare) | [Paper](https://arxiv.org/abs/2210.13291) | [Blogs](https://developer.nvidia.com/blog/tag/federated-learning) | [Talks & Papers](https://nvflare.readthedocs.io/en/main/publications_and_talks.html) | [Research](./research/README.md) | [Documentation](https://nvflare.readthedocs.io/en/main)

[![Blossom-CI](https://github.com/NVIDIA/nvflare/workflows/Blossom-CI/badge.svg?branch=main)](https://github.com/NVIDIA/nvflare/actions)
[![documentation](https://readthedocs.org/projects/nvflare/badge/?version=main)](https://nvflare.readthedocs.io/en/main/?badge=main)
Expand Down
7 changes: 6 additions & 1 deletion docs/getting_started.rst
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,9 @@
Getting Started
###############

See :ref:`installation`.
### To get started, take a look at the instructions and code examples on our [**website**](https://nvidia.github.io/NVFlare/).

* For more details, see our getting started `tutorials <https://github.com/NVIDIA/NVFlare/tree/main/examples/getting_started>`__.
* For more advanced examples and `step-by-step <https://github.com/NVIDIA/NVFlare/tree/main/examples/hello-world/step-by-step>`__ walk-troughs, see our `examples <https://github.com/NVIDIA/NVFlare/tree/main/examples>`__.
* There are also detailed instructions on how to convert your standalone/centralized training code to `federated learning code. <https://github.com/NVIDIA/NVFlare/tree/main/examples/hello-world/ml-to-fl>`__.
* If you'd like to write your own NVIDIA FLARE components, a detailed programming guide can be found `here <https://nvflare.readthedocs.io/en/main/programming_guide.html>`__.
19 changes: 15 additions & 4 deletions docs/publications_and_talks.rst
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,15 @@ including papers using NVIDIA FLARE's predecessor libraries included in the `Cla

Publications: 2024
------------------
* **2024-02** `Empowering Federated Learning for Massive Models with NVIDIA FLARE <https://arxiv.org/abs/2402.07792>`__ (Accepted to `FL@FM-TheWebConf'24 <https://federated-learning.org/fl@fm-www-2024>`__))
* **2024-12** `C-FedRAG: A Confidential Federated Retrieval-Augmented Generation System <https://arxiv.org/abs/2412.13163>`__ (preprint)
* **2024-11** `Toward a tipping point in federated learning in healthcare and life sciences <https://www.sciencedirect.com/science/article/pii/S2666389924002368>`__ (`Patterns, Volume 5, Issue 11, 2024, <https://www.sciencedirect.com/journal/patterns>`__)
* **2024-07** `FedBPT: Efficient Federated Black-box Prompt Tuning for Large Language Models <https://arxiv.org/abs/2310.01467>`__ (`ICML 2024 <https://icml.cc/virtual/2024/poster/34753>`__)
* **2024-07** `Fair evaluation of federated learning algorithms for automated breast density classification: The results of the 2022 ACR-NCI-NVIDIA federated learning challenge <https://arxiv.org/abs/2405.14900>`__ (`Medical Image Analysis, Volume 95, July 2024 <https://www.sciencedirect.com/science/article/abs/pii/S1361841524001312>`__)
* **2024-07** `Easy and Scalable Federated Learning in the Age of Large Language Models with NVIDIA FLARE <https://federated-learning.org/fl@fm-icme-2024/HRoth.pdf>`__ (`FL@FM-ICME'24 <https://federated-learning.org/fl@fm-icme-2024/#k3>`__)
* **2024-05** `Federated Learning Privacy: Attacks, Defenses, Applications, and Policy Landscape - A Survey <https://arxiv.org/abs/2405.03636>`__ (preprint)
* **2024-05** `Supercharging Federated Learning with Flower and NVIDIA FLARE <https://arxiv.org/abs/2407.00031>`__ (Presented at `FL@FM-IJCAI <https://federated-learning.org/fl@fm-ijcai-2024>`__ In preparation for Lecture Notes in AI)
* **2024-05** `An in-depth evaluation of federated learning on biomedical natural language processing for information extraction <https://www.nature.com/articles/s41746-024-01126-4>`__ (`Nature Digital Medicine 7, 127, 2024 <https://www.nature.com/npjdigitalmed>`__)
* **2024-02** `Empowering Federated Learning for Massive Models with NVIDIA FLARE <https://arxiv.org/abs/2402.07792>`__ (Presented at `FL@FM-TheWebConf'24 <https://federated-learning.org/fl@fm-www-2024>`__, `Springer Book Chapter <https://link.springer.com/book/9783031788406#about-this-book>`__))

Publications: 2023
------------------
Expand All @@ -21,7 +29,6 @@ Publications: 2023

Publications: 2022
------------------
* **2022-11** `Federated Learning with Azure Machine Learning <https://www.youtube.com/watch?v=aTj4AqbCWEA>`__ (Video)
* **2022-10** `Auto-FedRL: Federated Hyperparameter Optimization for Multi-institutional Medical Image Segmentation <https://arxiv.org/abs/2203.06338>`__ (`ECCV 2022 <https://eccv2022.ecva.net/>`__)
* **2022-10** `Joint Multi Organ and Tumor Segmentation from Partial Labels Using Federated Learning <https://link.springer.com/chapter/10.1007/978-3-031-18523-6_6>`__ (`DeCaF @ MICCAI 2022 <https://decaf-workshop.github.io/decaf-2022/>`__)
* **2022-10** `Split-U-Net: Preventing Data Leakage in Split Learning for Collaborative Multi-modal Brain Tumor Segmentation <https://arxiv.org/abs/2208.10553>`__ (`DeCaF @ MICCAI 2022 <https://decaf-workshop.github.io/decaf-2022/>`__)
Expand Down Expand Up @@ -52,6 +59,8 @@ NVIDIA FLARE related blogs and other media.

Blogs & Videos: 2024
--------------------
* **2024-04** `Differential Privacy and Federated Learning for Medical Data <https://towardsdatascience.com/differential-privacy-and-federated-learning-for-medical-data-0f2437d6ece9>`__ (Roche Technical Blog)
* **2024-03** `Announcing NVIDIA and Flower Collaboration <https://flower.ai/blog/2024-03-15-announcing-nvidia-and-flower-collaboration/>`__ (Flower Blog)
* **2024-03** `Turning Machine Learning to Federated Learning in Minutes with NVIDIA FLARE 2.4 <https://developer.nvidia.com/blog/turning-machine-learning-to-federated-learning-in-minutes-with-nvidia-flare-2-4/>`__ (NVIDIA Technical Blog)
* **2024-02** `Scalable Federated Learning with NVIDIA FLARE for Enhanced LLM Performance <https://developer.nvidia.com/blog/scalable-federated-learning-with-nvidia-flare-for-enhanced-llm-performance/>`__ (NVIDIA Technical Blog)

Expand All @@ -66,7 +75,6 @@ Blogs & Videos: 2023

Blogs & Videos: 2022
--------------------

* **2022-10** `Federated Learning from Simulation to Production with NVIDIA FLARE <https://developer.nvidia.com/blog/federated-learning-from-simulation-to-production-with-nvidia-flare/?ncid=so-nvsh-705336#cid=ix11_so-nvsh_en-us>`__ (NVIDIA Technical Blog)
* **2022-08** `Using Federated Learning to Bridge Data Silos in Financial Services <https://developer.nvidia.com/blog/using-federated-learning-to-bridge-data-silos-in-financial-services/>`__ (NVIDIA Technical Blog)
* **2022-06** `Experimenting with Novel Distributed Applications Using NVIDIA Flare 2.1 <https://developer.nvidia.com/blog/experimenting-with-novel-distributed-applications-using-nvidia-flare-2-1/>`__ (NVIDIA Technical Blog)
Expand Down Expand Up @@ -94,7 +102,9 @@ Recent talks and Webinars covering federated learning research and NVIDIA FLARE.

Talks: 2024
-----------
* **2024-03** `Empowering Federated Learning for Massive Models with NVIDIA FLARE <https://youtu.be/-P2ML9Kw0FA>`__ (`SFBigAnalytics Meetup <https://www.meetup.com/sf-big-analytics/?_cookie-check=Efm7MGh7mO4YiV8A>`__)
* **2024-12** `Real-world Federated Learning with NVIDIA FLARE <https://ucsf.zoom.us/rec/share/ncALLA6B583mh7_Lrj4aPgA-qsonqVq5-Ufi5XZCMm6vH-uK5nVYXyVlsexE5KbC.LAYkmvF-sm-yYjTB?startTime=1733957588000>`__ [Passcode: !Ms8Tw.u8H] (`UCSF Biostatistics and Bioinformatics Seminar <https://epibiostat.ucsf.edu/events/real-world-federated-learning-nvidia-flare>`__)
* **2024-04** `Federated Learning: Towards Real-world Studies <hhttps://youtu.be/3ELbZYwy3bI?t=2474>`__ (`SFBigAnalytics Meetup <https://www.meetup.com/sf-big-analytics/events/300302421>`__)
* **2024-03** `Empowering Federated Learning for Massive Models with NVIDIA FLARE <https://youtu.be/-P2ML9Kw0FA>`__ (`SFBigAnalytics Meetup <https://www.meetup.com/sf-big-analytics/events/299337374>`__)

Talks: 2023
-----------
Expand All @@ -103,6 +113,7 @@ Talks: 2023

Talks: 2022
-----------
* **2022-11** `Federated Learning with Azure Machine Learning <https://www.youtube.com/watch?v=aTj4AqbCWEA>`__ (Microsoft Developer Video)
* **2022-10** `Modern Tools for Collaborative Medical Image Analysis <https://drive.google.com/file/d/1hmlyG7g1SU8vhQ5wdTFhkqFO9Ty8BiYG/view?usp=sharing>`__ (`Keynote - DART @ MICCAI 2022 <https://sites.google.com/view/dart2022/home?authuser=0>`__)
* **2022-07** `NVIDIA FLARE Tutorial for Beginners <https://www.youtube.com/watch?v=8x7oY3xAgek&t=11s&ab_channel=NVIDIADeveloper>`__ (United Imaging Meetup)
* **2022-07** `Techniques and Tools for Collaborative Development of AI Models across Institutes <https://www.bilibili.com/video/BV1y14y147nc/?spm_id_from=333.337.search-card.all.click>`__ (`VALSE Webinar <http://valser.org/article-572-1.html>`__)
Expand Down
105 changes: 77 additions & 28 deletions docs/resources/log.config
Original file line number Diff line number Diff line change
@@ -1,28 +1,77 @@
[loggers]
keys=root

[handlers]
keys=consoleHandler,errorFileHandler

[formatters]
keys=baseFormatter

[logger_root]
level=INFO
handlers=consoleHandler,errorFileHandler

[handler_consoleHandler]
class=StreamHandler
level=DEBUG
formatter=baseFormatter
args=(sys.stdout,)

[handler_errorFileHandler]
class=FileHandler
level=ERROR
formatter=baseFormatter
args=('error_log.txt', 'a')

[formatter_baseFormatter]
class=nvflare.fuel.utils.log_utils.BaseFormatter
format=%(asctime)s - %(name)s - %(levelname)s - %(message)s
{
"version": 1,
"disable_existing_loggers": false,
"formatters": {
"baseFormatter": {
"()": "nvflare.fuel.utils.log_utils.BaseFormatter",
"fmt": "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
},
"colorFormatter": {
"()": "nvflare.fuel.utils.log_utils.ColorFormatter",
"fmt": "%(asctime)s - %(name)s - %(levelname)s - %(message)s"
},
"jsonFormatter": {
"()": "nvflare.fuel.utils.log_utils.JsonFormatter",
"fmt": "%(asctime)s - %(name)s - %(fullName)s - %(levelname)s - %(message)s"
}
},
"filters": {
"FLFilter": {
"()": "nvflare.fuel.utils.log_utils.LoggerNameFilter",
"logger_names": ["custom", "nvflare.app_common", "nvflare.app_opt"]
}
},
"handlers": {
"consoleHandler": {
"class": "logging.StreamHandler",
"level": "DEBUG",
"formatter": "colorFormatter",
"filters": [],
"stream": "ext://sys.stdout"
},
"logFileHandler": {
"class": "logging.handlers.RotatingFileHandler",
"level": "DEBUG",
"formatter": "baseFormatter",
"filename": "log.txt",
"mode": "a",
"maxBytes": 20971520,
"backupCount": 10
},
"errorFileHandler": {
"class": "logging.handlers.RotatingFileHandler",
"level": "ERROR",
"formatter": "baseFormatter",
"filename": "log_error.txt",
"mode": "a",
"maxBytes": 20971520,
"backupCount": 10
},
"jsonFileHandler": {
"class": "logging.handlers.RotatingFileHandler",
"level": "DEBUG",
"formatter": "jsonFormatter",
"filename": "log.json",
"mode": "a",
"maxBytes": 20971520,
"backupCount": 10
},
"FLFileHandler": {
"class": "logging.handlers.RotatingFileHandler",
"level": "DEBUG",
"formatter": "baseFormatter",
"filters": ["FLFilter"],
"filename": "log_fl.txt",
"mode": "a",
"maxBytes": 20971520,
"backupCount": 10,
"delay": true
}
},
"loggers": {
"root": {
"level": "INFO",
"handlers": ["consoleHandler", "logFileHandler", "errorFileHandler", "jsonFileHandler", "FLFileHandler"]
}
}
}
1 change: 1 addition & 0 deletions docs/resources/researcher.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading

0 comments on commit 4e6d563

Please sign in to comment.