Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Handle param converter according to exchange format #3115

Merged
merged 27 commits into from
Dec 17, 2024
Merged

Handle param converter according to exchange format #3115

merged 27 commits into from
Dec 17, 2024

Conversation

ZiyueXu77
Copy link
Collaborator

Fixes # .

Description

Torch can use both numpy and tensor converter, numpy/tf use numpy, raw use none

Types of changes

  • Non-breaking change (fix or new feature that would not break existing functionality).
  • Breaking change (fix or new feature that would cause existing functionality to change).
  • New tests added to cover the changes.
  • Quick tests passed locally by running ./runtest.sh.
  • In-line docstrings updated.
  • Documentation updated.

nvidianz and others added 23 commits November 26, 2024 21:10
@nvidianz
Added support for bfloat16 tensor using JIT
048da33
@ZiyueXu77
directly send tensor via jit serialization
0dcefd3
@ZiyueXu77
polish sft_job
a2e6849
@ZiyueXu77
polish sft_job
247e7c7
@ZiyueXu77
polish local training script
6acec33
@ZiyueXu77
polish tensor params converter
15240d8
@ZiyueXu77
polish decomposer
531304b
@ZiyueXu77
format correction
059c64f
@ZiyueXu77
header update
f694a14
@ZiyueXu77
Merge branch 'tensor_comm' into bf16_decomposer
5bd20f3
@ZiyueXu77
Merge pull request #8 from nvidianz/bf16_decomposer
92f4225 
Added support for BF16 in TensorDecomposer
@ZiyueXu77
Merge branch 'NVIDIA:main' into tensor_comm
999375d
@ZiyueXu77
update decomposer
5c567c7
@ZiyueXu77
end to end tensor communication passed
3cf01c4
@ZiyueXu77
Merge branch 'NVIDIA:main' into tensor_comm
6cc4c8c
@ZiyueXu77
update quantization filters to handle tensor
dca4fd8
@ZiyueXu77
bug fixes and unittest updates
395f2de
@ZiyueXu77
Merge branch 'main' into tensor_comm
25fd3ca
@ZiyueXu77
unit test cannot run on gpu, update case
e59210f
@ZiyueXu77
bug fixes and polish
54ea4fb
@ZiyueXu77
format update
8376894
@ZiyueXu77
handle param converter according to exchange_format
92fcc45
@ZiyueXu77
Merge branch 'main' into tensor_comm
4bb2b5a
@ZiyueXu77 ZiyueXu77 requested review from YuanTingHsieh, holgerroth and SYangster and removed request for YuanTingHsieh December 17, 2024 18:00
SYangster
SYangster reviewed Dec 17, 2024
View reviewed changes
Copy link
Collaborator

@SYangster SYangster left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly LGTM, minor comment

nvflare/job_config/script_runner.py Show resolved Hide resolved
@ZiyueXu77
Copy link
Collaborator Author

/build

@ZiyueXu77 ZiyueXu77 enabled auto-merge (squash) December 17, 2024 21:56
@ZiyueXu77 ZiyueXu77 requested a review from holgerroth December 17, 2024 21:56
holgerroth
holgerroth approved these changes Dec 17, 2024
View reviewed changes
Copy link
Collaborator

@holgerroth holgerroth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

@ZiyueXu77 ZiyueXu77 merged commit 6454eea into NVIDIA:main Dec 17, 2024
20 checks passed
@ZiyueXu77 ZiyueXu77 deleted the tensor_comm branch December 17, 2024 23:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@SYangster SYangster SYangster left review comments

@holgerroth holgerroth holgerroth approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@ZiyueXu77 @holgerroth @SYangster @nvidianz