Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bug fix: printing non-distributed data #1756

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

ClaudiaComito
Copy link
Contributor

@ClaudiaComito ClaudiaComito commented Dec 30, 2024

Due Diligence

  • General:
  • Implementation:
    • unit tests: all split configurations tested
    • unit tests: multiple dtypes tested
    • benchmarks: created for new functionality does not apply
    • benchmarks: performance improved or maintained
    • documentation updated where needed

Description

I noticed that printing out moderately large DNDarrays in non-distributed mode (i.e. interactive session or split=None) takes a disproportionate amount of time (see below) compared to printing the underlying tensor. Culprit is the Formatter call.

I've changed the printing module to bypass Formatter if the input dndarray is not distributed. Torch takes care of the data formatting, tests pass.

Example:

data =  ht.random.randn(91392, 52, 4)
print(data)

On 1 process:
main branch: 84 seconds
this PR: 0.01 seconds

Issue/s resolved: #

Changes proposed:

  • let torch handle formatting of non-distributed data

Type of change

  • Bug fix (non-breaking change which fixes an issue)

Memory requirements

NA

Performance

see above

Does this change modify the behaviour of other functions? If so, which?

no

@ClaudiaComito ClaudiaComito added the bug Something isn't working label Dec 30, 2024
@ClaudiaComito ClaudiaComito added this to the 1.5.1 milestone Dec 30, 2024
Copy link
Contributor

Thank you for the PR!

Copy link

codecov bot commented Dec 30, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 92.25%. Comparing base (87f2812) to head (3edaca4).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1756      +/-   ##
==========================================
- Coverage   92.26%   92.25%   -0.01%     
==========================================
  Files          84       84              
  Lines       12445    12447       +2     
==========================================
+ Hits        11482    11483       +1     
- Misses        963      964       +1     
Flag Coverage Δ
unit 92.25% <100.00%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

Thank you for the PR!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant