Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

splitters need to be warmed up #173

Merged
merged 4 commits into from
Jan 22, 2025
Merged

Conversation

davidsbatista
Copy link
Contributor

Proposed Changes:

  • The HierarchicalDocumentSplitter relies on the DocumentSplitter to build different hierarchy levels. The DocumentSplitter now needs to be warmup if it's meant to split_by="sentence". Therefore in the HierarchicalDocumentSplitteris configured to split_by="sentence" needs to call the .warm_up() on all the different splitters for each level.

How did you test it?

  • manual verification + CI tests

Checklist

@coveralls
Copy link

coveralls commented Jan 21, 2025

Pull Request Test Coverage Report for Build 12912847635

Details

  • 0 of 0 changed or added relevant lines in 0 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage increased (+0.01%) to 78.691%

Totals Coverage Status
Change from base Build 12898346371: 0.01%
Covered Lines: 1359
Relevant Lines: 1727

💛 - Coveralls

@davidsbatista davidsbatista marked this pull request as ready for review January 21, 2025 11:23
@davidsbatista davidsbatista requested a review from a team as a code owner January 21, 2025 11:23
@davidsbatista davidsbatista requested review from mpangrazzi and removed request for a team January 21, 2025 11:23
@davidsbatista davidsbatista merged commit 49ab954 into main Jan 22, 2025
8 checks passed
@davidsbatista davidsbatista deleted the fixing-hierarchical-splitter branch January 22, 2025 16:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants