Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve performance by supporting repo.lstree with a specific path #457

Open
wants to merge 3 commits into
base: master
Choose a base branch
from

Conversation

dometto
Copy link
Member

@dometto dometto commented Jan 11, 2025

This PR adds support for new adapter features which allow us to call lstree for a specific directory in the repo, and non-recursively:

gollum/rugged_adapter#67
gollum/rjgit_adapter#26
Specs: gollum/adapter_specs#19

It depends on those changes.

This will eventually allow us to improve performance of the Overview route, as we'll no longer need to get and loop over the entire contents of the repository.

@dometto
Copy link
Member Author

dometto commented Jan 11, 2025

Still getting some tests errors. Most are just due to changed expectations in the Git Access tests, as the results now contain TreeEntry objects as well as BlobEntry. But there is also this one:

Error: test_should_inherit_header_footer_sidebar_pages_from_parent_directories(with_a_page_file_dir_enabled): Rugged::RepositoryError: could not find repository at '/'
/Users/3060845/.rvm/gems/ruby-3.0.0@gollum-lib/gems/gollum-rugged_adapter-3.0/lib/rugged_adapter/git_layer_rugged.rb:596:in `new'
/Users/3060845/.rvm/gems/ruby-3.0.0@gollum-lib/gems/gollum-rugged_adapter-3.0/lib/rugged_adapter/git_layer_rugged.rb:596:in `initialize'
/Users/3060845/Code/gollum_project/gollum-lib/lib/gollum-lib/git_access.rb:16:in `new'
/Users/3060845/Code/gollum_project/gollum-lib/lib/gollum-lib/git_access.rb:16:in `initialize'
/Users/3060845/Code/gollum_project/gollum-lib/lib/gollum-lib/wiki.rb:722:in `new'
/Users/3060845/Code/gollum_project/gollum-lib/lib/gollum-lib/wiki.rb:722:in `tree_map_for'
/Users/3060845/Code/gollum_project/gollum-lib/lib/gollum-lib/page.rb:327:in `find_sub_pages'
/Users/3060845/Code/gollum_project/gollum-lib/lib/gollum-lib/page.rb:278:in `sidebar'
/Users/3060845/Code/gollum_project/gollum-lib/test/test_page.rb:278:in `block (2 levels) in <top (required)>'

In general, we should make sure the default case (path == '/') is handled well, and potential cases where non-existent directory paths are passed to lstree should be handled correctly.

if (sha = ref_to_sha(ref))
get_cache(:tree, sha) { tree!(sha) }
get_cache(:tree, "#{sha}#{path}") { tree!(sha, path, recursive) }
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cache key currently doesn't differentiate between recursive being true or false.

def tree!(sha, path = '/', recursive = true)
tree = @repo.lstree(sha, (Pathname.new(@page_file_dir.to_s) + path).to_s, recursive: recursive )
tree.reduce([]) do |accumulator, entry|
if !@page_file_dir || entry[:path].start_with?("#{@page_file_dir}/") # guard against directory traversal
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure this line is needed, if we're already passing @page_file_dir as the base path.

BlobEntry.new(entry[:sha], entry[:path], entry[:size], entry[:mode])
when 'tree'
TreeEntry.new(entry[:sha], entry[:path])
end
end
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could also opt to return blobs and trees in separate Arrays, so callers who need only the blobs can ignore the trees.

Either way, I guess initializing the TreeEntries has minor performance impact when looping into the entire wiki.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant