Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Question about topk_index = topk_index.view(-1) in cnets.py #158

Open
HaoWuSR opened this issue Nov 7, 2024 · 0 comments
Open

Question about topk_index = topk_index.view(-1) in cnets.py #158

HaoWuSR opened this issue Nov 7, 2024 · 0 comments

Comments

@HaoWuSR
Copy link

HaoWuSR commented Nov 7, 2024

Dear Authors,
Thank you for your outstanding work on EAGLE and EAGLE-2! I have a question regarding the implementation, specifically about the line topk_index = topk_index.view(-1).
Let me illustrate my confusion with an example:
When executing topk_index, topk_prob, op = self.sample(last_headout, logits_processor, k=top_k), with top_k=3, we obtain 4 branches:
[
[55, 67, 33],
[25578, 13, 45],
[2, 56, 42],
[32, 64, 89]
]
After applying topk_index = topk_index.view(-1), the output becomes:
[55, 67, 33, 25578, 13, 45, 2, 56, 42, 32, 64, 89]

Then, if self.tree_buffer['tree_indices'][i] = [0, 2], we get select_index = [55, 33]. Is my understanding correct?
Initially, I expected the implementation to select tokens from each branch. However, it appears we're selecting tokens from specific positions in the flattened array. Is there a particular reason for ignoring certain branches? Or is this a deliberate design choice in the tree buffer indices?
I would greatly appreciate ur time!
Regards,

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant