Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Likely remove Parts' tokens #167

Closed
2 tasks done
juanmirocks opened this issue Feb 5, 2016 · 1 comment
Closed
2 tasks done

Likely remove Parts' tokens #167

juanmirocks opened this issue Feb 5, 2016 · 1 comment
Assignees

Comments

@juanmirocks
Copy link
Collaborator

juanmirocks commented Feb 5, 2016

A list of tokens was added by Ashish for the Rostlab/relna project. However, lists of tokens are already incorporated into sentences. Therefore, this might be redundant and confusing for nalaf users.

  • Investigate if tokens are needed in parts / sentences / entities (see General list of tasks of merging nalaf, nala, relna #162)
    -> so far left in Entity as it's extensively used -- unnecessary in Part as it's already in its sentences
  • ...take necessary steps
    -> remove tokens from Part
@juanmirocks juanmirocks added this to the 4-Upcoming milestone Feb 5, 2016
@juanmirocks
Copy link
Collaborator Author

[05/02/16 12:31:40] Aleksandar Bojchevski: Any idea why the Part object has:
self.tokens = []
[05/02/16 12:32:07] Aleksandar Bojchevski: we never store tokens there... the tokens are part of 
self.sentences = [[]]
[05/02/16 12:34:45] Aleksandar Bojchevski: seems to be added recently
[05/02/16 12:38:07] Aleksandar Bojchevski: https://github.com/Rostlab/nalaf/commit/8a6e3783e987f0616a0cb6533d8df30c0a255d66
[05/02/16 12:38:18] Aleksandar Bojchevski: apperentaly it was added by ashish
[05/02/16 12:38:45] Aleksandar Bojchevski: hmm... might be an artefact... we should check relna
[05/02/16 12:46:17] Juan Miguel Cejuela: let me check
[05/02/16 12:46:35] Juan Miguel Cejuela: What I can recall is that he needed tokens in entities
[05/02/16 12:48:18] Aleksandar Bojchevski: that I know
[05/02/16 12:48:22] Aleksandar Bojchevski: but not in parts
[05/02/16 12:49:05] Juan Miguel Cejuela: Yes, in his file parsers.py I see that he is also using tokens in parts

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

1 participant