Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Track internal dendropy reorganization #126

Merged
merged 2 commits into from
Feb 12, 2024
Merged

Conversation

mmore500
Copy link
Contributor

#125 appears to be due to reorganization of internal implementation details in dendropy. This PR patches sepp to follow the new internal organization for dendropy >=4.6.0

Also bumped the pinned version on sepp's dendropy requirement in setup.py.

@aravenscraft
Copy link

I am having the same issue as #125: When I try runupp.py -h, it fails with error "ImportError: cannot import name '_convert_node_to_root_polytomy'. As far as I can discern, it looks like this post provides a temporary solution to the problem, but I am not experienced enough to understand what the solution is. Could you please explain, as if to a child, how I can use the "PR patch" to get UPP/SEPP running? (I just did a fresh install of SEPP, so I guess this patch isn't included in the current install?)

@mmore500
Copy link
Contributor Author

mmore500 commented Jul 14, 2023 via email

@aravenscraft
Copy link

Thank you for the quick reply! I've spent hours and hours trying to solve this one problem, and I'm no closer to a solution. I tried your suggestion but got the same error:

(py37) biol-128379:seppRuns ravenscrafta$ python3 -m pip install dendropy --upgrade
Requirement already satisfied: dendropy in /Users/ravenscrafta/miniconda3/envs/py37/lib/python3.7/site-packages/DendroPy-4.6.1-py3.7.egg (4.6.1)
Requirement already satisfied: setuptools in /Users/ravenscrafta/miniconda3/envs/py37/lib/python3.7/site-packages (from dendropy) (68.0.0)
(py37) biol-128379:seppRuns ravenscrafta$ run_sepp.py -h
/Users/ravenscrafta/miniconda3/envs/py37/bin/run_sepp.py:4: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
import('pkg_resources').run_script('sepp==4.5.2', 'run_sepp.py')
Traceback (most recent call last):
File "/Users/ravenscrafta/miniconda3/envs/py37/bin/run_sepp.py", line 4, in
import('pkg_resources').run_script('sepp==4.5.2', 'run_sepp.py')
File "/Users/ravenscrafta/miniconda3/envs/py37/lib/python3.7/site-packages/pkg_resources/init.py", line 722, in run_script
self.require(requires)[0].run_script(script_name, ns)
File "/Users/ravenscrafta/miniconda3/envs/py37/lib/python3.7/site-packages/pkg_resources/init.py", line 1561, in run_script
exec(code, namespace, namespace)
File "/Users/ravenscrafta/miniconda3/envs/py37/lib/python3.7/site-packages/sepp-4.5.2-py3.7.egg/EGG-INFO/scripts/run_sepp.py", line 3, in
from sepp.exhaustive import ExhaustiveAlgorithm
File "/Users/ravenscrafta/miniconda3/envs/py37/lib/python3.7/site-packages/sepp-4.5.2-py3.7.egg/sepp/exhaustive.py", line 6, in
from sepp.algorithm import AbstractAlgorithm
File "/Users/ravenscrafta/miniconda3/envs/py37/lib/python3.7/site-packages/sepp-4.5.2-py3.7.egg/sepp/algorithm.py", line 12, in
from sepp.tree import PhylogeneticTree
File "/Users/ravenscrafta/miniconda3/envs/py37/lib/python3.7/site-packages/sepp-4.5.2-py3.7.egg/sepp/tree.py", line 25, in
from dendropy.datamodel.treemodel import _convert_node_to_root_polytomy as
ImportError: cannot import name '_convert_node_to_root_polytomy' from 'dendropy.datamodel.treemodel' (/Users/ravenscrafta/miniconda3/envs/py37/lib/python3.7/site-packages/DendroPy-4.6.1-py3.7.egg/dendropy/datamodel/treemodel/init.py)

@mmore500
Copy link
Contributor Author

mmore500 commented Jul 14, 2023 via email

@smirarab
Copy link
Owner

@mmore500 thanks for the suggested fix. Were you able to test this? Did it work fine in your hands?

@mmore500
Copy link
Contributor Author

mmore500 commented Jul 15, 2023 via email

@aravenscraft
Copy link

Tried it, no luck. :-/ Not sure if this is related, but it tells me to update my version of conda. I tried to, but the update command didn't work.

biol-128379:~ ravenscrafta$ conda activate py37
(py37) biol-128379:~ ravenscrafta$ conda update dendropy
Collecting package metadata (current_repodata.json): done
Solving environment: done

==> WARNING: A newer version of conda exists. <==
current version: 4.14.0
latest version: 23.5.2

Please update conda by running

$ conda update -n base -c conda-forge conda

All requested packages already installed.

Retrieving notices: ...working... done
(py37) biol-128379:~ ravenscrafta$ run_sepp.py -h
/Users/ravenscrafta/miniconda3/envs/py37/bin/run_sepp.py:4: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
import('pkg_resources').run_script('sepp==4.5.2', 'run_sepp.py')
Traceback (most recent call last):
File "/Users/ravenscrafta/miniconda3/envs/py37/bin/run_sepp.py", line 4, in
import('pkg_resources').run_script('sepp==4.5.2', 'run_sepp.py')
File "/Users/ravenscrafta/miniconda3/envs/py37/lib/python3.7/site-packages/pkg_resources/init.py", line 722, in run_script
self.require(requires)[0].run_script(script_name, ns)
File "/Users/ravenscrafta/miniconda3/envs/py37/lib/python3.7/site-packages/pkg_resources/init.py", line 1561, in run_script
exec(code, namespace, namespace)
File "/Users/ravenscrafta/miniconda3/envs/py37/lib/python3.7/site-packages/sepp-4.5.2-py3.7.egg/EGG-INFO/scripts/run_sepp.py", line 3, in
from sepp.exhaustive import ExhaustiveAlgorithm
File "/Users/ravenscrafta/miniconda3/envs/py37/lib/python3.7/site-packages/sepp-4.5.2-py3.7.egg/sepp/exhaustive.py", line 6, in
from sepp.algorithm import AbstractAlgorithm
File "/Users/ravenscrafta/miniconda3/envs/py37/lib/python3.7/site-packages/sepp-4.5.2-py3.7.egg/sepp/algorithm.py", line 12, in
from sepp.tree import PhylogeneticTree
File "/Users/ravenscrafta/miniconda3/envs/py37/lib/python3.7/site-packages/sepp-4.5.2-py3.7.egg/sepp/tree.py", line 25, in
from dendropy.datamodel.treemodel import _convert_node_to_root_polytomy as
ImportError: cannot import name '_convert_node_to_root_polytomy' from 'dendropy.datamodel.treemodel' (/Users/ravenscrafta/miniconda3/envs/py37/lib/python3.7/site-packages/dendropy/datamodel/treemodel/init.py)
(py37) biol-128379:~ ravenscrafta$ conda update -n base -c conda-forge conda
Collecting package metadata (current_repodata.json): done
Solving environment: done

==> WARNING: A newer version of conda exists. <==
current version: 4.14.0
latest version: 23.5.2

Please update conda by running

$ conda update -n base -c conda-forge conda

All requested packages already installed.

Retrieving notices: ...working... done
(py37) biol-128379:~ ravenscrafta$ conda update dendropy
Collecting package metadata (current_repodata.json): done
Solving environment: done

==> WARNING: A newer version of conda exists. <==
current version: 4.14.0
latest version: 23.5.2

Please update conda by running

$ conda update -n base -c conda-forge conda

All requested packages already installed.

Retrieving notices: ...working... done
(py37) biol-128379:~ ravenscrafta$ run_sepp.py -h
/Users/ravenscrafta/miniconda3/envs/py37/bin/run_sepp.py:4: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
import('pkg_resources').run_script('sepp==4.5.2', 'run_sepp.py')
Traceback (most recent call last):
File "/Users/ravenscrafta/miniconda3/envs/py37/bin/run_sepp.py", line 4, in
import('pkg_resources').run_script('sepp==4.5.2', 'run_sepp.py')
File "/Users/ravenscrafta/miniconda3/envs/py37/lib/python3.7/site-packages/pkg_resources/init.py", line 722, in run_script
self.require(requires)[0].run_script(script_name, ns)
File "/Users/ravenscrafta/miniconda3/envs/py37/lib/python3.7/site-packages/pkg_resources/init.py", line 1561, in run_script
exec(code, namespace, namespace)
File "/Users/ravenscrafta/miniconda3/envs/py37/lib/python3.7/site-packages/sepp-4.5.2-py3.7.egg/EGG-INFO/scripts/run_sepp.py", line 3, in
from sepp.exhaustive import ExhaustiveAlgorithm
File "/Users/ravenscrafta/miniconda3/envs/py37/lib/python3.7/site-packages/sepp-4.5.2-py3.7.egg/sepp/exhaustive.py", line 6, in
from sepp.algorithm import AbstractAlgorithm
File "/Users/ravenscrafta/miniconda3/envs/py37/lib/python3.7/site-packages/sepp-4.5.2-py3.7.egg/sepp/algorithm.py", line 12, in
from sepp.tree import PhylogeneticTree
File "/Users/ravenscrafta/miniconda3/envs/py37/lib/python3.7/site-packages/sepp-4.5.2-py3.7.egg/sepp/tree.py", line 25, in
from dendropy.datamodel.treemodel import _convert_node_to_root_polytomy as
ImportError: cannot import name '_convert_node_to_root_polytomy' from 'dendropy.datamodel.treemodel' (/Users/ravenscrafta/miniconda3/envs/py37/lib/python3.7/site-packages/dendropy/datamodel/treemodel/init.py)
(py37) biol-128379:~ ravenscrafta$ conda deactivate

@aravenscraft
Copy link

I also tried deleting SEPP and dendropy and running a fresh install, as you recommended. The installation works, except that it gives me the error: Could not find suitable distribution for Requirement.parse('dendropy>=4.0.0'). I think I got this same error when I previously tried to install SEPP. ...Could my operating system just be too old?

biol-128379:~ ravenscrafta$ git clone https://github.com/smirarab/sepp.git
Cloning into 'sepp'...
remote: Enumerating objects: 4088, done.
remote: Counting objects: 100% (701/701), done.
remote: Compressing objects: 100% (208/208), done.
remote: Total 4088 (delta 505), reused 494 (delta 493), pack-reused 3387
Receiving objects: 100% (4088/4088), 33.97 MiB | 1.31 MiB/s, done.
Resolving deltas: 100% (2618/2618), done.
biol-128379:~ ravenscrafta$ cd sepp
biol-128379:sepp ravenscrafta$ python setup.py config
running config

Creating main sepp config file at /Users/ravenscrafta/.sepp/main.config and tools at /Users/ravenscrafta/.sepp
biol-128379:sepp ravenscrafta$ sudo python setup.py install
Password:
running install
Checking .pth file support in /Library/Python/2.7/site-packages/
/usr/bin/python -E -c pass
TEST PASSED: /Library/Python/2.7/site-packages/ appears to support .pth files
running bdist_egg
running egg_info
creating sepp.egg-info
writing requirements to sepp.egg-info/requires.txt
writing sepp.egg-info/PKG-INFO
writing top-level names to sepp.egg-info/top_level.txt
writing dependency_links to sepp.egg-info/dependency_links.txt
writing manifest file 'sepp.egg-info/SOURCES.txt'
reading manifest file 'sepp.egg-info/SOURCES.txt'
writing manifest file 'sepp.egg-info/SOURCES.txt'
installing library code to build/bdist.macosx-10.14-intel/egg
running install_lib
running build_py
creating build
creating build/lib
creating build/lib/sepp
copying sepp/algorithm.py -> build/lib/sepp
copying sepp/tree.py -> build/lib/sepp
copying sepp/math_utils.py -> build/lib/sepp
copying sepp/config.py -> build/lib/sepp
copying sepp/alignment.py -> build/lib/sepp
copying sepp/exhaustive_upp.py -> build/lib/sepp
copying sepp/decompose_tree.py -> build/lib/sepp
copying sepp/backtranslate.py -> build/lib/sepp
copying sepp/init.py -> build/lib/sepp
copying sepp/scratch.py -> build/lib/sepp
copying sepp/filemgr.py -> build/lib/sepp
copying sepp/jobs.py -> build/lib/sepp
copying sepp/problem.py -> build/lib/sepp
copying sepp/exhaustive.py -> build/lib/sepp
copying sepp/ensemble.py -> build/lib/sepp
copying sepp/scheduler.py -> build/lib/sepp
copying sepp/checkpointing.py -> build/lib/sepp
creating build/bdist.macosx-10.14-intel
creating build/bdist.macosx-10.14-intel/egg
creating build/bdist.macosx-10.14-intel/egg/sepp
copying build/lib/sepp/algorithm.py -> build/bdist.macosx-10.14-intel/egg/sepp
copying build/lib/sepp/tree.py -> build/bdist.macosx-10.14-intel/egg/sepp
copying build/lib/sepp/math_utils.py -> build/bdist.macosx-10.14-intel/egg/sepp
copying build/lib/sepp/config.py -> build/bdist.macosx-10.14-intel/egg/sepp
copying build/lib/sepp/alignment.py -> build/bdist.macosx-10.14-intel/egg/sepp
copying build/lib/sepp/exhaustive_upp.py -> build/bdist.macosx-10.14-intel/egg/sepp
copying build/lib/sepp/decompose_tree.py -> build/bdist.macosx-10.14-intel/egg/sepp
copying build/lib/sepp/backtranslate.py -> build/bdist.macosx-10.14-intel/egg/sepp
copying build/lib/sepp/init.py -> build/bdist.macosx-10.14-intel/egg/sepp
copying build/lib/sepp/scratch.py -> build/bdist.macosx-10.14-intel/egg/sepp
copying build/lib/sepp/filemgr.py -> build/bdist.macosx-10.14-intel/egg/sepp
copying build/lib/sepp/jobs.py -> build/bdist.macosx-10.14-intel/egg/sepp
copying build/lib/sepp/problem.py -> build/bdist.macosx-10.14-intel/egg/sepp
copying build/lib/sepp/exhaustive.py -> build/bdist.macosx-10.14-intel/egg/sepp
copying build/lib/sepp/ensemble.py -> build/bdist.macosx-10.14-intel/egg/sepp
copying build/lib/sepp/scheduler.py -> build/bdist.macosx-10.14-intel/egg/sepp
copying build/lib/sepp/checkpointing.py -> build/bdist.macosx-10.14-intel/egg/sepp
byte-compiling build/bdist.macosx-10.14-intel/egg/sepp/algorithm.py to algorithm.pyc
byte-compiling build/bdist.macosx-10.14-intel/egg/sepp/tree.py to tree.pyc
byte-compiling build/bdist.macosx-10.14-intel/egg/sepp/math_utils.py to math_utils.pyc
byte-compiling build/bdist.macosx-10.14-intel/egg/sepp/config.py to config.pyc
byte-compiling build/bdist.macosx-10.14-intel/egg/sepp/alignment.py to alignment.pyc
byte-compiling build/bdist.macosx-10.14-intel/egg/sepp/exhaustive_upp.py to exhaustive_upp.pyc
byte-compiling build/bdist.macosx-10.14-intel/egg/sepp/decompose_tree.py to decompose_tree.pyc
byte-compiling build/bdist.macosx-10.14-intel/egg/sepp/backtranslate.py to backtranslate.pyc
byte-compiling build/bdist.macosx-10.14-intel/egg/sepp/init.py to init.pyc
byte-compiling build/bdist.macosx-10.14-intel/egg/sepp/scratch.py to scratch.pyc
byte-compiling build/bdist.macosx-10.14-intel/egg/sepp/filemgr.py to filemgr.pyc
byte-compiling build/bdist.macosx-10.14-intel/egg/sepp/jobs.py to jobs.pyc
byte-compiling build/bdist.macosx-10.14-intel/egg/sepp/problem.py to problem.pyc
byte-compiling build/bdist.macosx-10.14-intel/egg/sepp/exhaustive.py to exhaustive.pyc
byte-compiling build/bdist.macosx-10.14-intel/egg/sepp/ensemble.py to ensemble.pyc
byte-compiling build/bdist.macosx-10.14-intel/egg/sepp/scheduler.py to scheduler.pyc
byte-compiling build/bdist.macosx-10.14-intel/egg/sepp/checkpointing.py to checkpointing.pyc
installing package data to build/bdist.macosx-10.14-intel/egg
running install_data
copying home.path -> build/bdist.macosx-10.14-intel/egg/
creating build/bdist.macosx-10.14-intel/egg/EGG-INFO
installing scripts to build/bdist.macosx-10.14-intel/egg/EGG-INFO/scripts
running install_scripts
running build_scripts
creating build/scripts-2.7
copying and adjusting run_sepp.py -> build/scripts-2.7
copying and adjusting run_upp.py -> build/scripts-2.7
copying and adjusting split_sequences.py -> build/scripts-2.7
changing mode of build/scripts-2.7/run_sepp.py from 644 to 755
changing mode of build/scripts-2.7/run_upp.py from 644 to 755
changing mode of build/scripts-2.7/split_sequences.py from 644 to 755
creating build/bdist.macosx-10.14-intel/egg/EGG-INFO/scripts
copying build/scripts-2.7/run_upp.py -> build/bdist.macosx-10.14-intel/egg/EGG-INFO/scripts
copying build/scripts-2.7/split_sequences.py -> build/bdist.macosx-10.14-intel/egg/EGG-INFO/scripts
copying build/scripts-2.7/run_sepp.py -> build/bdist.macosx-10.14-intel/egg/EGG-INFO/scripts
changing mode of build/bdist.macosx-10.14-intel/egg/EGG-INFO/scripts/run_upp.py to 755
changing mode of build/bdist.macosx-10.14-intel/egg/EGG-INFO/scripts/split_sequences.py to 755
changing mode of build/bdist.macosx-10.14-intel/egg/EGG-INFO/scripts/run_sepp.py to 755
copying sepp.egg-info/PKG-INFO -> build/bdist.macosx-10.14-intel/egg/EGG-INFO
copying sepp.egg-info/SOURCES.txt -> build/bdist.macosx-10.14-intel/egg/EGG-INFO
copying sepp.egg-info/dependency_links.txt -> build/bdist.macosx-10.14-intel/egg/EGG-INFO
copying sepp.egg-info/requires.txt -> build/bdist.macosx-10.14-intel/egg/EGG-INFO
copying sepp.egg-info/top_level.txt -> build/bdist.macosx-10.14-intel/egg/EGG-INFO
zip_safe flag not set; analyzing archive contents...
sepp.filemgr: module MAY be using inspect.getouterframes
sepp.config: module references file
sepp.exhaustive_upp: module references file
sepp.init: module references path
creating dist
creating 'dist/sepp-4.5.2-py2.7.egg' and adding 'build/bdist.macosx-10.14-intel/egg' to it
removing 'build/bdist.macosx-10.14-intel/egg' (and everything under it)
Processing sepp-4.5.2-py2.7.egg
removing '/Library/Python/2.7/site-packages/sepp-4.5.2-py2.7.egg' (and everything under it)
creating /Library/Python/2.7/site-packages/sepp-4.5.2-py2.7.egg
Extracting sepp-4.5.2-py2.7.egg to /Library/Python/2.7/site-packages
sepp 4.5.2 is already the active version in easy-install.pth
Installing run_upp.py script to /usr/local/bin
Installing split_sequences.py script to /usr/local/bin
Installing run_sepp.py script to /usr/local/bin

Installed /Library/Python/2.7/site-packages/sepp-4.5.2-py2.7.egg
Processing dependencies for sepp==4.5.2
Searching for dendropy>=4.0.0
Reading https://pypi.python.org/simple/dendropy/
Couldn't find index page for 'dendropy' (maybe misspelled?)
Scanning index of all packages (this may take a while)
Reading https://pypi.python.org/simple/
No local packages or download links found for dendropy>=4.0.0
error: Could not find suitable distribution for Requirement.parse('dendropy>=4.0.0')

@aravenscraft
Copy link

In the SEPP tutorial, it says that I can run SEPP on a virtual machine. I installed VirtualBox but when I click the link in the tutorial to download the Ubuntu VM image with SEPP installed, I get a Google Drive page that says "Sorry, the file you have requested does not exist." Is the VM image no longer available?

@mmore500
Copy link
Contributor Author

mmore500 commented Jul 15, 2023 via email

@aravenscraft
Copy link

THAT WORKED! Thank you!!

@mmore500
Copy link
Contributor Author

mmore500 commented Jul 17, 2023 via email

@smirarab
Copy link
Owner

I don’t have an anaconda install so I wasn’t able to test — just guessing! Only other suggestions I could provide are to try uninstalling everything and starting fresh. Sometimes that works. Past that, you might also look into working in a virtual environment or even a container.

-- Matthew Andres Moreno https://mmore500.github.io

Sorry, I didn't mean your solution for aravenscraft's problem. I meant the PR. Did that code change work for you? I don't suppose you know if the edited code works with old dendropy as well?

@mmore500
Copy link
Contributor Author

Apologies for the confusion —- I must have gotten mixed up in replying to the messages!

Just opened this up and used some commands from .travis.yml to install sepp then run test/unittest with nose. Passed!

Only note is that tests aren’t compatible with Python3.10+ due to removal of deprecated aliases to Collections Abstract Base Classes from the collections module https://docs.python.org/3/whatsnew/3.10.html#removed

@mmore500
Copy link
Contributor Author

Would you be interested in a pull request to get CI back up and running? Looks like Travis is no longer active.

@lrauschning
Copy link

Would it be possible to merge this PR and release a fix? It seems to be working on my machine at least, and the current state causes the bioconda package to break (see bioconda/bioconda-recipes#43112).
jeetsukumaran/DendroPy#169 could be resolved so as to maintain the call from Node._convert_node_to_root_polytomy, so this fix is forward-compatible (or otherwise the code can be switched to use the new public interface when it exists).
Thanks,
Leon

@ac-simpson
Copy link

A note for those coming here to solve their sepp issue (as I did today) - I had dendropy installed both via conda and pip, and so downgrading dendropy via conda didn't work until I uninstalled the pip version as well. (look at "conda list" vs "pip list").

@mmore500
Copy link
Contributor Author

Hi @smirarab any word on getting this merged? I’m getting lots of folks on the Dendropy issue tracker related to this issue and would really appreciate getting this fix in place to cut down on that workload on my end.

@smirarab smirarab merged commit c276adf into smirarab:master Feb 12, 2024
@smirarab
Copy link
Owner

Hi @smirarab any word on getting this merged? I’m getting lots of folks on the Dendropy issue tracker related to this issue and would really appreciate getting this fix in place to cut down on that workload on my end.

I just merged but I am currently a bit unavailable and unable to test. Can you check this merged version works fine?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants