Skip to content

Commit

Permalink
Merge branch 'feature/documentation' into develop
Browse files Browse the repository at this point in the history
  • Loading branch information
aliavni committed Sep 6, 2020
2 parents c199bce + f88bbb6 commit 6ec9b95
Show file tree
Hide file tree
Showing 3 changed files with 29 additions and 4 deletions.
4 changes: 3 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -137,4 +137,6 @@ dmypy.json
# Cython debug symbols
cython_debug/

.DS_Store
.DS_Store

.env
29 changes: 26 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,10 +1,33 @@
# Drugbank Scraper

## Development
## Run

This repo uses `Python 3.8.5`.

1. See [Pipenv & Virtual Environments](https://docs.python-guide.org/dev/virtualenvs/) guide to create a virtual environment.
1. Activate the virtual environment.
1. See [Pipenv & Virtual Environments](https://docs.python-guide.org/dev/virtualenvs/) guide to create a virtual environment and activate the virtual environment
1. Install requirements with `pip install -r requirements.txt`.
1. Create PostgreSQL database.
1. Create .env file with `cp .env.template .env` and fill environment variables.

## Spiders

### Drug

Run `scrapy crawl drug` to run drug spider and populate database. This will scrape data, create and populate database tables. Final data will be in `drugbank` schema. This will:

1. Scrape following data:
* DrugBank ID
* SMILES string
* Gene name
* Actions and alternative identifiers of every target.
2. Save scraped data into the previously created PostgreSQL database.


![Drugbank Schema](static/drugbank_schema.png "Drugbank Schema")


## Development

1. See the virtual environment step above.
1. Install requirements with `pip install -r requirements-dev.txt`.
1. Run `pre-commit install` to install pre-commit hooks. This repo is already set up to use some pre-commit hooks for code quality purposes. Configuration file is available [here](.pre-commit-config.yaml). More information about pre-commit is available on [their website](https://pre-commit.com/).
Binary file added static/drugbank_schema.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 6ec9b95

Please sign in to comment.