-
Notifications
You must be signed in to change notification settings - Fork 61
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
1 changed file
with
25 additions
and
4 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -38,14 +38,20 @@ A research paper about the framework and our INTELLECT-1 10B experiment is comin | |
|
||
## Getting Started | ||
|
||
1. Install `uv`: | ||
1. Clone: | ||
|
||
```bash | ||
git clone [email protected]:PrimeIntellect-ai/prime.git | ||
``` | ||
|
||
2. Install `uv`: | ||
|
||
```bash | ||
curl -LsSf https://astral.sh/uv/install.sh | sh | ||
source $HOME/.local/bin/env | ||
``` | ||
|
||
2. Set up the environment: | ||
3. Set up the environment: | ||
```bash | ||
sudo apt install iperf -y | ||
uv venv | ||
|
@@ -55,12 +61,27 @@ uv pip install flash-attn --no-build-isolation | |
git submodule update --init --recursive | ||
``` | ||
|
||
3. Log into Hugging Face: | ||
prime uses gated models tokenizers [mistralai/Mistral-7B-v0.1](https://huggingface.co/mistralai/Mistral-7B-v0.1), [meta-llama/Llama-2-7b-hf](https://huggingface.co/meta-llama/Llama-2-7b-hf) and pulls the [C4:en](https://huggingface.co/datasets/allenai/c4) dataset subset. It is required to request access to the models then log into Hugging Face with a read token to begin training. | ||
|
||
4. Log into Hugging Face: | ||
```bash | ||
huggingface-cli login | ||
``` | ||
|
||
all steps: | ||
|
||
``` | ||
git clone [email protected]:PrimeIntellect-ai/prime.git | ||
cd prime | ||
curl -LsSf https://astral.sh/uv/install.sh | sh | ||
source $HOME/.local/bin/env | ||
sudo apt install iperf -y | ||
uv venv | ||
source .venv/bin/activate | ||
uv sync --extra all | ||
uv pip install flash-attn --no-build-isolation | ||
git submodule update --init --recursive | ||
``` | ||
|
||
|
||
### Quick Check | ||
|
||
|