Code

Model loading#31

Code

Merged

MenuaB merged 17 commits intomainYerevaNN/YNNtitan:mainfrom model_loadingYerevaNN/YNNtitan:model_loadingCopy head branch name to clipboardOct 19, 2024

+2,509,133-185

Commits

Commits on Oct 2, 2024

• prepare llama3.2-1b for pretraining

  tigranfah

  committed
  bc00070

  Copy full SHA for bc00070

• temporeraly remove many of the properties for llama 3.2 training

  tigranfah

  committed
  7778ea3

  Copy full SHA for 7778ea3

Commits on Oct 4, 2024

• add llama3.2 export and share the aim hash to all ranks to have them save the checkpoint into the same place

  tigranfah

  committed
  9f5099d

  Copy full SHA for 9f5099d

• Merge branch 'hparams' into model_loading

  tigranfah

  committed
  a88f514

  Copy full SHA for a88f514

• tune the config of llama3.2 for pretraining wiht safe representation and some molecular properties

  tigranfah

  committed
  d27cdee

  Copy full SHA for d27cdee

Commits on Oct 6, 2024

• add loading from specific step

  tigranfah

  committed
  40ae625

  Copy full SHA for 40ae625

Commits on Oct 7, 2024

• small fices

  tigranfah

  committed
  2139ac6

  Copy full SHA for 2139ac6

Commits on Oct 8, 2024

• change the properties used

  tigranfah

  committed
  3b171f7

  Copy full SHA for 3b171f7

• build llama 3.2 tokenizer with special tokens for molecule and property tags, implement safe data processing

  tigranfah

  committed
  8ac0a17

  Copy full SHA for 8ac0a17

• rename llama 3.2 tokenizer

  tigranfah

  committed
  bae05b5

  Copy full SHA for bae05b5

• resize the llama 3.2 token embedding size

  tigranfah

  committed
  5b23fa3

  Copy full SHA for 5b23fa3

Commits on Oct 9, 2024

• add data processing with downsampling

  tigranfah

  committed
  ed0c9f2

  Copy full SHA for ed0c9f2

Commits on Oct 10, 2024

• add new learning rates, log the learning rate and track it on aim

  tigranfah

  committed
  3d4946c

  Copy full SHA for 3d4946c

• change warmup steps to 1000 in llama 3 1b for lr tunning

  tigranfah

  committed
  e8fd8a3

  Copy full SHA for e8fd8a3

Commits on Oct 11, 2024

• prepare for llama 3.2 1b training, lr=2.5-e4

  tigranfah

  committed
  5dc2dbb

  Copy full SHA for 5dc2dbb

Commits on Oct 17, 2024

• Add CONFORMER tag

  tigranfah

  committed
  1a03114

  Copy full SHA for 1a03114

Commits on Oct 18, 2024

• add logits matching verified when downloading and exporting to hf

  tigranfah

  committed
  7b97d4d

  Copy full SHA for 7b97d4d