Code

Fixing MuP #1061

Code

Open

marcobellagente93 wants to merge 10 commits intoEleutherAI:mainEleutherAI/gpt-neox:mainfrom marcobellagente93:mup_fixmarcobellagente93/gpt-neox:mup_fixCopy head branch name to clipboard

+59-18

Commits

Commits on Oct 19, 2023

• added mup version with fixes to neox root

  marcobellagente93

  committed
  f85352c

  Copy full SHA for f85352c

• removed buggy attention scale factor

  marcobellagente93

  committed
  a24ba2d

  Copy full SHA for a24ba2d

• added comment and removed buggy embedding multiplier

  marcobellagente93

  committed
  e6ae3dc

  Copy full SHA for e6ae3dc

• set use_mup attribute at re-init and fix output multiplier

  marcobellagente93

  committed
  faff65e

  Copy full SHA for faff65e

• do not reinit the output layer

  marcobellagente93

  committed
  4ebfb86

  Copy full SHA for 4ebfb86

• update readme-mup

  marcobellagente93

  committed
  fe70494

  Copy full SHA for fe70494

• Delete mup/CODE_OF_CONDUCT.md

  StellaAthena

  authored
  b5308f3

  Copy full SHA for b5308f3

• Delete mup/mup directory

  StellaAthena

  authored
  57ddc14

  Copy full SHA for 57ddc14

• Delete mup directory

  StellaAthena

  authored
  a34a6c6

  Copy full SHA for a34a6c6

• Update requirements.txt

  StellaAthena

  authored
  05ff7df

  Copy full SHA for 05ff7df