You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently, there are two ways to format keys in codebook-related dictionaries: 1) "layer{x}_{cb_at}_gcb{y}" (adv name), and 2) "layer{x}_head{y}" (base name). The Second was introduced for convenience but shouldn't be used as it doesn't incorporate the cb_at parameter. So for a model, if in a given layer there are two places codebooks are applied at, the second way wouldn't be able to distinguish between the two codebooks. So this format can be removed and we can use 1) universally.
The text was updated successfully, but these errors were encountered:
Currently, there are two ways to format keys in codebook-related dictionaries: 1) "layer{x}_{cb_at}_gcb{y}" (adv name), and 2) "layer{x}_head{y}" (base name). The Second was introduced for convenience but shouldn't be used as it doesn't incorporate the
cb_at
parameter. So for a model, if in a given layer there are two places codebooks are applied at, the second way wouldn't be able to distinguish between the two codebooks. So this format can be removed and we can use 1) universally.The text was updated successfully, but these errors were encountered: