v0.4.0: PopTorch 3.0, DistilBERT and new notebooks

SDK 3.0

The main feature this release provide is the support of PopTorch version 3.0 (#183), which comes with a new way of tracing PyTorch models: the PyTorch dispatcher is used instead of torch.jit.trace. Not only tracing is now faster, it is also much more powerful (more about it here).

General

• LAMB does not update the bias parameters anymore (#178)
• The DistilBERT architecture is now supported for the tasks: (#181)
  • Masked language modeling
  • Multiple choice
  • Question answering
  • Sequence classification
  • Token classification
• The masked language modeling task is now supported by Deberta
• The IPUTrainer and IPUTrainingArguments were synchronized with their transformers counterparts (#179)
• Some parameters in the IPUConfig were removed:
  • use_popdist
  • decompose_grad_sum
  • profile_dir

Bug fixes

• Documentation building fixes
• Wav2vec2 with dataloder_mode=async_rebatched fixed (#168)

Notebooks

• Audio classification for HuBERT notebook (#157)
• Language modeling finetuning notebook (#161)
• Question answering notebook (#163)
• Multiple choice notebook (#166)
• A notebook showing how to train a model supported in the librarie (#171)

Documentation

The documentation was updated, and contains more content, for instance:

• The IPUTrainer API is described
• The IPUConfig attributes are explained
• A new page explaining how to contribute by adding a new model architecture to the library