v1.4.2

Small documentation hot-fix release that fixes a couple of issues with docstrings and adds more documentation around the new TensorFlow API.

v1.4.0

This is a noteworthy release as we've added support for TensorFlow for all the backward compatibility loss functions and implemented a Backward Compatibility model class for training TensorFlow models with New Error loss.

Also we've released a Model Comparison widget for that can be used to compare any two models or integrated into a dashboard or other model comparison workflows. The development environment has been updated to support running the Model Comparison widget outside of Jupyter notebook.

As always, this release incorporates some enhancements to existing widget components and a few features and fixes. Please upgrade to the latest version, and let us know what you think.

New support for TensorFlow:

• Added TensorFlow support for all the Backward Compatibility loss functions.
• Implemented a Backward Compatibility model class for training TensorFlow models with New Error loss.

Enhancements to existing widget components:

• Compatibility sweeps are automatically logged with MLflow.

New widget components released:

• Model Comparison widget.

Documentation updates:

• Added TensorFlow notebook examples.
• Added Model Comparison widget notebook example.
• Added Compatibility Model class notebook example.
• Added section on how to use and integrate the Model Comparison widget.

Fixes:

• Fixed issues preventing the user from using the Compatibility Analysis widget multiple times in the same notebook.
• Fixed issues with the Venn diagram not resetting in some circumstances.
• Fixed some pagination issues with the Error instances table.
• Fixed crash when filtering on dataset or dissonance functions.

The latest PyPi package is here: https://pypi.org/project/backwardcompatibilityml/1.4.0/

The latest documentation is here: https://backwardcompatibilityml.readthedocs.io/en/v1.4.0/

v1.3.0

This release includes some enhancements to existing widget components, new widget components and new widget components interactivity, and documentation, in preparation for a public demo during the NeurIPS 2020 presentation.

Please upgrade to the latest version, and let us know what you think.

Enhancements to existing widget components:

• Updated error instances table pagination UI to match histogram
• Added information tooltips to all components to clarify terms and usage

New widget components interactivity:

• Venn Diagram:
  • Filtering instances by clicking on the diagram label (particularly useful when Venn sections are very small)

Documentation updates:

• Added section on how to use the analysis widget and interpreting the visualizations.

Fixes:

• Removed UserWarning warnings on some example Jupyter notebooks
• Updated development harness to fix errors due to training API updates

The latest PyPi package is here: https://pypi.org/project/backwardcompatibilityml/1.3.0/
The latest documentation is here: https://backwardcompatibilityml.readthedocs.io/en/v1.3.0/

v1.2.0

This is a significant release as it includes all widget components as envisioned by the original project concept and a new development environment for running the widget outside of Jupyter notebooks. Also, this release incorporates some enhancements to existing widget components, new widget components interactivity and a few fixes.

Please upgrade to the latest version, and let us know what you think.

New widget components released:

• Error instances table:
  • Supporting both tabular and image data display.
  • Simple API for users to display selected data about dataset.

New development harness:

• Created React app that can be run independently of Jupyter notebooks, using node and flask, for faster development turnaround.
• Published an initial automated testing framework using TS-Jest and React Testing Library for increasing testing effectiveness and coverage.

Enhancements to existing widget components:

• Raw values component:
  • Enhanced labelling for easier model metrics comparison.
• Histogram:
  • Added pagination for datasets with more than 5 classes.
  • Improved UX by highlighting column on click.
  • Sorting is now by class name ascending.
• Venn diagram:
  • Added tool-tip with error instance number and percents.
  • Enabled filtering by intersection of error instances.
  • Improved UX by highlighting Venn sections on click.
  • Percent displayed with respect to all instances in dataset.

New widget components interactivity:

• Filtering of error instances table by clicking on scatter plots, Venn diagram or histogram.

Fixes:

• Fixed Venn diagram crash for extreme cases (total intersection and fully disjoint sets.)
• Improved reproducibility by fixing random seeds in examples.
• Histogram now shows distribution of incompatible points (of h2 with respect to h1, per class) instead of misclassifications of h2 per class.

Known issues:

• Please visit Issues for a list of known issues.

The latest PyPi package is here: https://pypi.org/project/backwardcompatibilityml/1.2.0/

The latest documentation is here: https://backwardcompatibilityml.readthedocs.io/en/v1.2.0/

v1.1.0

This release includes some enhancements to existing widget components, new widget components and new widget components interactivity.

Please upgrade to the latest version, and let us know what you think.

Enhancements to existing widget components:

• Scatter plots:
  • Added performance of h1 as dotted horizontal line
  • Added color legends
• Venn diagram:
  • Added error instance fraction tool-tips
  • Added color legends
  • Made regions' sizes relative

New widget components released:

• Implemented data set and dissonance filtering drop-downs

New widget components interactivity:

• Scatter plots:
  • Added lambda value tool-tips
  • Update Venn diagram and Histogram on data point click

The latest PyPi package is here: https://pypi.org/project/backwardcompatibilityml/1.1.0/

The latest documentation is here: https://backwardcompatibilityml.readthedocs.io/en/v1.1.0/

v1.0.0

We are happy to announce that we have released our open source v1.0.0 to the public.

The public repository is available here: https://github.com/microsoft/BackwardCompatibilityML/

Full documentation is available here: https://backwardcompatibilityml.readthedocs.io/en/latest/

Getting Started: https://backwardcompatibilityml.readthedocs.io/en/latest/backwardcompatibilityml/topics/getting\_started.html

To install backwardcompatibilityml, please run:

On Linux:

pip install backwardcompatibilityml

On windows:

pip install backwardcompatibilityml -f https://download.pytorch.org/whl/torch\_stable.html

Check it out and let us know what you think.