Skip to content

Commit

Permalink
Update doc according to review
Browse files Browse the repository at this point in the history
  • Loading branch information
benlansdell committed Nov 6, 2023
1 parent e7c2850 commit 0a9effb
Show file tree
Hide file tree
Showing 3 changed files with 14 additions and 4 deletions.
10 changes: 9 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ This library interprets pose-tracking files and behavior annotations to create f

At present pose tracking data from DLC, SLEAP and NWB formats are supported, and behavior annotations from BORIS and NWB formats are supported.

Full documentation is posted here: [https://benlansdell.github.io/ethome/](https://benlansdell.github.io/ethome/).

## Features

* Read in animal pose data and corresponding behavior annotations to make supervised learning easy
Expand All @@ -26,6 +28,8 @@ At present pose tracking data from DLC, SLEAP and NWB formats are supported, and
pip install ethome-ml
```

`ethome` has been tested with Python 3.7 and 3.8.

## Quickstart

It's easiest to start with an NWB file, which has metadata already connected to the pose data.
Expand Down Expand Up @@ -60,7 +64,7 @@ There are featuresets specifically tailored for social mice studies (resident in
```python
dataset.features.add('cnn1d_prob')
```
Uses a pretrained CNN to output probabilities of 3 behaviors (attack, mount, social investigation). For this, you must have labeled your body parts in a certain way (refer to How To). Other, more generic, feature creation functions are provided that work for any animal configuration.
Uses a pretrained CNN to output probabilities of 3 behaviors (attack, mount, social investigation). For this, you must have labeled your body parts in a certain way (refer to [How To](https://benlansdell.github.io/ethome/how-to/)). Other, more generic, feature creation functions are provided that work for any animal configuration.

Now you can access a features table, labels, and groups for learning with `dataset.ml.features, dataset.ml.labels, dataset.ml.group`. From here it's easy to use some ML libraries to train a behavior classifier. For example:
```python
Expand Down Expand Up @@ -94,3 +98,7 @@ dataset.io.save_movie(['label', 'prediction'], '.')
where `label` and `prediction` reference column names to annotate the video with.

A more detailed run through of features is provided in the How To guide. Also checkout `examples` for working demos to quickly see how things work.

### Contributing

Refer to `CONTRIBUTING.md` for guidelines on how to contribute to the project, and report bugs, etc.
6 changes: 4 additions & 2 deletions docs/how-to.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,15 +137,17 @@ This will compute and add the distances between all body parts of all animals.

### 3a In-built support for resident-intruder setup

First, if your setup is a social mouse study, involving two mice, similar enough to the standard resident-intruder setup, then you can use some pre-designed feature sets. The body parts that are tracked must be those from the MARS dataset [1] (See Figure). You will have to have labeled and tracked your mice in DLC/SLEAP in the same way. (with the same animal and body part names. Though `ethome`'s `create_dataset` function can help you rename them appropriately)
First, if your setup is a social mouse study, involving two mice, similar enough to the standard resident-intruder setup, then you can use some pre-designed feature sets. The body parts that are tracked must be those from the MARS dataset [1] (See Figure). The MARS dataset is the Mouse Action Recognition System dataset, published by Segalin et al. It is a large, annotated mouse pose dataset that is useful for building ML models of mouse behavior. `ethome` uses pretrained models and feature sets that have been demonstrated to perform well at mouse action recognition on this dataset -- a combination of these models and features placed 3rd in AICrowd's [Multiagent behavior contest](https://www.aicrowd.com/challenges/multi-agent-behavior-representation-modeling-measurement-and-applications) in 2021.

You will have to have labeled and tracked your mice in DLC/SLEAP in the same way. (with the same animal and body part names. Though `ethome`'s `create_dataset` function can help you rename them appropriately)

![Resident-intruder keypoints](assets/mars_keypoints.png)

Specifically, the animals *must* be named `resident` and `intruder`, and the body parts must be: `nose`, `leftear`, `rightear`, `neck`, `lefthip`, `righthip`, and `tail`.

The `cnn1d_prob`, `mars`, `mars_reduced` and `social` functions can be used to make features for this setup.

* `cnn1d_prob` runs a 1D CNN and outputs prediction probabilities of three behaviors (attack, mount, and investigation). Even if you're not interested in these exact behaviors, they may still be useful for predicting the occurance of other behaviors, as part of an ensemble model.
* `cnn1d_prob` runs a 1D CNN and outputs prediction probabilities of three behaviors (attack, mount, and investigation). Even if you're not interested in these exact behaviors, they may still be useful for predicting the occurance of other behaviors, as part of an ensemble model. The details of this pretrained model are found here: [https://github.com/benlansdell/mabetask1_ml](https://github.com/benlansdell/mabetask1_ml).
* `mars` computes a long list of features as used in the MARS paper. You can refer to that paper for more details.
* `mars_reduced` is a reduced version of the MARS features
* `social` is a set of features that only involve measures of one animal in relation to the other.
Expand Down
2 changes: 1 addition & 1 deletion ethome/features/features.py
Original file line number Diff line number Diff line change
Expand Up @@ -55,7 +55,7 @@ def transform(self, edf, **kwargs):
#Validate columns:
checks = [col in self.required_columns for col in edf.columns]
if sum(checks) < len(self.required_columns):
raise RuntimeError("DataFrame doesn't have necessary columns to compute this set of features.")
raise RuntimeError(f"DataFrame doesn't have necessary columns to compute this set of features. Requires columns: {self.required_columns}")
if edf[self.required_columns].isnull().values.any():
warnings.warn("Missing values in required data columns. May result in unexpected behavior. Consider interpolating or imputing missing data first.")
new_features = self.feature_maker(edf, self.required_columns, **self.kwargs, **kwargs)
Expand Down

0 comments on commit 0a9effb

Please sign in to comment.