Registration of Two Lineages #10

maarzt · 2023-01-20T13:25:58Z

Add a plugin that sorts the TrackScheme of one Mastodon project to match the order in another project.

Sorting a TrackScheme here is understood as sorting the order of child nodes for every dividing spot in the TrackScheme.

The algorithm implemented here does this by calculating the division directions for each cell division in both embryos. The division direction are compared between the two embryos in order to sort the child nodes of every dividing spot in the same way.

Requirement for the algorithm:

Stereotypical development
Root nodes need to be named in both embryos, and the names need to match pairwise between the embryos.

Steps of the algorithm:

Match the root nodes in both embryos based on their name.
A transformation is calculated that maps the coordinates of one embryo to the other. The coordinates of the matched nodes are used a key points to calculated the transform. The transform allows scale, rotation and translation but no sheering. (This keeps angles in both coordinate systems comparable)
Pick a pair of root nodes (a, b)
Calculate the division directions of both root nodes a and b: d_a, d_b. This division directions are vectors.
The transformation (form step 2 to) is applied to d_a: T(d_a)
The angle between the two division directions is calculated alpha = angle(T(d_a), d_b)
The child nodes of root node b are flipped if the angle is larger than 90 degree.
Recursively repeat step 4 to 8 for the root nodes of the subtrees of a and b.

TODOs

Acceptance criteria

Open two of Mette's embryos (A) in Mastodon
In the Mastodon menu select "Plugins > Tree Management > Lineage Registration" to show the dialog for Lineage Registration.
Make sure that the dialog allows performing the following operations:
- tag unmatched and flipped cells / branches
- copy a tag set from one project to the other
- sort the trackscheme in one project to match the order of descendants in the other project
- color lineages -> this creates a new tag set in both projects, that have one tag per lineage
- Activating one of the lock symbols synchronizes, focused (branch) spot & highlighted (branch) spot in both projects.

xulman

Hi,
I have only a couple of minor comments that I believe could improve the readability of the code, plus a few things that might be worth double-checking .. all remarks are given in the comments to the code.

The algorithm itself is I think reasonable given its expected setting, and good that it echos the expected settings in the dialog.

I have to honestly admit that I haven't paid too much attention to the tests... and haven't yet found time to compile and test it.

src/main/java/org/mastodon/mamut/tomancak/lineage_registration/LineageTreeUtils.java

src/main/java/org/mastodon/mamut/tomancak/lineage_registration/RootsPairing.java

src/main/java/org/mastodon/mamut/tomancak/lineage_registration/LineageRegistrationPlugin.java

src/main/java/org/mastodon/mamut/tomancak/lineage_registration/EstimateTransformation.java

...main/java/org/mastodon/mamut/tomancak/lineage_registration/LineageRegistrationAlgorithm.java

src/main/java/org/mastodon/mamut/tomancak/lineage_registration/LineageTreeUtils.java

stefanhahmann

This pull request adds some nice functionality to register lineage trees of two projects with each other.

If the matching is successful the re-sorting of the lineage tree can be seen quite nicely in the TrackViews:

src/main/java/org/mastodon/mamut/tomancak/lineage_registration/LineageRegistrationDialog.java

src/main/java/org/mastodon/mamut/tomancak/lineage_registration/RootsPairing.java

src/test/java/org/mastodon/mamut/tomancak/lineage_registration/Glasbey.java

.../java/org/mastodon/mamut/tomancak/lineage_registration/LineageRegistrationAlgorithmTest.java

* An transformation between the two dataset is estimated from the labeled spots in the first time point of the two datasets * The tree is sorted such that the spindle direction matches between the two datasets. Scalar product > 0.

The EstimateTransform method uses code that was probably implemented as part of the multi-view reconstruction plugin for Fiji. But we furtunately don't need the full multi-view reconstruction dependency.

This is a preparation step. There is a significant difference between the LineageRegistrationPlugin and other plugins in this repo. The lineage registration plugin will operate on multiple open Mastodon projects. It therefor needs some specialized MastodonPlugin.

The updated dialog supports a wider range of operations.

…ppModels

… projectes

…BranchEnd

stefanhahmann · 2023-03-02T15:20:28Z

(fixed) It would be good, if the Lineage Registration Dialog would always come to front, when "Plugins" > "Lineage Registration" is selected. Otherwise, the user may have forgotten that the dialog is already open and just not visible anymore, because other Windows are in front of it. Consider e.g. the following workflow:

stefanhahmann · 2023-03-02T15:29:00Z

(fixed) It would be good, if the Lineage Registration Dialog disappears, if the last Mastodon related Window has been closed. The dialog is then not needed anymore and the user would need to close it manually.

However, an argument against this may be that all other Mastodon related windows also have to be closed one by one.

But all other windows that have to be closed manually actually appear in the task manager:

If the dialog remains to be closed manually, the latter should be consistent. Also, if the dialog would appear in the task manager, it would be easier to bring to the front again, if disappears under other windows.

stefanhahmann · 2023-03-03T08:26:23Z

The lineage registration dialog shows Mastodon projects that are not open anymore:

Before this screenshot, I did the following:

Open Project "Mastodon" (a folder project)
Open Project "e2-complete-3" (a file project)
Open the Lineage Registration Dialog
Close the Lineage Registration Dialog
Close Project "Mastodon"
Close Project "e2-complete-3"
Open Project "e1-Mastodon"
Open Project "e2-complete-3"
Open the Lineage Registration Dialog again

Result: four project are shown in selection dialog, while only two projects are open.

The function "Update list of projects" does not seem to change anything in list of available projects, even though the name of that function would indicate that.

Also the lists of tag sets in the "copy tag set function" does not seem to be correctly updated (shows tag sets of projects that are not open anymore).

stefanhahmann · 2023-03-03T08:52:23Z

This tooltip disappears rather quickly, at least more quickly than I was able to read / understand it:

Consider making it appear a bit longer. The same also applies to other tooltips in this dialog (some of them are even longer and require even more time to read)

stefanhahmann · 2023-03-03T08:58:11Z

Would it be possible to automatically, regenerate the branch graph after sorting the TrackScheme via the Lineage Registration dialog.

At the moment, if you are not an experienced user, it may be easily overseen that the icon to regenerate the branch graph in the sorted project changed and that only after pressing this button the actual sorting visually happens.

stefanhahmann · 2023-03-03T09:07:47Z

(fixed) In this tooltip the user learns, how the tag sets are not created (i.e. not making use of the correspondences). It would at least be as interesting how the tag sets are then actually created (I assume using the root cell names?)

stefanhahmann · 2023-03-03T09:23:52Z

One may consider naming the buttons "project A", "project B" the same as the selected project.

Probably, this could go into a new Pull Request / Issue

stefanhahmann · 2023-03-03T09:51:48Z

I am not sure, if the function "Tag unmatched & flipped cells" work as expected:

After pressing all the buttons, I could not find any new tag set in both projects.

Also the tooltips should not be all the same for all three of the buttons.

…uested

Microsoft Windows treats JDialog and JFrame differently. A JDialog does not show up in the task manager. Extending JFrame therefore improves usability. There are also other reasons for this change. The LineageRegistrationDialog does not behave like a dialog: It is not modal. There are no Ok or Cancel buttons. There's only a close button. It's meant to stay open for a longer duration, while interacting with other windows. It therefor behaves more like a JFrame.

maarzt · 2023-03-07T14:31:29Z

The lineage registration dialog shows Mastodon projects that are not open anymore:

Well spotted. I'm aware of this problem. But I don't know no way for this plugin to get informed when the Mastodon project is closed. The plugin API has no unregister method MastodonPlugin. I tried to use WeakReferences but this didn't work. Mastodon projects are never garbage collected (on linux). So there is also garbage collection bug in Mastodon.

The function "Update list of projects" does not seem to change anything in list of available projects, even though the name of that function would indicate that.

The button adds newly opened projects to the list. It doesn't remove the already existing entries. I would like to get rid of this button but it's required for stupid reasons. When initializing a plugin the project path is not yet set, so it's not possible to update this list, as the project name cannot yet be deduced. This means I cannot reliably update this list when a new mastodon project is opened. This button is a workaround for that problem.

Also the lists of tag sets in the "copy tag set function" does not seem to be correctly updated (shows tag sets of projects that are not open anymore).

I cannot reproduce this problem. The list behave correctly as far as I can tell. Please provide a more detailed problem description.
(Also the Mastodon garbage collection bug means that projects are basically never closed. There is nothing like a Mastodon project being "not open".)

maarzt · 2023-03-07T14:42:52Z

This tooltip disappears rather quickly, at least more quickly than I was able to read / understand it:

There are only hacky ways in java swing to show a tooltip longer.

Please keep in mind that there are currently only two intended users for this plugin. Too much time invested here is time wasted!

maarzt · 2023-03-07T14:52:50Z

One may consider naming the buttons "project A", "project B" the same as the selected project.

I also had this idea. But no, we shouldn't do this. I learned from some provided datasets that projects name tend to become very long. So having only the short letters is best for readability.

maarzt · 2023-03-07T15:01:57Z

I am not sure, if the function "Tag unmatched & flipped cells" work as expected:

After pressing all the buttons, I could not find any new tag set in both projects.

Also the tooltips should not be all the same for all three of the buttons.

This should work. The projects should get a tag set called "lineage registration" added. Could you verify if this problem persists. Please tell me how to reproduce the problem. For me the functionality works as expected.

maarzt · 2023-03-07T15:14:36Z

@stefanhahmann & @xulman when reviewing this, please keep in mind that I currently only expect two users for this plugin: Johannes and Mette. Tell me if you see significant problems with the functionality or code. There is no need to polish all the little details. I want to make this algorithm available for now. Let's see if the algorithm works at all, it probably still need a lot off improvement and the UI will need to change with it. This plugin needs to be considered a prototype!

This drastically improves the feedback to the user if the sorting the TrackScheme is used together with the "TrackScheme Branch" or "TrackScheme Hierarchy" window. As the user no longer needs to klick. The tiny "regen." button.

stefanhahmann · 2023-03-08T15:04:00Z

One may consider naming the buttons "project A", "project B" the same as the selected project.

I also had this idea. But no, we shouldn't do this. I learned from some provided datasets that projects name tend to become very long. So having only the short letters is best for readability.

I agree. Hadn't thought of long filenames.

This tooltip disappears rather quickly, at least more quickly than I was able to read / understand it:

There are only hacky ways in java swing to show a tooltip longer.

Please keep in mind that there are currently only two intended users for this plugin. Too much time invested here is time wasted!

I agree. Thanks for clarification.

The lineage registration dialog shows Mastodon projects that are not open anymore:

Well spotted. I'm aware of this problem. But I don't know no way for this plugin to get informed when the Mastodon project is closed. The plugin API has no unregister method MastodonPlugin. I tried to use WeakReferences but this didn't work. Mastodon projects are never garbage collected (on linux). So there is also garbage collection bug in Mastodon.

I did not know this. Thanks for making that clear.

The function "Update list of projects" does not seem to change anything in list of available projects, even though the name of that function would indicate that.

The button adds newly opened projects to the list. It doesn't remove the already existing entries. I would like to get rid of this button but it's required for stupid reasons. When initializing a plugin the project path is not yet set, so it's not possible to update this list, as the project name cannot yet be deduced. This means I cannot reliably update this list when a new mastodon project is opened. This button is a workaround for that problem.

That was unclear to me. Thanks for making that clear.

Also the lists of tag sets in the "copy tag set function" does not seem to be correctly updated (shows tag sets of projects that are not open anymore).

I cannot reproduce this problem. The list behave correctly as far as I can tell. Please provide a more detailed problem description. (Also the Mastodon garbage collection bug means that projects are basically never closed. There is nothing like a Mastodon project being "not open".)

I am not sure, if the function "Tag unmatched & flipped cells" work as expected:

After pressing all the buttons, I could not find any new tag set in both projects.
Also the tooltips should not be all the same for all three of the buttons.

This should work. The projects should get a tag set called "lineage registration" added. Could you verify if this problem persists. Please tell me how to reproduce the problem. For me the functionality works as expected.

It works as intended. It did not understand that expected result are 2 tag sets, but this is actually said in the tooltip.

stefanhahmann

All the acceptance criteria seem to work now. This is very good.
Thanks also for the UI improvements. Especially that the Dialog is now a JFrame is an usability improvement.

There are some minor comments leave. Can you please have a look on them?

The coupling mechanism may deserve a more detailed code review. I did only a coarse one for the sake of the getting this to the user.

src/main/java/org/mastodon/mamut/tomancak/lineage_registration/LineageTreeUtils.java

src/main/java/org/mastodon/mamut/tomancak/lineage_registration/RootsPairing.java

pom.xml

src/main/java/org/mastodon/mamut/tomancak/lineage_registration/LineageTreeUtils.java

src/main/java/org/mastodon/mamut/tomancak/lineage_registration/RefMapUtils.java

src/main/java/org/mastodon/mamut/tomancak/lineage_registration/RootsPairing.java

Previously those cells that had differently sorted child cells where tagged. This commit changes the code such that the child cells themselves get tagged. Which is hopefully more intuitively to understand. Incoming edges are tagged too. The order of the buttons in the LineageRegistrationFrame is changed. Tagging cells is usually the first action a user would want the use. The buttons for tagging cells are therefor now in the top row.

maarzt marked this pull request as draft January 20, 2023 13:26

maarzt requested review from tinevez, xulman and stefanhahmann and removed request for tinevez January 25, 2023 08:57

xulman reviewed Jan 31, 2023

View reviewed changes

stefanhahmann reviewed Feb 1, 2023

View reviewed changes

maarzt added 3 commits February 10, 2023 16:08

Add coding style

299fe30

LineageRegistrationAlgorithm: add getMapping() method

99974b7

stefanhahmann mentioned this pull request Feb 13, 2023

Plugin to Test Model Graph Consistency #12

Open

maarzt added 9 commits February 14, 2023 11:23

Refactor EstimateTransform & reduce dependencies

96e16b2

The EstimateTransform method uses code that was probably implemented as part of the multi-view reconstruction plugin for Fiji. But we furtunately don't need the full multi-view reconstruction dependency.

Rename RootsPairing.pairRoots

f0b4149

Improve and fix LineageRegistrationDialog

9258efc

Change LineageTreeUtils.getBranchEnd to not allocation memory

ca09963

Change test methods to meet reviewers comments

a50b0a6

Add and improve javadoc

2ad057e

LineageRegistrationDialog: allow selection of two open Mastodon projects

2a9ae00

The updated dialog supports a wider range of operations.

Add functionality for the coloring of lineages

14d654b

maarzt force-pushed the lineage-registration branch from a040706 to 14d654b Compare February 15, 2023 15:28

maarzt added 9 commits February 22, 2023 13:22

Extract method to sort TrackScheme from LineageRegistrationAlgorithm

310b881

Add buttons for tagging unmatched and flipped branches

bb00599

Implement tagging of unmatched and flipped branches

2e7be5f

Allow to copy tag set from one mastodon project to a registered project

742c109

Add code that allows to couple the focused spot etc. of two MastodonA…

f660888

…ppModels

Add buttons to the LineageRegistrationDialog that allow to couple the…

d321f80

… projectes

LineageRegistrationAlgorithm: make run method public

a844f5b

Introduce class TagSetUtils

0b59966

Remove LineageTreeUtils.getBranchEnd in favor of BranchGraphUtils.get…

1dc9b20

…BranchEnd

maarzt force-pushed the lineage-registration branch from 13121eb to 7fb432d Compare February 28, 2023 13:22

maarzt requested review from stefanhahmann and xulman February 28, 2023 13:25

stefanhahmann closed this Mar 2, 2023

stefanhahmann reopened this Mar 2, 2023

maarzt added 2 commits March 7, 2023 14:54

LineageRegistrationControlService: bring dialog to the front when req…

6cabc68

…uested

LineageRegistrationFrame: Improve tool tip text

04dd5ce

stefanhahmann reviewed Mar 8, 2023

View reviewed changes

maarzt added 3 commits March 12, 2023 21:32

LineageResgistration: add a log window to give the user more feedback

4dcd974

RefMapUtils: update TODO comment

38d6faa

maarzt marked this pull request as ready for review March 13, 2023 13:32

maarzt merged commit 7b7d154 into master Mar 13, 2023

maarzt deleted the lineage-registration branch August 1, 2023 08:35

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Registration of Two Lineages #10

Registration of Two Lineages #10

maarzt commented Jan 20, 2023 •

edited

Loading

xulman left a comment

stefanhahmann left a comment

stefanhahmann commented Mar 2, 2023 •

edited by maarzt

Loading

stefanhahmann commented Mar 2, 2023 •

edited by maarzt

Loading

stefanhahmann commented Mar 3, 2023 •

edited

Loading

stefanhahmann commented Mar 3, 2023 •

edited

Loading

stefanhahmann commented Mar 3, 2023

stefanhahmann commented Mar 3, 2023 •

edited by maarzt

Loading

stefanhahmann commented Mar 3, 2023

stefanhahmann commented Mar 3, 2023

maarzt commented Mar 7, 2023 •

edited

Loading

maarzt commented Mar 7, 2023

maarzt commented Mar 7, 2023

maarzt commented Mar 7, 2023

maarzt commented Mar 7, 2023

stefanhahmann commented Mar 8, 2023

stefanhahmann left a comment

Registration of Two Lineages #10

Registration of Two Lineages #10

Conversation

maarzt commented Jan 20, 2023 • edited Loading

TODOs

Acceptance criteria

xulman left a comment

Choose a reason for hiding this comment

stefanhahmann left a comment

Choose a reason for hiding this comment

stefanhahmann commented Mar 2, 2023 • edited by maarzt Loading

stefanhahmann commented Mar 2, 2023 • edited by maarzt Loading

stefanhahmann commented Mar 3, 2023 • edited Loading

stefanhahmann commented Mar 3, 2023 • edited Loading

stefanhahmann commented Mar 3, 2023

stefanhahmann commented Mar 3, 2023 • edited by maarzt Loading

stefanhahmann commented Mar 3, 2023

stefanhahmann commented Mar 3, 2023

maarzt commented Mar 7, 2023 • edited Loading

maarzt commented Mar 7, 2023

maarzt commented Mar 7, 2023

maarzt commented Mar 7, 2023

maarzt commented Mar 7, 2023

stefanhahmann commented Mar 8, 2023

stefanhahmann left a comment

Choose a reason for hiding this comment

maarzt commented Jan 20, 2023 •

edited

Loading

stefanhahmann commented Mar 2, 2023 •

edited by maarzt

Loading

stefanhahmann commented Mar 2, 2023 •

edited by maarzt

Loading

stefanhahmann commented Mar 3, 2023 •

edited

Loading

stefanhahmann commented Mar 3, 2023 •

edited

Loading

stefanhahmann commented Mar 3, 2023 •

edited by maarzt

Loading

maarzt commented Mar 7, 2023 •

edited

Loading