reorganize the build of plugins

[tlmquintino]

We should make the inclusion and build of plugins more modular and dynamic.

For example, in terms of CDash we should be using subprojects, see http://testing.sandia.gov/cdash/index.php?project=Trilinos

In terms of git, following an idea of @wdeconinck , we could be using git submodules, which allow for synchronized update of sources.
See http://schacon.github.com/git/git-submodule.html.

[barche]

I like the idea of subprojects for the CDash, much cleaner. It's already a pain to find the tests you want in the huge list we have now, and it would remove the need to put the name of the plugin in the test name.

I'm not so sure about the submodules, that seems like a lot of extra complication for no gain with the plugins we have now. It definitely is the way to go for plugins that need to stay closed-source or for plugins where the author insists that he wants to develop them as a stand-alone project. From the page you linked, especially "you cannot modify the contents of the submodule from within the main project" sounds painful.

I am still in favor of shipping coolfluid with a set of core plugins, which are maintained in the main repository.

[wdeconinck]

I suggest the following:

All plugins get their own git-submodule. git-submodules must always be registered in a parent git repository.
This is nice because it consolidates all coolfluid3 plugins floating around in the open-source world. It ties everybody together. Developers of a plugin should just update the file where the plugins are registered, and send a pull-request.
It also avoids 2 plugins that are coincidentally named the same and have then colliding namespaces.

I foresee this registration to also be in a CMakeList, that then defines variables such as "CF3_PLUGIN_SFDM"
A user can then enable this (CF3_PLUGIN_SFDM=ON), which triggers the fetching of the SFDM plugin as a git submodule, and builds all dependencies it requires. (We can then also include the "CMake external project builds" for third party dependencies on the fly)

[barche]

I don't see how this improves the workflow for the plugins we already have. Once we get "independent" authors working on plugins, we can include their plugins as submodules, but I see no reason to do this for plugins maintained by the core team. It's more work for the users who pull the code, and more work for us to keep things in sync.

[tbanyai]

I think having all modules on a separate git is a lot of marioneting in terms of maintenance.

Independent modules should be maintaned by its owner and this owner can (on a personal basis?) involve a head developer for (not just) keeping a look having things up-to-date to core.

Your actual example of trilinos is misleading because as far as i see into it, the packages a very independent and different. The only common tie is "data structure" they build on (epetra & tpetra).

[wdeconinck]

For the plugins bundled with the kernel now (UFEM,RDM,SDM and others), I would also use git-submodules, but not at this point yet of course. I would do this when the kernel is stable.
Both SDM, RDM, UFEM plugins will grow in size. It will be best that these are managed on their own, with forks on their own for other developers to contribute to, without having to change kernel versions.
But as mentioned, only when the kernel is stabilized.

[tlmquintino]

There are clearly 2 issues here:

• using CMake subprojects
• using Git submodules

I think we should map one CMake subproject to one Git submodule.
However I agree with @wdeconinck that moving those plugins to their own repositories should be done later on, when the kernel is more stabilized.

However, nothing stops us from start using CMake subprojects.

[barche]

OK, I've been experimenting with the subprojects, as you can see at http://coolfluidsrv.vki.ac.be/cdash/index.php?project=coolfluid

I think I have discovered a much simpler method than what is described here http://www.kitware.com/products/html/CDashSubprojects.html by making use of our directory structure, I'll hack some more on it tomorrow night.

As for the git submodules, some things to consider:

• Even trilinos is not using that, they have just one big git repo: https://software.sandia.gov/trilinos/developer/git.html
• We are very much under-using the plugin system, all concrete mesh writers should go to plugins, for example. Do we really want a git repo for each of those?
• How does putting a core plugin (maintained by the cf3 team) in a separate repository make things better?
• looking at other projects, submodules seem to be used mostly to get external dependencies, like VTK for paraview or partio for aqsis for example
• for our plugins, it should be decided on a per-case basis, not a "one size fits all" rule, because clearly that won't be practical