Join forces with ReservoirComputing.jl?

[MartinuzziFrancesco]

Hi there! I am restructuring ReservoirComputing.jl to be more SciML friendly, and I see that there could be some common ground for collaborating here. If you want we can discuss a road forward for reservoir computing models and packages, to either not go too much in parallel with our works or work together!

[bertini97]

Hi @MartinuzziFrancesco, yes, I would very much like that. I created this repo because ReservoirComputing.jl looked like a finished project and I needed some features quickly. It was never intended to compete. Right now I'm in the middle of a full rewrite which to port it to Lux.jl for layers and parameters, and using a continuous time solve to evolve it.

It's nice to see that it's getting worked on. Here is my wishlist for a new version, and I'd be happy to contribute something if there is consensus:

• Usage of Lux.jl for layers.
• Integration with other SciML constructs: input/output in the form of a (interpolatable) time series, much like ODESolution, (VectorOfArray, DiffEqArray, etc)
• Using a general surrogate as an output layer, not only linear.
• Possibility of going continuous time, using OrdinaryDiffEq's solve. This makes the RNN a NeuralCDE.

Let me know what you think. Should I open an issue on your repo?

[bertini97]

I took a look at SciML/ReservoirComputing.jl#206 and it looks very similar to my implementation. Great!

[MartinuzziFrancesco]

I created this repo because ReservoirComputing.jl looked like a finished project and I needed some features quickly.

yeh sorry I have not been too active lately, but I do have some plans for improvements in the coming weeks.

I took a look at SciML/ReservoirComputing.jl#206 and it looks very similar to my implementation. Great!

I wanted exactly to link that! I have continued working on it locally, but I still have not pulled the trigger for the refactor since I need to maintain the current features of the library. My vision is to have something like

esn = ReservoirChain(
    Recurrent(ESNCell(in_dims => res_dims))
    NLAT1(),
    Collect(),
    Readout(res_dims => out_dims)
)

at the high level, where Collect() is a layer only needed to give a flag to collect_states(esn, input_data) in order to collect the internal states. This should allow compose much more complex models like DeepESNs etcetc.
If this all falls into place I think that

• Usage of Lux.jl for layers.

and

• Using a general surrogate as an output layer, not only linear.

should be taken care of by default! I will be using LuxCore and not Lux proper since I would like to maintain lighter dependencies since at the end I don't need 90% of Lux features. If you'd like we can discuss how to proceed with the ReservoirChain approach on the issue you linked already. If you had other approach ideas I'm happy to discuss them!

I like both the other points, if you want to track some progress on that you can open one issue for each on ReservoirComputing.jl and we can discuss possible implementations there!