You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
All recurrent SOMs would benefit from some kind of batching scheme. I currently have batched code, but it doesn't work. My hunch is that batching the recurrent/recursive/merging SOM is non-trivial because there is no way of making sure the concurrently processed batches converge on the same BMU.
Because we take the mean over batches, this means that updates will generally end up being more noisy as the batch size increases.
Two starting ideas:
A possible solution could be to initialize the context Weights and weights of the SOM to some value which allows for convergence (e.g. the weights are skewed in such a way that the concurrently processed batches all go in the same direction), but this entails setting the parameters in advance, which kind of defeats any purpose the SOM has.
A second solution would be to slowly increase the batch size, allowing the SOM to converge to something stable first, and then increasing batch size.
The text was updated successfully, but these errors were encountered:
All recurrent SOMs would benefit from some kind of batching scheme. I currently have batched code, but it doesn't work. My hunch is that batching the recurrent/recursive/merging SOM is non-trivial because there is no way of making sure the concurrently processed batches converge on the same BMU.
Because we take the mean over batches, this means that updates will generally end up being more noisy as the batch size increases.
Two starting ideas:
A possible solution could be to initialize the context Weights and weights of the SOM to some value which allows for convergence (e.g. the weights are skewed in such a way that the concurrently processed batches all go in the same direction), but this entails setting the parameters in advance, which kind of defeats any purpose the SOM has.
A second solution would be to slowly increase the batch size, allowing the SOM to converge to something stable first, and then increasing batch size.
The text was updated successfully, but these errors were encountered: