You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The current fast data downloader is fine, but only handles recording channels. They're not the only things that are desired for parallel downloading, especially given @rowleya's megamodel!
There are also:
General Regions (used for weights, provenance, etc.)
IOBUFs
The downloading of general regions requires a data model update. In particular, we have to think of them as being snapshots of the state (instead of streams) and that requires additional information, especially what the current simulation timestep is. (In theory, it also requires the run number, but I'm not sure if that is not implicit in which database we're writing to.)
IOBUF downloading is mostly there, but I don't know if we need to extend what we've got. There's also the question of whether to pull back data into a pile of files (which are then easily examinable with text editors) or to instead put it into a database. I also don't know whether the code has kept pace with @Christian-B's changes to mapping dictionaries.
Feedback/design assistance would be welcome.
The text was updated successfully, but these errors were encountered:
The current fast data downloader is fine, but only handles recording channels. They're not the only things that are desired for parallel downloading, especially given @rowleya's megamodel!
There are also:
The downloading of general regions requires a data model update. In particular, we have to think of them as being snapshots of the state (instead of streams) and that requires additional information, especially what the current simulation timestep is. (In theory, it also requires the run number, but I'm not sure if that is not implicit in which database we're writing to.)
IOBUF downloading is mostly there, but I don't know if we need to extend what we've got. There's also the question of whether to pull back data into a pile of files (which are then easily examinable with text editors) or to instead put it into a database. I also don't know whether the code has kept pace with @Christian-B's changes to mapping dictionaries.
Feedback/design assistance would be welcome.
The text was updated successfully, but these errors were encountered: