Rsdk 9697 support multiple nvs #376

npmenard · 2025-01-08T20:51:10Z

Implements the storage traits for any type we can iterate over, we can then easily pass an array of a vec as a storage to viam server.
With this something like
let storage = vec![ NVSStorage::new("nvs").unwrap(), NVSStorage::new("nvs2").unwrap(), ];
Will allow use to store the robot data over multiple NVS

I didn't want to add an extra type to represent a collection of storage and this way the logic is driven by the underlying storage implementation.

npmenard · 2025-01-08T20:54:31Z

micro-rdk/src/esp32/nvs_storage.rs

 }

 #[derive(Clone)]
 pub struct NVSStorage {
    // esp-idf-svc partition driver ensures that only one handle of a type can be created
    // so inner mutability can be achieves safely with RefCell
    nvs: Rc<RefCell<EspCustomNvs>>,
-    partition_name: CString,


store a copy of the partition rather than the partition name to avoid cloning a CString as we pass the Storage through different par to of the code

npmenard · 2025-01-08T20:59:28Z

micro-rdk/src/common/credentials_storage.rs

@@ -102,15 +103,15 @@ impl From<CertificateResponse> for TlsCertificate {
 pub trait WifiCredentialStorage {
    type Error: Error + Debug + Into<ServerError>;
    fn has_wifi_credentials(&self) -> bool;
-    fn store_wifi_credentials(&self, creds: WifiCredentials) -> Result<(), Self::Error>;
+    fn store_wifi_credentials(&self, creds: &WifiCredentials) -> Result<(), Self::Error>;


I changed some functions of the storage traits to accept reference instead of taking ownership. Otherwise we would have to clone each time until we can find a storage where to store the object . It's not dramatic so happy to revert that change if we feel like it doesn't make sense

I think I like it better this way actually.

npmenard · 2025-01-08T20:59:53Z

micro-rdk/src/common/credentials_storage.rs

    fn reset_ota_metadata(&self) -> Result<(), Self::Error>;
 }

 pub trait StorageDiagnostic {
    fn log_space_diagnostic(&self);
 }

+#[derive(Error, Debug)]
+#[error("empty storage collection")]
+pub struct EmptyStorageCollectionError;


this is a special error to handle the case where a collection is empty

npmenard · 2025-01-08T21:00:27Z

micro-rdk/src/common/credentials_storage.rs

@@ -158,6 +163,22 @@ struct RAMCredentialStorageInner {
 #[derive(Default, Clone)]
 pub struct RAMStorage(Rc<Mutex<RAMCredentialStorageInner>>);

+#[derive(Error, Debug)]
+pub enum RAMStorageError {


add an error type for RAMStorage to make it easier to test

npmenard · 2025-01-08T21:03:27Z

micro-rdk/src/common/credentials_storage.rs

+        self.into_iter().any(OtaMetadataStorage::has_ota_metadata)
+    }
+    fn get_ota_metadata(&self) -> Result<OtaMetadata, Self::Error> {
+        self.into_iter().fold(


the idea here is to use fold to call get_ota_metadata until Ok is returned. If all storage in a collection return an error only the last error is reported.
We don't have an iterator that behave like try_fold but actually short-circuit on the first Ok, so we will iterate through the whole collection even after a get_ota_metadata has succeeded. The function get_ota_metadata will however not be called anymore on storage

npmenard · 2025-01-08T21:04:47Z

micro-rdk/src/common/credentials_storage.rs

+{
+    type Error = Storage::Error;
+    fn has_ota_metadata(&self) -> bool {
+        self.into_iter().any(OtaMetadataStorage::has_ota_metadata)


short-circuit on the first true

npmenard · 2025-01-08T21:06:11Z

micro-rdk/src/common/credentials_storage.rs

+#[cfg(feature = "ota")]
+impl<Iterable, Storage: OtaMetadataStorage> OtaMetadataStorage for Iterable
+where
+    for<'a> &'a Iterable: IntoIterator<Item = &'a Storage>,


will only work on collections that implements impl<'a, T,> IntoIterator for &'a ...

npmenard · 2025-01-08T21:07:08Z

micro-rdk/src/esp32/nvs_storage.rs

@@ -136,7 +137,7 @@ impl StorageDiagnostic for NVSStorage {
    fn log_space_diagnostic(&self) {
        let mut stats: nvs_stats_t = Default::default();
        if let Err(err) =
-            esp!(unsafe { nvs_get_stats(self.partition_name.as_ptr(), &mut stats as *mut _) })
+            esp!(unsafe { nvs_get_stats(self.part.handle() as _, &mut stats as *mut _) })


handle is the partition name

npmenard · 2025-01-08T21:24:59Z

micro-rdk/src/common/credentials_storage.rs

+    fn reset_ota_metadata(&self) -> Result<(), Self::Error> {
+        self.into_iter().fold(
+            Err::<_, Self::Error>(EmptyStorageCollectionError.into()),
+            |val, s| val.or(s.reset_ota_metadata()),


taking advantage of the eagerness of or might not be the best approach

Can you elaborate a bit? I think I know what you are saying, but I'd rather not guess.

As opposed to or_else or will always evaluate a function in argument if there is one (because it doesn't actually accept a closure) before considering replacing val hence the eagerness. We use it our advantage to make sure all copies of an object are erased through all storage. In theory we shouldn't need this since we use or_else elsewhere so only on copy should be in the storage array at anytime.

acmorrow

This is neat, LGTM, a few small comments. I'd like to think about it a bit tomorrow before it merges so I can make sure I totally understand how the various folds are working and think about whether there are any tricky edge cases.

acmorrow · 2025-01-08T21:40:30Z

micro-rdk/src/common/credentials_storage.rs

@@ -102,15 +103,15 @@ impl From<CertificateResponse> for TlsCertificate {
 pub trait WifiCredentialStorage {
    type Error: Error + Debug + Into<ServerError>;
    fn has_wifi_credentials(&self) -> bool;
-    fn store_wifi_credentials(&self, creds: WifiCredentials) -> Result<(), Self::Error>;
+    fn store_wifi_credentials(&self, creds: &WifiCredentials) -> Result<(), Self::Error>;


I think I like it better this way actually.

acmorrow · 2025-01-08T22:18:26Z

micro-rdk/src/common/credentials_storage.rs

        let mut inner_ref = self.0.lock().unwrap();
        let _ = inner_ref.robot_creds.insert(creds);
        Ok(())
    }
    fn get_robot_credentials(&self) -> Result<RobotCredentials, Self::Error> {
        let inner_ref = self.0.lock().unwrap();
-        let cfg = inner_ref.robot_creds.clone().unwrap_or_default().clone();
-        Ok(cfg)
+        log::info!("get robot creds");


Stray log, or debug!?

As an aside, I do miss some of the logging we used to have pre #316 that gave a little more context about progression through the startup sequence, whether we were using cached credentials or configs, etc. I'm usually for "silence is golden" but in this case that's very useful information to understand how things are coming up, especially in offline mode. Would you be OK with a ticket to re-introduce some of that logging? Or did you not propagate into the refactored world because you found it too chatty?

stray log i will remove it.
Yes let's make a ticket for some logging housekeeping

acmorrow · 2025-01-08T22:19:10Z

micro-rdk/src/common/credentials_storage.rs

    }
    fn reset_robot_credentials(&self) -> Result<(), Self::Error> {
        let mut inner_ref = self.0.lock().unwrap();
        let _ = inner_ref.robot_creds.take();
+        log::info!("reset robot creds");


acmorrow · 2025-01-08T22:26:20Z

micro-rdk/src/common/credentials_storage.rs

+        assert!(!v.has_robot_credentials());
+        assert!(ram2
+            .store_robot_credentials(&CloudConfig {
+                app_address: "http://downloadramstorage.org".to_owned(),


maybe .example.org

yes that was for the joke :P

acmorrow · 2025-01-08T22:29:41Z

micro-rdk/src/common/credentials_storage.rs

+    fn reset_ota_metadata(&self) -> Result<(), Self::Error> {
+        self.into_iter().fold(
+            Err::<_, Self::Error>(EmptyStorageCollectionError.into()),
+            |val, s| val.or(s.reset_ota_metadata()),


Can you elaborate a bit? I think I know what you are saying, but I'd rather not guess.

acmorrow · 2025-01-08T22:33:47Z

How do you envision configuring the space of available NVS, for, say, callers of viam_server_start?

npmenard · 2025-01-08T23:33:48Z

How do you envision configuring the space of available NVS, for, say, callers of viam_server_start?

What do you mean?

npmenard added 2 commits January 8, 2025 15:42

iterate over storage

53637d7

pass object to store as reference

312ff5f

npmenard requested a review from a team as a code owner January 8, 2025 20:51

remove partition_name

75d8b32

npmenard commented Jan 8, 2025

View reviewed changes

remove dead code

fd4d2ed

npmenard commented Jan 8, 2025

View reviewed changes

or

7c17538

npmenard commented Jan 8, 2025

View reviewed changes

acmorrow approved these changes Jan 8, 2025

View reviewed changes

remove stray comments

9fc0988

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Rsdk 9697 support multiple nvs #376

Rsdk 9697 support multiple nvs #376

npmenard commented Jan 8, 2025 •

edited

Loading

npmenard Jan 8, 2025

npmenard Jan 8, 2025

acmorrow Jan 8, 2025

npmenard Jan 8, 2025 •

edited

Loading

npmenard Jan 8, 2025

npmenard Jan 8, 2025 •

edited

Loading

npmenard Jan 8, 2025

npmenard Jan 8, 2025 •

edited

Loading

npmenard Jan 8, 2025

npmenard Jan 8, 2025

acmorrow Jan 8, 2025

npmenard Jan 8, 2025 •

edited

Loading

acmorrow left a comment

acmorrow Jan 8, 2025

acmorrow Jan 8, 2025

npmenard Jan 8, 2025

acmorrow Jan 8, 2025

acmorrow Jan 8, 2025

npmenard Jan 8, 2025

acmorrow Jan 8, 2025

acmorrow commented Jan 8, 2025

npmenard commented Jan 8, 2025

Rsdk 9697 support multiple nvs #376

Are you sure you want to change the base?

Rsdk 9697 support multiple nvs #376

Conversation

npmenard commented Jan 8, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

npmenard Jan 8, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

npmenard Jan 8, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

npmenard Jan 8, 2025 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

npmenard Jan 8, 2025 • edited Loading

Choose a reason for hiding this comment

acmorrow left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

acmorrow commented Jan 8, 2025

npmenard commented Jan 8, 2025

npmenard commented Jan 8, 2025 •

edited

Loading

npmenard Jan 8, 2025 •

edited

Loading

npmenard Jan 8, 2025 •

edited

Loading

npmenard Jan 8, 2025 •

edited

Loading

npmenard Jan 8, 2025 •

edited

Loading