AWM Gateway down - missing 2 hours of data

[gauthig]

After noticing missing data due to Emporia's AWS gateway being down for a period. Also seeing missing data from my own shutting down my hypervisor for maintenance, I realized we need a procedure to check the past day for missing data and import it.

I will fork current code and look at adding this optional feature. This may also reduce API usage as normally I would just reload all data if some was missing.

Since it is for historic data, will only look for missing hour and day measurements.

[chase1124]

Hi -- I saw you also opened an issue for this, are you still working on such a fix?

I'm wondering if @jertel has also considered this use case: I only want to import data from emporia as a batch process and the case you are solving here seems more complex but might share functionality (although I personally want per minute when available). The goal to me would be simply to run the container at an infrequent (hourly/daily) schedule or random frequency (synchronously with need to check graphs) and have some kind of checkpointing available (could be local sqllite storage or on the influxdb instance itself) so it knows what time/dates to request from the last successful import and write from emporia. Does this make sense? I imagine it would considerably lower API resources for Emporia but also remove unnecessary requirements from my side. For instance, currently needing to run my server 24/7 to import data most easily when I only check the information once in awhile and the API import and write can be done synchronously to that need. For me, I use the app when I need per second details and otherwise use the power of this tool for historical analysis and to control my own historical data and its precision. I would think that would be a common use if it were supported.

I likely will make these changes myself if its not centrally supported but I'm not a developer and the changes have no hope of making it upstream :) Just wondering if I am the only one that would find such a feature incredibly helpful. Amazing project, many thanks

[jertel]

Hello. It's certainly possible, but I wrote Vuegraf primarily to be able to trigger alerts when something was abnormal. Examples:

• Pool pump is consuming more energy than it should be: pump is probably locked up, unable to spin, need to quickly correct before pool has an algae bloom.
• Sprinkler pump hasn't run in 4 days: timer may have lost its programming or lost power, need to quickly correct before plants die.
• AC is running for 8 hours at a time: A/C is suffering a malfunction and no longer able to cool properly. Correct soon to avoid an expensive electric bill.
• etc

So while your suggestion is possible, it's not something I personally need, since I need very timely alerts. In fact, Vuegraf has already saved me thousands of dollars by notifying me of problems immediately as they occurred which allowed me to prevent damage from occurring.

However, if you have any interest in adding this new capability to the project, so that Vuegraf can run in either mode (long-running, or single run), that would be a great addition for others in your situation.

[chase1124]

That makes sense. It's kind of crazy how limited the emporia web app is considering what you mention would be the next obvious use case to get long-term value out of the system for their users. They could charge a small premium for people that don't want to setup custom solutions like this community does (hopefully not closing the API at the same time).

I'll try to coordinate these changes with you so they can get pulled once I get to the point where I can look at the code and figure out how I would approach it. Thanks for your reply!