Grott Time conversion does not consider daylight savings time #455
Comments
|
Have you tried to change the timezone setting in the grott config? That resolved my issues regarding DST.Op 16 okt 2023 om 00:13 heeft John Ruiz ***@***.***> het volgende geschreven:
Currently, on the east coast of the US, it is 17:40 hours. Current UTC time is 21:40. That's a 4-hour difference. In grott logs, I can see this: - Grott original time : 2023-10-15T17:40:24 adjusted UTC time for influx : 2023-10-15T22:40:24. That's a 5-hour difference. The result is that my influxdb records are timestamped one hour into the future!
Here is the code that tries to get the timezone:
try:
local = pytz.timezone(conf.tmzone)
except :
if conf.verbose :
if conf.tmzone == "local": print("\t - " + "Timezone local specified default timezone used")
else : print("\t - " + "Grott unknown timezone : ",conf.tmzone,", default timezone used")
conf.tmzone = "local"
local = int(time.timezone/3600)
conf.tmzone is currently set to "local". That's not a timezone that pytz understands.
>> pytz.timezone("local")
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/home/jar349/src/grott/.venv/lib/python3.11/site-packages/pytz/__init__.py", line 188, in timezone
raise UnknownTimeZoneError(zone)
pytz.exceptions.UnknownTimeZoneError: 'local'
and so, the exception block is what runs.
>> time.timezone
18000
>> time.timezone/3600
5.0
That represents a 5 hour difference. But we are currently in Eastern Daylight Time (EDT) which is only a 4 hour difference. Thus, the timestamps being sent to InfluxDB2 are one hour into the future.
I suggest that in order to always get the correct timestamp in the UTC timezone, you use Python's built-in timezones like this:
# just showing what you need, I don't expect the import right here but at the top of the file
from datetime import datetime, timezone
jsondate = "2023-10-15T17:40:24"
local_time = datetime.strptime (jsondate, "%Y-%m-%dT%H:%M:%S")
utc_time = local_time.astimezone(timezone.utc)
ifdt = utc_time.strftime("%Y-%m-%dT%H:%M:%S")
if conf.verbose : print("\t - " + "Grott original time : ",jsondate,"adjusted UTC time for influx : ",ifdt)
this correctly shows:
- Grott original time : 2023-10-15T17:40:24 adjusted UTC time for influx : 2023-10-15T21:40:24
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>
|
I don't see a timezone setting in the grott config. Can you please clarify?
☝️ is all I see and it's unclear what value I should use other than |
|
I’ve got this in my config:
```
time = server
timezone = Europe/Amsterdam
```
We’re suffering from dst here too, that’s why I investigated the timezone setting.
|
|
I can give that a try, although if I step back from this thread of conversation, I would observe that I shouldn't need to correctly configure anything in order for a program to successfully change an arbitrary timestamp to UTC. |
|
Correct, but then the timestamp needs to contain TZ info, so the software knows where to convert from, but the date/time from the Growatt inverter doesn't contain this info. |
|
Ok. Every year there seams to be some issues with DST. I tried to solve that in Grott but apparently it does not works for everybody. I think the problem only occurs with influxdb ? Is that correct? For witing to influx I have chosen to do this in GMT. The reason for this that is is useual to write log data in GMT (especially in time series databases) and the set timezones during the presentation of the data (e.g. Grafana). I use the timezone offset in the server on which grott is running, for that or the timezone you specified in the timezone parameter in the .ini. (see also: https://github.com/johanmeijer/grott/wiki/InfluxDB-Support ). The challenge is that Growatt inverter data records are written in the timezone you specified in the Growatt settings on the website (Growatt sends a time update configuration ones in a while). Maybe this causes the problem. If the timezone= (e.g. timezone=Europe/Amsterdam) not works we have to find out what is going wrong. I need more information. E.g. The growatt original datarecord, the server type and setting you running on, your timezone and the settings in the grott.ini (or docker environmentals). |
|
I looked in the issue history and think that setting timezone= in the grott.ini should solve this issue please verify and feedback on that! |
|
@johanmeijer I can say that setting the timezone in grot.ini works for me. |
No the same holds for HomeAssistant (i.e. mqtt data stream) in my case. |
|
That is strange while Grott does not change the MQTT time. Then it might be a problem of the Growatt time settings(website, phone app). Can you sent me the information I requested above to so I can look at and test with your data? |
I only use InfluxDB2, so I cannot say for anything else.
That's exactly my opinion of how to handle times.
I recommend permitting python's built-in libraries to transpose timestamps to UTC as they're likely better tested than any 3rd-party library (i.e. pytz). I think it's more likely to be correct than attempting to handle all the offset calculations yourself.
Growatt may send time updates but I think it's far more likely that your users have their servers setup correctly (with regards to time) than their growatt cloud account settings.
All of that can be skipped if you permit python's built-in libraries to do it for you. Check out my associated PR for concrete code examples of what I mean. |
I have some doubts if your suggested change will work. The datetime coming from the inverter doesn't contain any timezone info, so you have to assume some tz for that incoming date time. This is where the |
|
I think using the PYTZ or Python's own method is not really relevant. It is more on what you are doing. What I try to do is to correct the Growatt (inverter / datalogger) time to GMT. Mt first "Brilliant" idea was to take the offset of GMT from the Local server time. That did not work properly (some strange behaviour with DST indeed) so I introduced the timezone parameter in the grott.ini. This now used to calculated the GMT and works pretty well I think (at least for me). I am aware of every programmer is doing things at his own way and how he/she is used to. At the moment I created the code PYTZ did it for me (I am not sure if I tried then Python timezone or not, maybe it was not even there yet). For know if it works I am not going to break it (just for the coding esthetics). |
|
Could I suggest to port in the future to the source: https://pypi.org/project/pytz/ The only downside, it's only python 3.9+ but the official support for python 3.8 end in 11 months. To compensate this, there is backports of zoneinfo |
|
I will look at it in a next version. Python version compatibility is an attention point! |
|
But be aware. Normally Grott does not any time processing. It uses data/time from the inverter record (which is controlled by the Growatt website/phoen app settings). Grott will not change this by default. De inverter date/time is used for MQTT and PVOutput. Exceptions:
Be aware:
|
|
My data stopped working this morning and as I investigated why, I realized that this issue has struck again. Today is the day that our clocks spring forward one hour. And I see this in the logs: However, it is currently 09:30 UTC, not 08:30 UTC. My server knows the correct local time: 5:30 EDT would translate correctly to 09:30 UTC. But grott uses the datetime coming from growatt instead of from the local server.
Previously, I mentioned that I think it is better to use the server's current time than what is in the growatt records. I still think that is the case. I don't mean to use the server's local timezone, I mean to start with the server's local time and transpose that to UTC. |
I've set mine to: And now I'm getting the correct time. Of course, I had to restart grott to do that, so it's not clear to me that the configuration change is what solved it, or whether it was just restarting. Either way, it works now. |
Currently, on the east coast of the US, it is 17:40 hours. Current UTC time is 21:40. That's a 4-hour difference. In grott logs, I can see this:
- Grott original time : 2023-10-15T17:40:24 adjusted UTC time for influx : 2023-10-15T22:40:24. That's a 5-hour difference. The result is that my influxdb records are timestamped one hour into the future!Here is the code that tries to get the timezone:
conf.tmzoneis currently set to"local". That's not a timezone that pytz understands.and so, the exception block is what runs.
That represents a 5 hour difference. But we are currently in Eastern Daylight Time (EDT) which is only a 4 hour difference. Thus, the timestamps being sent to InfluxDB2 are one hour into the future.
I suggest that in order to always get the correct timestamp in the UTC timezone, you use Python's built-in timezones like this:
this correctly shows:
The text was updated successfully, but these errors were encountered: