Werkzeug 1.0.0 breaks flask libraries

[tomres]

Attempted to install Mycodo on a clean install of raspbian buster lite (February 2020 version). Message says mycodo installed successfully but I get an error 502 when trying to access the web interface.

here's my setup log
setup.log

[kizniche]

I see 3 issues.

1:

#### Creating InfluxDB database and user
#### Attempting to connect...
#### Could not connect to Influxdb. Waiting 30 seconds then trying again...
#### Attempting to connect...
#### Could not connect to Influxdb. Waiting 30 seconds then trying again...
#### Attempting to connect...
#### Could not connect to Influxdb. Waiting 30 seconds then trying again...

This means the database and user for influxdb was not created.

2:

#### Connecting to http://localhost (creates Mycodo database if it doesn't exist)
#### Could not connect to http://localhost. Waiting 60 seconds then trying again (up to 5 times)...
#### Trying again...
#### Could not connect to http://localhost. Waiting 60 seconds then trying again (up to 5 times)...
#### Trying again...
#### Could not connect to http://localhost. Waiting 60 seconds then trying again (up to 5 times)...
#### Trying again...
#### Could not connect to http://localhost. Waiting 60 seconds then trying again (up to 5 times)...
#### Trying again...
#### Could not connect to http://localhost. Waiting 60 seconds then trying again (up to 5 times)...
#### Trying again...

This means the web server couldn't be connected to. I suspect the web server never started, from the next error.

3:

#### Restarting the Mycodo daemon
Traceback (most recent call last):
File "/home/pi/Mycodo/mycodo/scripts/restart_daemon.py", line 12, in
from mycodo.config import DAEMON_PID_FILE
File "/home/pi/Mycodo/mycodo/config.py", line 10, in
from flask_babel import lazy_gettext
File "/home/pi/Mycodo/env/lib/python3.7/site-packages/flask_babel/init.py", line 19, in
from werkzeug import ImmutableDict
ImportError: cannot import name 'ImmutableDict' from 'werkzeug' (/home/pi/Mycodo/env/lib/python3.7/site-packages/werkzeug/init.py)

This error is unexpected.

The Pi zero is very slow, which is why 1 and 2 have wait periods and repetitions, specifically for the Pi zero. However, it seems for some reason it wasn't long enough to allow influxdb to start (1), as Influxdb needs to be started before a database and user can be created. 2 is trying to connect to the web server (which the first connect creates the sqlite settings database). I suspect the error in 3 preventing the daemon from starting is also preventing the web server from starting because the same lazy_gettext is used that caused the daemon to crash. Hence, the webserver connect timed out all 5 times.

Error 3 is likely what's causing all the problems, and I suspect it's the result of filesystem corruption, which can be caused by a bad Raspbian write to your SD card or a bad SD card.

I would recommend redownloading the Raspbian image and writing it to a different SD card and trying the install again and see if the exact same issues (1, 2, and 3) appear.

I am building a project that I'll need to install Mycodo fresh to a Pi3, so I'll be able able to test shortly. But in the meantime, I would recommend testing the above potential fix.

[tomres]

Hey Kyle, thanks for the quick response!
I tried with a fresh install of raspbian on a different SD card and had the same error. Maybe the raspberry pi zero is not the right choice for this project. I have a pi 2 model b so I will give it a shot on that.
Thanks again for all your work on this awesome project!

[kizniche]

It has worked on the zero. I've spent quite a bit of time fixing issues in the past so it would. There are also several active installs on zeros. Perhaps this is an issue with a piece of 3rd party software. I'll try to replicate the issue this weekend if I have time and if I can, I'll try to diagnose the issue/push a fix.

[fry42]

Hi!

Just got the same errors on a RPi4 with a fresh raspbian buster lite (2020-02-05) install.

There seem to be two problems:

• Timeout on influx-db
  I just ran the script to create db&user seperately and it seems to have fixed that problem:
  sudo ./Mycodo/mycodo/scripts/upgrade_commands.sh update-influxdb-user
  Maybe some permission or timeout problem? (Creating the DB&user takes a few seconds, even on a pi4)

• Several errors due to werkzeug 1.0.0?
  I found some errors importing functions form werkzeug.
  After some research I found that some things have been changed in release 1.0.0 - so I started downgrading:
  pip uninstall werkzeug
  pip install werkzeug==0.16.0
  pip uninstall flask-babel
  pip install flask-babel==0.12.2

Still got errors importing the libs - so I changed:

./Mycodo/env/lib/python3.7/site-packages/flask_babel/init.py
from werkzeug import ImmutableDict => from werkzeug.datastructures import ImmutableDict

./Mycodo/env/lib/python3.7/site-packages/flask_wtf/recaptcha/widgets.py
./Mycodo/env/lib/python3.7/site-packages/flask_wtf/recaptcha/validators.py
from werkzeug import url_encode => from werkzeug.urls import url_encode

./Mycodo/env/lib/python3.7/site-packages/flask_restplus/fields.py"
./Mycodo/env/lib/python3.7/site-packages/flask_restplus/api.py
from werkzeug import cached_property => from werkzeug.utils import cached_property

After running sudo ./Mycodo/install/setup.sh again I managed to finish the basic setup successfully.

[kizniche]

It looks like Flask-Babel just released (2 days ago) 1.0.0 that fixes the werkzeug issue. All you need to (theoretically) do is install the latest Flask-Babel. I'll test soon.

[kizniche]

I just pushed potential fixes, so if you would like to test, you can install from master with the following commands. First, make sure there isn't a ~/Mycodo directory so you're installing fresh.

sudo apt install git
cd ~
git clone https://github.com/kizniche/Mycodo
cd Mycodo/install
sudo ./setup.sh

[kizniche]

For reference, this seems to be the issue: python-babel/flask-babel#157

[kizniche]

I see TravisCI showing now three's an issue with flask_wtf. Stand by while I investigate.

[kizniche]

That fixed flask-wtf, now flask_restplus is having issues...

[kizniche]

Seems Flask-RestPlus is dead, replaced by Flask-RESTX. Until I can merge from RestPlus to RESTX, I'm testing the fix by downgrading Werkzeug to 0.16.1.

[kizniche]

Merging to Flask-RESTX was easier than I thought, so that's now done. They're currently working on a solution for using Werkzeug 1.0.0, which should be a part of their next release. Until then, Mycodo will use Werkzeug 0.16.1.

It should now be safe to install from master to test.

[kizniche]

Timeout on influx-db

I didn't mention, but I also increased the wait period for both influxdb database/user creation and daemon restart to 10 tries with 60 second pauses between. That should be more than enough.

[tomres]

I just pushed potential fixes, so if you would like to test, you can install from master with the following commands. First, make sure there isn't a ~/Mycodo directory so you're installing fresh.

sudo apt install git
cd ~
git clone https://github.com/kizniche/Mycodo
cd Mycodo/install
sudo ./setup.sh

Everything seems to be working on the raspi zero now,

[kizniche]

Great! Thanks for testing on the zero.

[kizniche]

I'm going to keep this open until Flask-RESTX updates to be compatible with Werkzeug 1.0.0.

[kizniche]

flask-restx just released 0.2.0. I'll test and include the version change in the next release if I don't find any issues.