0.26.0 Release

.github/workflows/publish-to-pypi.yml

@@ -1,8 +1,8 @@
 name: Publish distributions to PyPI and TestPyPI
 on:
-  push:
-    tags:
-      - "*"
+  release:
+    types:
+      - released
 
 jobs:
   build-and-publish:

README.md

@@ -14,17 +14,18 @@ bellows interacts with the Zigbee Network Coprocessor (NCP) with EmberZNet PRO Z
 ## Hardware requirement
 
 EmberZNet based Zigbee radios using the EZSP protocol (via the [bellows](https://github.com/zigpy/bellows) library for zigpy)
+ - [Tube's Zigbee Gateways (Silabs EFR32 variant)](https://github.com/tube0013/tube_gateways) Note! ESP32 based Ethernet bridge available as pre-assembed or as a DIY project.
  - [ITEAD Sonoff ZBBridge](https://www.itead.cc/smart-home/sonoff-zbbridge.html) (**Note! WiFi-based bridges are not recommended for ZHA with EZSP radios.** Also, this first have to be flashed with [Tasmota firmware and EmberZNet firmware](https://www.digiblur.com/2020/07/how-to-use-sonoff-zigbee-bridge-with.html))
+ - ITead Zigbee 3.0 USB Dongle (EFR32MG21) Model 9888010100045 Note! Currently not recommended due to stability issues.
  - [Nortek GoControl QuickStick Combo Model HUSBZB-1 (Z-Wave & Zigbee Ember 3581 USB Adapter)](https://www.nortekcontrol.com/products/2gig/husbzb-1-gocontrol-quickstick-combo/) (Note! Not a must but recommend [upgrade the EmberZNet NCP application firmware](https://github.com/walthowd/husbzb-firmware))
  - [Elelabs Zigbee USB Adapter](https://elelabs.com/products/elelabs_usb_adapter.html) (Note! Not a must but recommend [upgrade the EmberZNet NCP application firmware](https://github.com/Elelabs/elelabs-zigbee-ezsp-utility))
  - [Elelabs Zigbee Raspberry Pi Shield](https://elelabs.com/products/elelabs_zigbee_shield.html) (Note! Not a must but recommend [upgrade the EmberZNet NCP application firmware](https://github.com/Elelabs/elelabs-zigbee-ezsp-utility))
  - [DEFARO SprutStick Pro (also known as Defaro SprutStick ZigBee 2 Pro)](https://defaro.ru/index.php/product/89-controllers/257-sprutstick-pro)
- - Telegesis ETRX357USB (Note! This first have to be [flashed with other EmberZNet firmware](https://github.com/walthowd/husbzb-firmware))
- - Telegesis ETRX357USB-LRS (Note! This first have to be [flashed with other EmberZNet firmware](https://github.com/walthowd/husbzb-firmware))
- - Telegesis ETRX357USB-LRS+8M (Note! This first have to be [flashed with other EmberZNet firmware](https://github.com/walthowd/husbzb-firmware))
- - [IKEA Billy EZSP](https://github.com/MattWestb/IKEA-TRADFRI-ICC-A-1-Module).
- - [Tuya TYGWZ01 and with labeled (Silvercrest / Lidl) Smart Home Gateway](https://paulbanks.org/projects/lidl-zigbee/).
- - Bitron Video/Smabit BV AV2010/10 USB-Stick (a.k.a. Telekom Magenta Stick) based on Silicon Labs Ember 3587 
+ - Telegesis ETRX357USB/ETRX357USB-LRS/TRX357USB-LRS+8M (Note! This first have to be [flashed with other EmberZNet firmware](https://github.com/walthowd/husbzb-firmware))
+ - [IKEA Billy EZSP - DIY ICC-1 / ICC-A-1 module from IKEA TRÅDFRI devices](https://github.com/MattWestb/IKEA-TRADFRI-ICC-A-1-Module). (Note! This first have to be hacked and flashed with other EmberZNet firmware)
+ - [Tuya TYGWZ01 and rebranded Lidl Silvercrest Smart Home Gateway](https://paulbanks.org/projects/lidl-zigbee/) (Note! This first have to be hacked and flashed with other EmberZNet firmware)
+ - Bitron Video/Smabit BV AV2010/10 USB-Stick (a.k.a. Telekom Magenta Stick) based on Silicon Labs Ember 3587
+ - [EByte E180-Z120B SMD Module and EByte E180-Z120B-TB Evaluation Board](https://www.cnx-software.com/2020/04/27/ebyte-e180-zg120b-tb-zigbee-3-0-evaluation-board-features-silicon-labs-efr32mg1b-zigbee-thread-soc/) (Note! This first have to be [hacked and flashed with other EmberZNet firmware](https://github.com/zha-ng/EZSP-Firmware/tree/master/EByte-E180-Z120B))
 
 ### Warning about Zigbee to WiFi bridges
 
@@ -104,7 +105,7 @@ $ bellows zcl 00:0d:6f:00:05:7d:2d:34 1 1026 read_attribute 0
   - Plug in the dongle. It should now be recognized properly as ttyUSBx.
 
 #### Port configuration for network adapters via socket
-- To configure the use of a remote Ethernet or WiFi based network connected bridge/proxy Zigbee adapter, like exammple Sonoff ZBBridge or ZiGate WiFi Gateway, enter `socket://adapter-IP>:8888` and use 115200 baud rate as the port speed.
+- To configure the use of a remote Ethernet or WiFi based network connected bridge/proxy Zigbee adapter, like exammple Sonoff ZBBridge or ZiGate WiFi Gateway, enter `socket://<adapter-IP>:<port>` and use 115200 baud rate as the port speed.
 
 ### NVRAM Backup and restore
 

bellows/__init__.py

@@ -1,5 +1,5 @@
 MAJOR_VERSION = 0
-MINOR_VERSION = 25
+MINOR_VERSION = 26
 PATCH_VERSION = "0"
 __short_version__ = f"{MAJOR_VERSION}.{MINOR_VERSION}"
 __version__ = f"{__short_version__}.{PATCH_VERSION}"

bellows/cli/dump.py

@@ -1,4 +1,5 @@
 import asyncio
+import logging
 import time
 
 import click
@@ -7,6 +8,8 @@
 from . import util
 from .main import main
 
+LOGGER = logging.getLogger(__name__)
+
 
 @main.command()
 @click.option(
@@ -30,13 +33,29 @@ def dump(ctx, channel, outfile):
         start_time = ctx.obj.get("start_time", None)
         if start_time:
             duration = time.time() - start_time
-            click.echo("\nCaptured %s frames in %0.2fs" % (captured, duration))
+            click.echo(
+                "\nCaptured %s frames in %0.2fs" % (captured, duration), err=True
+            )
     finally:
         if "ezsp" in ctx.obj:
             loop.run_until_complete(ctx.obj["ezsp"].mfglibEnd())
             ctx.obj["ezsp"].close()
 
 
+def ieee_15_4_fcs(data: bytes) -> bytes:
+    # Modified from the implementation in `scapy.layers.dot15d4:Dot15d4FCS.compute_fcs`
+    crc = 0x0000
+
+    for c in data:
+        q = (crc ^ c) & 15  # Do low-order 4 bits
+        crc = (crc // 16) ^ (q * 0x1081)
+
+        q = (crc ^ (c // 16)) & 15  # And high 4 bits
+        crc = (crc // 16) ^ (q * 0x1081)
+
+    return crc.to_bytes(2, "little")
+
+
 async def _dump(ctx, channel, outfile):
     s = await util.setup(ctx.obj["device"], ctx.obj["baudrate"])
     ctx.obj["ezsp"] = s
@@ -49,21 +68,35 @@ async def _dump(ctx, channel, outfile):
 
     pcap = pure_pcapy.Dumper(outfile, 128, 195)  # DLT_IEEE_15_4
 
-    click.echo("Capture started")
+    click.echo("Capture started", err=True)
     ctx.obj["start_time"] = time.time()
     ctx.obj["captured"] = 0
 
+    done_event = asyncio.Event()
+
     def cb(frame_name, response):
         if frame_name == "mfglibRxHandler":
             data = response[2]
+
+            # Later releases of EmberZNet incorrectly use a static FCS
+            fcs = data[-2:]
+            if s.ezsp_version == 8 and fcs == b"\x0F\x00":
+                computed_fcs = ieee_15_4_fcs(data[0:-2])
+                LOGGER.debug("Fixing FCS (expected %s, got %s)", computed_fcs, fcs)
+                data = data[0:-2] + computed_fcs
+
             ts = time.time()
             ts_sec = int(ts)
             ts_usec = int((ts - ts_sec) * 1000000)
             hdr = pure_pcapy.Pkthdr(ts_sec, ts_usec, len(data), len(data))
-            pcap.dump(hdr, data)
+
+            try:
+                pcap.dump(hdr, data)
+            except BrokenPipeError:
+                done_event.set()
+
             ctx.obj["captured"] += 1
 
     s.add_callback(cb)
 
-    while True:
-        await asyncio.sleep(1)
+    await done_event.wait()

bellows/ezsp/__init__.py

@@ -1,5 +1,7 @@
 """EZSP protocol."""
 
+from __future__ import annotations
+
 import asyncio
 import functools
 import logging
@@ -10,6 +12,7 @@
 
 from bellows.config import (
     CONF_DEVICE,
+    CONF_DEVICE_BAUDRATE,
     CONF_DEVICE_PATH,
     CONF_PARAM_SRC_RTG,
     SCHEMA_DEVICE,
@@ -21,7 +24,7 @@
 from . import v4, v5, v6, v7, v8
 
 EZSP_LATEST = v8.EZSP_VERSION
-PROBE_TIMEOUT = 3
+PROBE_TIMEOUT = 2
 NETWORK_OPS_TIMEOUT = 10
 LOGGER = logging.getLogger(__name__)
 MTOR_MIN_INTERVAL = 10
@@ -48,20 +51,21 @@ def __init__(self, device_config: Dict):
         self._protocol = None
 
     @classmethod
-    async def probe(cls, device_config: Dict) -> bool:
+    async def probe(cls, device_config: Dict) -> bool | dict[str, int | str | bool]:
         """Probe port for the device presence."""
-        ezsp = cls(SCHEMA_DEVICE(device_config))
-        try:
-            await asyncio.wait_for(ezsp._probe(), timeout=PROBE_TIMEOUT)
-            return True
-        except (asyncio.TimeoutError, serial.SerialException, APIException) as exc:
-            LOGGER.debug(
-                "Unsuccessful radio probe of '%s' port",
-                device_config[CONF_DEVICE_PATH],
-                exc_info=exc,
-            )
-        finally:
-            ezsp.close()
+        for config in (device_config, {**device_config, CONF_DEVICE_BAUDRATE: 115200}):
+            ezsp = cls(SCHEMA_DEVICE(config))
+            try:
+                await asyncio.wait_for(ezsp._probe(), timeout=PROBE_TIMEOUT)
+                return config
+            except (asyncio.TimeoutError, serial.SerialException, APIException) as exc:
+                LOGGER.debug(
+                    "Unsuccessful radio probe of '%s' port",
+                    device_config[CONF_DEVICE_PATH],
+                    exc_info=exc,
+                )
+            finally:
+                ezsp.close()
 
         return False
 

bellows/zigbee/application.py

@@ -149,6 +149,11 @@ async def startup(self, auto_form=False):
             assert status == t.EmberStatus.SUCCESS
 
         LOGGER.info("Node type: %s, Network parameters: %s", node_type, nwk_params)
+        self._ext_pan_id = nwk_params.extendedPanId
+        self._pan_id = nwk_params.panId
+        self._channel = nwk_params.radioChannel
+        self._channels = nwk_params.channels
+        self._nwk_update_id = nwk_params.nwkUpdateId
 
         await ezsp.update_policies(self.config)
         nwk = await ezsp.getNodeId()

tests/test_application.py

@@ -11,6 +11,7 @@
 from bellows.exception import ControllerError, EzspError
 import bellows.ezsp as ezsp
 import bellows.ezsp.v4.types as t
+import bellows.types.struct
 import bellows.uart as uart
 import bellows.zigbee.application
 
@@ -68,8 +69,7 @@ def ieee(init=0):
 @patch("zigpy.device.Device._initialize", new=AsyncMock())
 @patch("bellows.zigbee.application.ControllerApplication._watchdog", new=AsyncMock())
 async def _test_startup(app, nwk_type, ieee, auto_form=False, init=0, ezsp_version=4):
-    async def mockezsp(*args, **kwargs):
-        return [0, nwk_type, sentinel.nework_parameters]
+    nwk_params = MagicMock(spec_set=bellows.types.struct.EmberNetworkParameters())
 
     async def mock_leave(*args, **kwargs):
         app._ezsp.handle_callback("stackStatusHandler", [t.EmberStatus.NETWORK_DOWN])
@@ -85,9 +85,7 @@ async def mock_leave(*args, **kwargs):
     ezsp_mock.addEndpoint = AsyncMock(return_value=t.EmberStatus.SUCCESS)
     ezsp_mock.setConfigurationValue = AsyncMock(return_value=t.EmberStatus.SUCCESS)
     ezsp_mock.networkInit = AsyncMock(return_value=[init])
-    ezsp_mock.getNetworkParameters = AsyncMock(
-        return_value=[0, nwk_type, sentinel.nework_parameters]
-    )
+    ezsp_mock.getNetworkParameters = AsyncMock(return_value=[0, nwk_type, nwk_params])
     ezsp_mock.get_board_info = AsyncMock(
         return_value=("Mock Manufacturer", "Mock board", "Mock version")
     )
@@ -115,6 +113,22 @@ async def test_startup(app, ieee):
     await _test_startup(app, t.EmberNodeType.COORDINATOR, ieee)
 
 
+async def test_startup_nwk_params(app, ieee):
+    assert app.pan_id is None
+    assert app.extended_pan_id is None
+    assert app.channel is None
+    assert app.channels is None
+    assert app.nwk_update_id is None
+
+    await _test_startup(app, t.EmberNodeType.COORDINATOR, ieee)
+
+    assert app.pan_id is not None
+    assert app.extended_pan_id is not None
+    assert app.channel is not None
+    assert app.channels is not None
+    assert app.nwk_update_id is not None
+
+
 async def test_startup_ezsp_ver7(app, ieee):
     await _test_startup(app, t.EmberNodeType.COORDINATOR, ieee, ezsp_version=7)
 
@@ -1067,7 +1081,8 @@ async def test_probe_success(mock_connect, mock_reset):
     """Test device probing."""
 
     res = await ezsp.EZSP.probe(APP_CONFIG[config.CONF_DEVICE])
-    assert res is True
+    assert res
+    assert type(res) is dict
     assert mock_connect.call_count == 1
     assert mock_connect.await_count == 1
     assert mock_reset.call_count == 1
@@ -1077,7 +1092,8 @@ async def test_probe_success(mock_connect, mock_reset):
     mock_reset.reset_mock()
     mock_connect.reset_mock()
     res = await ezsp.EZSP.probe(APP_CONFIG[config.CONF_DEVICE])
-    assert res is True
+    assert res
+    assert type(res) is dict
     assert mock_connect.call_count == 1
     assert mock_connect.await_count == 1
     assert mock_reset.call_count == 1

tests/test_ezsp.py

@@ -255,7 +255,7 @@ async def test_probe_success(mock_connect, mock_reset):
     """Test device probing."""
 
     res = await ezsp.EZSP.probe(DEVICE_CONFIG)
-    assert res is True
+    assert type(res) is dict
     assert mock_connect.call_count == 1
     assert mock_connect.await_count == 1
     assert mock_reset.call_count == 1
@@ -265,7 +265,7 @@ async def test_probe_success(mock_connect, mock_reset):
     mock_reset.reset_mock()
     mock_connect.reset_mock()
     res = await ezsp.EZSP.probe(DEVICE_CONFIG)
-    assert res is True
+    assert type(res) is dict
     assert mock_connect.call_count == 1
     assert mock_connect.await_count == 1
     assert mock_reset.call_count == 1
@@ -286,10 +286,10 @@ async def test_probe_fail(exception):
         res = await ezsp.EZSP.probe(DEVICE_CONFIG)
 
     assert res is False
-    assert mock_connect.call_count == 1
-    assert mock_connect.await_count == 1
-    assert mock_reset.call_count == 1
-    assert mock_connect.return_value.close.call_count == 1
+    assert mock_connect.call_count == 2
+    assert mock_connect.await_count == 2
+    assert mock_reset.call_count == 2
+    assert mock_connect.return_value.close.call_count == 2
 
 
 @patch.object(ezsp.EZSP, "set_source_routing", new_callable=AsyncMock)