adb server claims usb device: results in USBErrorBusy

[barkside]

Description

connect() fails with USBErrorBusy. Only way to get past this is to run "adb kill-server" and then quickly retry.
Looks like self._transport.claimInterface(self._interface_number) fails.

transport.kernelDriverActive(iface_number) returns false before this too.

This is using a Xiaomi phone plugged in to Ubuntu 20.

Looking in /sys/bus/usb/drivers/usb/1-1.2/1-1.2:1.0/ (my device) I see driver is:
driver -> ../../../../../../../bus/usb/drivers/usbfs/

Looks like the adb command line - used for "adb devices" for example, claims the device indefinitely.

Anyway around this anyone?