BLD: line profiler no longer requires ipython, unpin needed for py3.12

[tangkong]

Apparently line-profiler hasn't strictly required ipython for a while, and at least locally it seems like I can install 4.1.2 while pinning ipython at 8.4.0.

Curiosity getting me here since I don't want to do other more "productive" things

[tangkong]

It seems the ipython - line-profiler pin is defined in the conda recipe, but pip allows you to install line-profiler without it. I tried moving to the pip installation to avoid this entirely
https://github.com/conda-forge/line_profiler-feedstock/blob/d811524f0cadca17537bd5d7a987f19644781751/recipe/meta.yaml#L39

[ZLLentz]

If this ends up being a blocker it's probably ok to update ipython to a more recent version

[tangkong]

The scientists scare me 😨 Looking at line profiler it's really just ipython magics that we don't use (but probably should).

I was wondering if we'd end up wanting to scream loudly into the abyss to try and get scientists to try the new ipython version. Maybe that's a fool's errand and we'll get yelled at anyway

[tangkong]

Ok I think this is what's happening, but zach the env master should correct me if I'm wrong.

py312 next-full Log failure

error    libmamba Could not solve for environment specs
    The following package could not be installed
    └─ line-profiler =* * does not exist (perhaps a typo or a missing channel).

Running create_base_env.sh test pcds 3.12 (workflow link)

We first create a base environment with the packages listed in conda-packages.txt and security packages.txt (src)

After this, we "get extra dependencies", and install those via conda/mamba (src)

• with the changes in this PR, line_profiler is no longer in the base environment
• line-profiler (note the dash -, not an underscore) is listed as an extra dependency for happi, and is collected by get_extras.py
  • https://github.com/pcdshub/happi/blob/master/dev-requirements.txt#L14
• line_profiler (with an underscore) is gathered as a extra dependency of typhos
  • https://github.com/pcdshub/typhos/blob/master/dev-requirements.txt#L5
• (I checked that get_package_extra_deps returns these different variants of line_profiler)

line-profiler does not exist on conda-forge, but line_profiler does.

I think one solution could be changing happi to use the underscore version of line_profiler pcdshub/happi#352

[ZLLentz]

I think the underscore solution is reasonable and your analysis is correct
Maybe there's other ways too (unclear) but this works for sure

[tangkong]

happi has been tagged, we now wait for conda-forge to tick over

[tangkong]

It's running!

I do find it confusing that we gather extras based on the pip specifications, which we then use to pick packages to install in conda. We should probably check conda's list of extra dependencies, but maybe that's easier said than done

[tangkong]

It looks like the Binstar client does not return any of the test dependencies from the conda recipe. I think we typically don't keep those up-to-date anyway, so I'll give up on that little investigation

[tangkong]

Failures are pyca (expected failures), and typhos. With a bit of poking over lunch, I hacked together a py312 environment and tried to replicate the package versions from the CI. I could not replicate the segfault unfortunately. The last test before the segfault is the cli-profiler test, which is indeed the subject of this PR, but I still can't seem to replicate the failure.

This feels close though, despite the flagrant disregard for line_profiler's conda ipython pins

[ZLLentz]

Do you want to claim forward progress and merge then?

[tangkong]

I'm not entirely sure I'm happy with this, unless we're pushing for an env I'd rather try to sort out the line-profiler issues all together. I think there's a world where this style of unpinning isn't correct, though I don't exactly have an alternative idea about it.

[tangkong]

Without falling off the deep end tracing line_profiler's dependencies, I can't exactly pinpoint why line_profiler has such strict ipython requirements. With ipython 8.4.0 and line_profiler 4.2.0, I can still run the ipython %lprun magics and get reasonable output. 🤷

I think I'll declare this good enough for now, with an additional comment describing what went on

[tangkong]

Also maybe kicking this pulls new pyca and changes some of the ci?

[tangkong]

pcdswidgets will probably need a tag for the extra deps fix to go through. I'll claim forward progress here and ask for a review

[ZLLentz]

Forward progress acknowledged!