Filtering by multiple values

[capnahab]

I am very impressed with visidata. I have a dataframe of anonmymised hospital data of 100 fields and 600k rows. I want to filter one of the columns by surgery type. The surgical codes are a string of 4 letters and numbers eg 'B041' . There are usually multiple codes for one operation. When there are only 2-4 I search the column using regex and select the values.
Sometimes though I have a list od 20+ codes. In python I would do this using .isin (in Jupyterlab).

Is there a good way to filter rows using a list of values.
Simpler the better as I am a surgeon not a data analyst....

[saulpw]

Hi @capnahab, you can do this one of several ways:

• Selection by default is additive, so you could use any selection primitive (like |B041) to add rows with each code to the selection. Then use " to pull them off into their own sheet and maybe Shift+F to verify that you got all the codes you want (and only the codes you want).
• You could use z| to use a Python expression like you would in Jupyterlab. You can't use .isin() (which is a Pandas DataFrame method), but you could do colname in 'B041 B052 B063 C023'.split() or something similar. If you find yourself using the same sets of codes repeatedly you could add some simple constants to your .visidatarc so you don't have to retype them.
• You could use Shift+F on that column to get the freq sheet with one row per unique code, and then search and individually select rows on that sheet. Then g Enter will pull all the source rows for those selected rows into a single sheet.

There might be others but these are the ones off the top of my head. Do any of these work for your case?

[capnahab]

Thats great, cleared up a number of my thoughts. Mainly why .isin wasn't working. Will give these suggestions a try. I probably don't know enough python to know why I can't pass in a list of eg pituitary surgery codes eg
['B041', 'B012', 'B014', 'B016'] - but have tried all combos of list.

[saulpw]

You can pass in the list: z|code in ['B041', B012'] where code is the column name. 'B041 B012'.split() is IMO more convenient shorthand. (and if there are no codes that are partial matches of real codes, like B04 in the above, you could be a total hacker and just use z|code in 'B041 B012 B014 B016').

[capnahab]

Fabulous, that works a treat. Thanks so much, you have made my day.

[geekscrapy]

I think you want a command like I have "select-col-from-clipboard". I use it pretty much everyday.
I copy a list of values to the clipboard (from another app) and move to the column which contains the same values. I then use this command and it iteratively goes through the clipboard list selecting each row of there is a match. I have 2 flavours of the command, regex and non regex (select exact) selecting.
Unfortunately I don't have the code to have, but would this be of interest?

[capnahab]

Yes such a command would be really helpful if you could share it.