How to create tables and columns using the api (plugin development)

[daviewales]

EDIT:

For those following along at home, I realised I need to read the docs in order, starting at the beginning.
In particular, the page on Loaders is clarifying a LOT of things for me

Things I didn't know when I started this...

1. If you want to make a new sheet, you need to make a new Loader. Loaders are not just for loading files from disk. They are for loading anything at all, into a new sheet.
2. Columns do not exist. Do not try to create columns directly from data. First, create rows. Columns are views into those rows.

Background (Original question)

I'm thinking about trying to make a plugin which does the following:

1. Open a folder full of CSVs in visidata
2. Select a couple of the CSVs
3. Run a command, and get a new table, where each column is all the columns of each selected table.
4. For bonus points, show a coloured diff of these columns. (Think vimdiff)

e.g.

table1.csv

id	fruit	price
1	apple	1.45

table2.csv (Oh no! They changed the schema!)

fruit_id	snack	price
1	apple	1.45
2	chips	2.50

1. Run vd.
2. Select table1 and table2.
3. <Space>-column-diff

(The name of each column would be the name of the respective sheet.)

Question

I've found the Plugin Author's Guide!
I've also figured out how to get the currently selected tables, and extract the column headers.

However, I'm wondering how to create a new table from raw columns. (e.g. from two lists.)
I'll keep reading through the docs, but I thought maybe somewhere here might be able to point me in the right direction!

Update 2021-05-19 2021-05-20 2021-05-24 2021-06-24

(Use the question history to see my previous attempts!)

OK! Now we're cooking!

from itertools import zip_longest
from visidata import vd, asyncthread, Progress, Sheet, TableSheet, SequenceSheet, Column


class DiffColumnsSheet(SequenceSheet):
    rowtype = 'fields'  # rowdef: [str]

    def iterload(self):
        selected_tables = self.source.selectedRows
        table_columns = []
        for table in selected_tables:
            column_headers = [column.name for column in table.columns]
            table_columns.append(column_headers)

        for line in zip_longest(*table_columns):
            with open('test-plugin.txt', 'a') as testfile:
                print(line, file=testfile)
            yield line

@Sheet.api
def show_diff_sheet(sheet):
    diff_sheet = DiffColumnsSheet("diff-sheet", source=sheet)
    vd.push(diff_sheet)

Sheet.addCommand(None, "show-diff-sheet", "vd.sheet.show_diff_sheet()")

Given the following files:

test.csv

this	is	a	test
column	this	wise	everything!
just	a	man	do
loves	column	once	not
being	I	said	test
a	see	many	nothing
column	here?	things	please

test2.csv

this	is	a	also	test
column	this	wise	I	everything!
just	a	man	will	do
loves	column	once	not	not
being	I	said	mind	test
a	see	many	at	nothing
column	here?	things	all	please

(Making sure to preload them and select them in the sheets sheet)

We get:

We've got a way to go, but we're finally on the right track!

[daviewales]

I've published the first working version as a plugin now!
See my VisiData Plugins repository on GitLab!

At the moment, it just compares the columns of the currently selected sheets side-by-side.

[saulpw]

This is really awesome, @daviewales ! I'm excited to see this in action and where you decide to take it!