Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PgSelect now yields an object, not an array #2335

Open
wants to merge 48 commits into
base: main
Choose a base branch
from
Open

Conversation

benjie
Copy link
Member

@benjie benjie commented Jan 24, 2025

Description

hasMore used to be hacked onto the returned array which was always a disgusting hack but it worked and I couldn't justify spending the time to fix it. But it turns out we need more stuff like that now that we're eradicating eval, for example we need to know what the indicies are that were used to select the orderBy values so that we can populate the cursors with them. Since we're likely to need more stuff like this in future, I've changed PgSelect to return an object, where one of the entries in the object is items which is the previous array; then I've added PgSelectRowsStep and PgUnionAllRowsStep to extract these items whilst still honouring stream/defer.

I've also updated Grafast to have a new convention method, ExecutableStep::items(), which will be called when a step is to be used in a list position. This gives the step a chance to return an alternative step to yield the list itself, this enables the PgSelect to be used directly in list positions (without having to .items() yourself manually), and to feed it to things like listTransform (each/filter/groupBy/etc). This minimizes the amount of rewriting necessary.

I've also merged the stream() and execute() methods - these are no longer separate methods, in fact stream() is no longer supported. The return signature of execute() has changed to accomodate this, but no change is needed on existing step classes unless they implement a stream() method.

I've also rewritten how the @stream directive is interpreted and moved it entirely to runtime, including in the output generation.

All of this is breaking.

Performance impact

Work towards:

Security impact

Working towards eradicating step eval, so positive.

Checklist

  • My code matches the project's code style and yarn lint:fix passes.
  • I've added tests for the new feature, and yarn test passes.
  • I have detailed the new feature in the relevant documentation.
  • I have added this feature to 'Pending' in the RELEASE_NOTES.md file (if one exists).
  • If this is a breaking change I've explained why.

benjie added 30 commits January 22, 2025 16:03
Copy link

changeset-bot bot commented Jan 24, 2025

🦋 Changeset detected

Latest commit: ec2df49

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 15 packages
Name Type
postgraphile Patch
@dataplan/pg Patch
grafast Patch
graphile-utils Patch
pgl Patch
graphile Patch
graphile-build-pg Patch
@localrepo/grafast-bench Patch
@dataplan/json Patch
@grafserv/persisted Patch
grafserv Patch
ruru-components Patch
@localrepo/grafast-website Patch
graphile-build Patch
graphile-export Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

…ist positions to return a _different_ step to actually return the list - useful for returning connection-capable steps in list positions.
…ute() now handles both stream() and defer() concerns.
…, which has meant that stream info is no longer passed at planning time - instead execute() can evaluate if it is being streamed or not and make decisions based on that.
…ather than arrays/streams; thanks to the new Grafast .items() method and these classes being "opaque" steps this is _mostly_ a non-breaking change.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: 🌳 Triage
Development

Successfully merging this pull request may close these issues.

1 participant