Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reimplement the ArgSplitter in Rust. #21814

Merged
merged 5 commits into from
Jan 11, 2025
Merged

Conversation

benjyw
Copy link
Contributor

@benjyw benjyw commented Jan 8, 2025

The only functionality in the legacy ArgSplitter not
replicated in Rust is distinguishing between
builtin/auxiliary goals and regular goals. This
can easily be done post-hoc, in Python, as
that distinction is not required in order to correctly
split args (now that we don't assign flags to
scopes in the ArgSplitter).

The tests are ported directly from the relevant
Python ones.

A future PR will actually wire this new ArgSplitter
up instead of the Python one.

@benjyw benjyw added the category:internal CI, fixes for not-yet-released features, etc. label Jan 8, 2025
@benjyw benjyw requested review from huonw and tdyas January 8, 2025 23:50
@benjyw
Copy link
Contributor Author

benjyw commented Jan 8, 2025

Note that the future PR to wire this up already works in a branch, but I wanted to break things up for easier reviewing.

@@ -662,9 +662,17 @@ class PyOptionId:
self, *components: str, scope: str | None = None, switch: str | None = None
) -> None: ...

class PySplitArgs:
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's possible that we won't need to access this in Python in the future, but for now I've implemented this, to make the port less invasive, and also so that I can run the existing Python tests against this new implementation.

@@ -109,7 +109,7 @@ def _get_known_goal_scopes(
yield alias, si

def split_args(self, args: Sequence[str]) -> SplitArgs:
"""Split the specified arg list (or sys.argv if unspecified).
"""Split the specified arg list.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The previous statement was false...

// We disambiguate thus: If it is a known goal name, assume the user intended a goal.
// Otherwise, if it looks like a path or target, or exists on the filesystem, assume
// the user intended a spec, otherwise it's an unknown goal.
// TODO: This is probably not good logic, since the CLI behavior will change based on
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This logic is preserved for compatibility, but we should probably consider changing it at some point.

Copy link
Contributor

@tdyas tdyas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some minor comments but LGTM otherwise.

src/rust/engine/options/src/arg_splitter.rs Outdated Show resolved Hide resolved
src/rust/engine/src/externs/options.rs Outdated Show resolved Hide resolved
src/rust/engine/src/externs/options.rs Outdated Show resolved Hide resolved
Copy link
Contributor

@huonw huonw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good other than questions (but those questions don't block landing this).

// which starts with a single dash, and we know is not a single dash flag).
if arg.starts_with("-")
|| SPEC_RE.is_match(&arg)
|| self.build_root.join(Path::new(&arg)).exists()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is sync IO. Does sync vs. async matter at this point of the application?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think so - the Python equivalent was blocking on the same thing at the same point. And args are split long before we create a scheduler - we have to split them in order to get the option values for setting up the scheduler.

src/rust/engine/options/src/arg_splitter.rs Show resolved Hide resolved
@benjyw benjyw merged commit c978b6b into pantsbuild:main Jan 11, 2025
24 checks passed
@benjyw benjyw deleted the arg_splitting branch January 11, 2025 01:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
category:internal CI, fixes for not-yet-released features, etc.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants