Add experimental Target API

[NiklasRosenstein]

This PR introduces an experimental new API for configuring projects using a new "Target" API and rules that perform actions using those targets. This design is largely inspired by Pants. The PR acts as a proof-of-concept to experiment with the possibilities using a rule engine.

In the current implementation, the rules immediately execute their logic, but it's also conceivable that the rules merely contribute to the traditional Kraken DAG.

In the former scenario, we'd get rid of the task DAG that Kraken currently uses. The latter has the advantage that the DAG can be fully built before task execution, exposing an entry point for inspecting and debugging the build graph without running it.

Example

from kraken.build.python import python_app

python_app(
    name="main",
    entry_point="mypackage.main:main",
    interpreter_constraint=">=3.10,<3.12",
)

Now the krakenx CLI can be used to select a target and combine it with a "goal". The python_app() function creates two targets: main and main.pex. The main target represents the Python application for local development, main.pex represents the distributable PEX binary that can be built from the archive.

Usage

To install the application for development:

$ krakenx install main   # todo: Rename to `develop`?

To run the application entry point inside the development environment (this will also automatically install the application for development, i.e. it effectively implies krakenx install main):

$ krakenx run main

To build the PEX:

$ krakenx build main.pex

To run the built PEX (as above, this implies building the PEX and we can optimize not building it if nothing changed):

$ krakenx run main.pex

Notable differences from the status quo

• The Target API + rule engine introduced a lot of flexibility for introducing new functionality into the system (e.g. note how a PexResult is runnable with krakenx run because there is a rule for PexResult -> Executable).
• You must pick exactly one goal (or "action" I suppose) when using krakenx CLI (and potentially more than one target?). This means if you want to run two different things, you have to run two different invokations of krakenx (for example krakenx lint main and krakenx run main.pex). This would also allow us to add command-line options to "goals" that can be considered by the rule's implementation.

Impact on the current core

Some changes were necessary to the core, but they are fully backwards compatible. There's some typing shenanigans I had to do to satisfy Mypy -- maybe because I did this late at night so this could require some review at another time.

• Added the Target and NamedTarget classes
• Added Context.rule_engine
• Added Context.resolve_tasks(object_type) parameter to allow passing in NamedTarget instead of Task (which remains the default for backwards compatibility)
• Updated Project._members which can now also hold NamedTarget objects
• Added Project.defined_target(), Project.target() and Project.targets()

Todo

• Prove that this meets enterprise requirements well (e.g. injecting credentials for private registries, opinionated presets)

Other notes

• black and isort are skipping over the sources in the src/kraken/build directory 😢

[ntrinquier]

@NiklasRosenstein I skimmed through the change rapidly, for now I'd like to understand what the experience would be for non-Python targets, say for Rust crates (also to have >1 type of target in mind and avoid overfitting).

This is what I imagine would happen with a Cargo workspace (correct me where I am wrong, or think it should work differently):

foo/
├── Cargo.toml
├── bar/
│   └── Cargo.toml
└── baz/
    └── Cargo.toml

A single target cargo_workspace("foo") is specified for the workspace. Other targets for workspace members bar and baz, are automatically derived from the root Cargo.toml.

The following goals / targets exist:

• krakenx build foo: builds the entire workspace (cargo build at the root)
• krakenx build bar: builds the entire workspace (cargo build in bar)
• krakenx build --release foo: builds the entire workspace (cargo build --release at the root)
• krakenx build --release bar: builds the entire workspace (cargo build --release in bar)
• krakenx check foo: checks the entire workspace (eg. cargo clippy, cargo deny, ... at the root)
• krakenx check bar: checks the entire workspace (eg. cargo clippy, cargo deny, ... in bar)
• krakenx test foo: tests the entire workspace (cargo test at the root)
• krakenx test bar: tests the entire workspace (cargo test in bar)
• ...

Does that sound right?