Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add python_project() function #192

Draft
wants to merge 88 commits into
base: develop
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
88 commits
Select commit Hold shift + click to select a range
7f7a145
remove dev-dependencies except for pytest for kraken-wrapper because …
NiklasRosenstein Feb 1, 2024
cecb34c
feature: Introduce `kraken.std.python.project.python_project()` funct…
NiklasRosenstein Feb 2, 2024
14ecfa0
address nits
NiklasRosenstein Feb 2, 2024
5fefdcf
fix
NiklasRosenstein Feb 2, 2024
a272011
fix example
NiklasRosenstein Feb 2, 2024
dbe33bb
use `python_project()` for pdm-project example
NiklasRosenstein Feb 2, 2024
05d3ef6
update
NiklasRosenstein Feb 2, 2024
e6b95f4
set mypy_path config
NiklasRosenstein Feb 2, 2024
84d599c
fix changelog
NiklasRosenstein Feb 2, 2024
84291c4
breaking change: Default project groups `lint` now only depends on `c…
NiklasRosenstein Feb 2, 2024
4bd6ff5
mypy fixes
NiklasRosenstein Feb 2, 2024
f79fe30
remove type ignore
NiklasRosenstein Feb 2, 2024
656f270
add import-untyped back
NiklasRosenstein Feb 2, 2024
d4e3886
update
NiklasRosenstein Feb 2, 2024
5e41c7c
now?
NiklasRosenstein Feb 2, 2024
9e8bfdd
Merge remote-tracking branch 'origin/develop' into nr/python-project
NiklasRosenstein Feb 27, 2024
b9807d7
add mising logging import after merge
NiklasRosenstein Feb 27, 2024
6a2098a
fix pyupgrade() factory function which was not considering source dir…
NiklasRosenstein Feb 27, 2024
0e444cc
implement missing publish logic
NiklasRosenstein Feb 27, 2024
833c78d
remove linter config from pyproject.toml
NiklasRosenstein Feb 27, 2024
df57e6b
pass line length to flake and virtual-env/src paths to isort
NiklasRosenstein Feb 27, 2024
ea3c688
use python_project() for kraken itself
NiklasRosenstein Feb 27, 2024
8a34fa8
fix mypy path
NiklasRosenstein Feb 27, 2024
0110a94
Merge remote-tracking branch 'origin/develop' into nr/python-project
NiklasRosenstein Mar 5, 2024
d6271ef
improvement: Raise `EmptyGitRepositoryError` and `NotAGitRepositoryEr…
NiklasRosenstein Mar 6, 2024
a1fe14f
fix passing pyupgrade_bin to the format task
NiklasRosenstein Mar 6, 2024
fc67b02
Merge remote-tracking branch 'origin/develop' into nr/python-project
NiklasRosenstein Mar 6, 2024
23917e1
improvement: Add order-only dependencies between check and format tas…
NiklasRosenstein Mar 6, 2024
db84932
improvement: `build_docker_image(dockerfile)` now treats the path rel…
NiklasRosenstein Mar 10, 2024
7c5462f
fix
NiklasRosenstein Mar 10, 2024
130a9b8
Merge remote-tracking branch 'origin' into nr/python-project
NiklasRosenstein Mar 17, 2024
7e7ffdd
update .kraken.py, fix mypy, black 24 fmt
NiklasRosenstein Mar 17, 2024
30a4d38
disable krakenw-based tests for now and re-instate selftests
NiklasRosenstein Mar 17, 2024
4f1ad4e
add mitmproxy back as dev dependency
NiklasRosenstein Mar 17, 2024
b2a6bf5
fix slap-project and make it use python_project()
NiklasRosenstein Mar 17, 2024
e1560b3
add `python_project()` and `python_package_index()` to `kraken.std` a…
NiklasRosenstein Mar 17, 2024
7713dfa
use python_project() for all examples
NiklasRosenstein Mar 17, 2024
ce301b2
bump python versions to 3.10 (Mypy will complain in its latest versio…
NiklasRosenstein Mar 17, 2024
9b90ee9
update python version requirement in more example projects
NiklasRosenstein Mar 17, 2024
7e15977
improvement: Override `MaturinPoetryPyprojectHandler.get_python_versi…
NiklasRosenstein Mar 17, 2024
1e2a456
feature: Add `pex_set_global_store_path()` function
NiklasRosenstein Mar 17, 2024
57ac00b
also lint consumer project in python integration test
NiklasRosenstein Mar 17, 2024
f441c7b
feature: Add `python_app()` function to build a PEX from your Python …
NiklasRosenstein Mar 17, 2024
a5ecc69
update
NiklasRosenstein Mar 18, 2024
6510ee2
fix
NiklasRosenstein Mar 18, 2024
db6a761
Merge remote-tracking branch 'origin/develop' into nr/python-project
NiklasRosenstein Mar 19, 2024
f0dc429
Protobuf
NiklasRosenstein Mar 19, 2024
33f11fe
Add Protobuf support for `python_project()` and add `codegen` paramet…
NiklasRosenstein Mar 19, 2024
5e9fffd
make `python_app()` automatically depend on codegen produced by `pyth…
NiklasRosenstein Mar 19, 2024
28f1be7
delete contents of proto output directory before regenerating
NiklasRosenstein Mar 19, 2024
2a2f7b2
also generate grpc code in python projects
NiklasRosenstein Mar 19, 2024
c48d749
update Protobuf docs
NiklasRosenstein Mar 19, 2024
d324744
update protobuf docs;
NiklasRosenstein Mar 20, 2024
69323c0
add examples/python-protobuf
NiklasRosenstein Mar 20, 2024
09e2583
feature: Add `kraken.std.util.fetch_tarball()`
NiklasRosenstein Mar 20, 2024
3e7b64e
Merge remote-tracking branch 'origin/develop' into nr/python-project
NiklasRosenstein Mar 21, 2024
f0b9da6
breaking change: Add `enable_error_code = "ignore-without-code, possi…
NiklasRosenstein Mar 21, 2024
d9826aa
improvement: `Supplier.of()` and related functions now take an `Itera…
NiklasRosenstein Mar 21, 2024
d8dd163
feature: Add `kraken.build.experimental.functional.v1` API with commo…
NiklasRosenstein Mar 21, 2024
72dc73d
feature: Install Buffrs CLI from GitHub releases
NiklasRosenstein Mar 21, 2024
ee5334f
improvement: `buffrs.install` task now produces a `.gitignore` file i…
NiklasRosenstein Mar 21, 2024
92d62e5
improvement: Pass a default `buf.yaml` configuration to `buf lint` to…
NiklasRosenstein Mar 21, 2024
55655ab
we moved fetch_tarball()
NiklasRosenstein Mar 21, 2024
7233e1a
fix buf import
NiklasRosenstein Mar 21, 2024
760c38b
fix: Remove default files that should be excluded for `isort`, `pycln…
NiklasRosenstein Mar 21, 2024
203040b
fmt on kraken-build/src/kraken/build/
NiklasRosenstein Mar 21, 2024
303549d
move new python_project() etc. to `kraken.build.python.v1alpha1` and …
NiklasRosenstein Mar 22, 2024
8891380
rename functional to utils
NiklasRosenstein Mar 22, 2024
73940aa
fix additional error code for mypy
NiklasRosenstein Mar 22, 2024
1275062
update docs
NiklasRosenstein Mar 22, 2024
bf06379
update examples;
NiklasRosenstein Mar 22, 2024
f082237
add back calling do
NiklasRosenstein Mar 22, 2024
88857ad
feature: Add `Supplier.agg()`
NiklasRosenstein Mar 22, 2024
1803e03
feature: Add `kraken.build.protobuf.v1alpha1.protobuf_project()`
NiklasRosenstein Mar 22, 2024
6b483fb
upodate example
NiklasRosenstein Mar 22, 2024
11b3cc5
Merge remote-tracking branch 'origin/develop' into nr/python-project
NiklasRosenstein Mar 29, 2024
1c882d1
fmt
NiklasRosenstein Mar 30, 2024
03f5425
feature: add `pex_build(python_shebang)` parameter
NiklasRosenstein Apr 3, 2024
0cd6dc5
add `python_app(python_shebang)` parameter
NiklasRosenstein Apr 3, 2024
075930a
feature: Add `pex_build(python_versions)`
NiklasRosenstein Apr 3, 2024
a3b5809
improvement: Improve the `--cache-to` argument produced by the `Build…
NiklasRosenstein Apr 4, 2024
a618c0d
Merge remote-tracking branch 'origin/develop' into nr/python-project
NiklasRosenstein Apr 10, 2024
24981d3
Merge remote-tracking branch 'origin/develop' into nr/python-project
NiklasRosenstein Sep 3, 2024
fa25491
fmt and fix most lints post merge develop
NiklasRosenstein Sep 3, 2024
dfaf3b2
move changelog entries
NiklasRosenstein Sep 3, 2024
9cc0d37
Add `python_project(enable)` option to select the tools and features
NiklasRosenstein Sep 3, 2024
f63f8e4
fmt
NiklasRosenstein Sep 3, 2024
1ced3cc
fix: dont include +dirty in Python version metadata derived from Git …
NiklasRosenstein Sep 7, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
161 changes: 161 additions & 0 deletions .changelog/_unreleased.toml
Original file line number Diff line number Diff line change
@@ -0,0 +1,161 @@
[[entries]]
id = "d9440e6f-b19b-4b49-bad4-dc7860228cf8"
type = "improvement"
description = "Add order-only dependencies between check and format tasks created for Python projects"
author = "@NiklasRosenstein"

[[entries]]
id = "84e8fb46-7aef-4dfc-b4ec-5ad507c82544"
type = "improvement"
description = "`build_docker_image(dockerfile)` now treats the path relative to the current project"
author = "@NiklasRosenstein"

[[entries]]
id = "788398a7-320e-45b2-9590-d977387c2ed1"
type = "improvement"
description = "Override `MaturinPoetryPyprojectHandler.get_python_version_constraint()` to fall back on `[project.version]` key if `[tool.poetry.version]` is not set"
author = "@NiklasRosenstein"

[[entries]]
id = "ef21ce61-3e39-4177-9182-72c67e80be63"
type = "feature"
description = "Add `pex_set_global_store_path()` function"
author = "@NiklasRosenstein"

[[entries]]
id = "79e9748a-4d81-4636-be2e-3772b150c976"
type = "feature"
description = "Add `python_app()` function to build a PEX from your Python project."
author = "@NiklasRosenstein"

[[entries]]
id = "897d94dc-79e7-45bf-abba-bd346a3fb7d5"
type = "feature"
description = "Add `PexBuildTask.always_rebuild` property and add `pex_build(always_rebuild, output_file)` parameters"
author = "@NiklasRosenstein"

[[entries]]
id = "1ae440c5-cd7a-4fdc-a6a9-da10165fa260"
type = "feature"
description = "Add `ProtocTask` to `kraken.std.protobuf`"
author = "@NiklasRosenstein"

[[entries]]
id = "691608ae-68bd-45d6-abbb-bb6a1b7200eb"
type = "breaking change"
description = "Remove `buf_lint()` and `buf_format()` helper functions from `kraken.std.protobuf`"
author = "@NiklasRosenstein"

[[entries]]
id = "07fb65a6-fe5a-420a-ad96-36a50714dee1"
type = "feature"
description = "Add `kraken.std.util.fetch_tarball()`"
author = "@NiklasRosenstein"

[[entries]]
id = "ec6bf736-6a6e-4625-9703-89bff7094d56"
type = "breaking change"
description = "Add `enable_error_code = \"ignore-without-code, possibly-undefined\"` to default Mypy settings"
author = "@NiklasRosenstein"

[[entries]]
id = "33f61844-e82b-40da-aba9-f552142d6abd"
type = "improvement"
description = "`Supplier.of()` and related functions now take an `Iterable[Supplier[Any]]` as the `derived_from` parameter instead of a `Sequence` as it is converted to a `tuple` later anyway."
author = "@NiklasRosenstein"

[[entries]]
id = "6bddf45d-7195-42d9-a490-2a567371d7f4"
type = "feature"
description = "Add `kraken.build.experimental.functional.v1` API with common tasks for operations in a functional manner (fetching a file, tarball, writing a file and executing a shell command)"
author = "@NiklasRosenstein"

[[entries]]
id = "640c9b5a-e459-429c-8aa1-b92430cb5065"
type = "feature"
description = "Install Buffrs CLI from GitHub releases"
author = "@NiklasRosenstein"

[[entries]]
id = "6cd1582c-d975-4888-809c-89cd7d66073c"
type = "improvement"
description = "`buffrs.install` task now produces a `.gitignore` file in the `proto/vendor` directory to exclude it"
author = "@NiklasRosenstein"

[[entries]]
id = "f8a491fb-daea-4279-b5d6-eb15f73a847b"
type = "breaking change"
description = "Remove `BufLintTask` and `BufInstallTask` in favor of `buf()` function that uses our new `kraken.build.experimental.functional.v1` API to implement the same functionality more streamlined"
author = "@NiklasRosenstein"

[[entries]]
id = "7d264f42-d0c1-4077-9fbb-c869ab093b46"
type = "improvement"
description = "Pass a default `buf.yaml` configuration to `buf lint` to disable `PACKAGE_VERSION_SUFFIX` and `PACKAGE_DIRECTORY_MATCH` lint rules for compatibility with Buffrs."
author = "@NiklasRosenstein"

[[entries]]
id = "a8814dcb-b28b-42ed-aa0b-92297a63877d"
type = "fix"
description = "Remove default files that should be excluded for `isort`, `pycln` and `black` so that a `build/` directory in the `src/` directory also gets checked. We can do this safely because we expect the `src/` folder in a Python project to contain only files that should be checked unless otherwise explicitly specified"
author = "@NiklasRosenstein"

[[entries]]
id = "863c7431-cfb0-4491-a6c1-7209a6e2cf59"
type = "feature"
description = "Add `Supplier.agg()`"
author = "@NiklasRosenstein"

[[entries]]
id = "16b2de57-29c2-410f-816a-018e9812871b"
type = "feature"
description = "Add `kraken.build.protobuf.v1alpha1.protobuf_project()`"
author = "@NiklasRosenstein"

[[entries]]
id = "54f6e294-e60f-402a-97ae-17cbce9e8bc6"
type = "feature"
description = "add `pex_build(python_shebang)` parameter"
author = "[email protected]"

[[entries]]
id = "9ac5c38b-cb7c-4aa2-bd01-5cf3601dc953"
type = "feature"
description = "Add `pex_build(python_versions)`"
author = "@NiklasRosenstein"

[[entries]]
id = "2b5d4590-5c9b-4fd0-a15f-87d93adb6e7e"
type = "improvement"
description = "Improve the `--cache-to` argument produced by the `BuildxBuildTask.cache_repo` property to include `image-manifest=true,oci-mediatypes=true` options for better OCI registry support (see https://aws.amazon.com/blogs/containers/announcing-remote-cache-support-in-amazon-ecr-for-buildkit-clients/)"
author = "@NiklasRosenstein"

[[entries]]
id = "5a4c9349-429d-4594-805b-404ed45020c3"
type = "improvement"
description = "Expose the `locked` property of `CargoBuildTask` via the `cargo_build` helper function"
author = "[email protected]"

[[entries]]
id = "4560750d-452d-43f0-af08-d791aaf5134a"
type = "improvement"
description = "Upgrade keyring to v25"
author = "@NiklasRosenstein"

[[entries]]
id = "fb3fcc4a-a38a-445f-b124-571395b7ac86"
type = "feature"
description = "Introduce `kraken.std.python.project.python_project()` function which creates all tasks for a Python project."
author = "[email protected]"

[[entries]]
id = "ae70e553-81c2-4eb2-a6f3-210ca6c24992"
type = "breaking change"
description = "Default project groups `lint` now only depends on `check` for order instead of strictly"
author = "[email protected]"

[[entries]]
id = "4cb0fe80-d359-4302-8893-c0fd62bf0631"
type = "improvement"
description = "Raise `EmptyGitRepositoryError` and `NotAGitRepositoryError` respectively in `git_describe()`"
author = "@NiklasRosenstein"
92 changes: 2 additions & 90 deletions .kraken.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,96 +2,8 @@

buildscript(requirements=["kraken-build>=0.33.2"])

import os

from kraken.build import project
from kraken.std import python
from kraken.std.git import git_describe, gitignore


def configure_project() -> None:
from kraken.build import project

python.ruff(additional_args=["--exclude", "tests/iss-263/example_project"])
python.mypy(additional_args=["--exclude", "src/tests/integration/.*/data/.*"], version_spec="==1.8.0")

if project.directory.joinpath("tests").is_dir():
# Explicit list of test directories, Pytest skips the build directory if not specified explicitly.
if project.directory.name == "kraken-build":
python.pytest(ignore_dirs=["src/tests/integration"], include_dirs=["src/kraken/build"])
elif project.directory.name == "kraken-wrapper":
python.pytest(doctest_modules=False)

if project.directory.joinpath("tests/integration").is_dir():
python.pytest(
name="pytestIntegration",
tests_dir="src/tests/integration",
ignore_dirs=["src/tests/integration/python/data"],
group="integrationTest",
)

python.install()
python.info()

(
python.python_settings()
.add_package_index(
"pypi",
credentials=(os.environ["PYPI_USER"], os.environ["PYPI_PASSWORD"]) if "PYPI_USER" in os.environ else None,
)
.add_package_index(
"testpypi",
credentials=(os.environ["TESTPYPI_USER"], os.environ["TESTPYPI_PASSWORD"])
if "TESTPYPI_USER" in os.environ
else None,
is_package_source=False,
)
)

do_publish = True
as_version: str | None = None
is_release = False
if "CI" in os.environ:
if os.environ["GITHUB_REF_TYPE"] == "tag":
# TODO (@NiklasRosenstein): It would be nice to add a test that checks if the version in the package
# is consistent (ie. run `slap release --validate <tag>`).
is_release = True
as_version = os.environ["GITHUB_REF_NAME"]
elif os.environ["GITHUB_REF_TYPE"] == "branch":
if os.environ["GITHUB_REF_NAME"] == "develop":
as_version = python.git_version_to_python(git_describe(project.directory), include_sha=False)
else:
# NOTE (@NiklasRosenstein): PyPI/TestPypi cannot use PEP 440 local versions (which the version with
# included SHA would qualify as), so we don't publish from branches at all.
do_publish = False
else:
raise EnvironmentError(
f"GITHUB_REF_TYPE={os.environ['GITHUB_REF_TYPE']}, GITHUB_REF_NAME={os.environ['GITHUB_REF_NAME']}"
)
else:
do_publish = False
as_version = python.git_version_to_python(git_describe(project.directory), include_sha=False)

build_task = python.build(as_version=as_version)

if do_publish:
testpypi = python.publish(
name="publishToTestPypi",
package_index="testpypi",
distributions=build_task.output_files,
skip_existing=True,
)
if is_release:
python.publish(
name="publishToPypi",
package_index="pypi",
distributions=build_task.output_files,
after=[testpypi],
)


gitignore()
project.subproject("docs")
for subproject in [project.subproject("kraken-build"), project.subproject("kraken-wrapper")]:
with subproject.as_current():
configure_project()
project.subproject("kraken-build")
project.subproject("kraken-wrapper")
5 changes: 5 additions & 0 deletions docs/docs/api/kraken.build.python.v1alpha1.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
title: kraken.build.python.v1alpha1
---

::: kraken.build.python.v1alpha1
5 changes: 5 additions & 0 deletions docs/docs/api/kraken.build.utils.v1alpha1.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
title: kraken.build.utils.v1alpha1
---

::: kraken.build.utils.v1alpha1
Loading
Loading