0.68.0

Highlights

• Stack Components on the Dashboard: We're bringing back stack components. With this release, you will get access to the list of your stack components on the ZenML dashboard. More functionality is going to follow in the next releases.
• Client-Side Caching: Implemented client-side computation for cached steps, significantly reducing time and costs associated with remote orchestrator spin-up.
• Streamlined Onboarding Process: Unified the starter and production setup into a single sequential flow, providing a more intuitive user experience.
• BentoML Integration: Updated to version 1.3.5 with enhanced containerization support.
• Artifact Management: Introduced register_artifact function enabling direct linking of existing data in the artifact store, particularly useful for tools like PyTorch-Lightning that manage their own checkpoints.
• Enhanced Error Handling: Added Error Boundary to visualization components for improved reliability and user experience.

Additional Features and Improvements

• Added multiple access points for deleting pipeline runs
• Improved pipeline detail view functionality
• Improved service account handling for Kaniko image builder

Breaking Changes and Deprecations

• Discontinued Python 3.8 support
• Removed legacy pipeline and step interface
• Removed legacy post execution workflow
• Removed legacy dashboard option
• Removed zenml stack up/down CLI commands
• Removed zenml deploy and zenml <stack-component> deploy
• Removed StepEnvironment class
• Removed the option to specify a specific model version for step output artifacts using the ArtifactConfig class
• Removed the option to use the ExternalArtifact class to load an artifact from a model version
• Removed Client.list_runs, replacing it with Client.list_pipeline_runs
• Removed ArtifactVersionResponse.read, replacing it with ArtifactVersionResponse.load

Documentation Updates

Added new guides for the following topics:

• Kubernetes per-pod configuration
• Factory generation of artifact names
• Common stacks best practices
• Azure 1-click dashboard deployment
• ZenML server upgrade best practices
• Custom Dataset classes and Materializers
• Comprehensive ZenML Pro documentation
• Image building optimization during pipeline runs
• Enhanced BentoML integration documentation

What's Changed

• Release 0.67.0 migration testing by @bcdurak in #3050
• Prevent too large requests by @avishniakov in #3048
• Fix Neptune linting after 1.12.0 release by @avishniakov in #3055
• Fix Lightning Orchestrator (remove -y from pip install) by @wjayesh in #3058
• Fix artifact pruning endpoint path by @schustmi in #3052
• Update python versioning in docs by @avishniakov in #3059
• Fix infinite loop while fetching artifact store in logs storage class by @avishniakov in #3061
• Make sync a setting for sagemaker/azureml orchestrator by @schustmi in #3062
• Remove some deprecated features by @schustmi in #2926
• Fix MySQL warning when filtering pipelines by latest run by @schustmi in #3051
• Remove more deprecated stuff by @schustmi in #3063
• Remove log versions from versioned buckets in S3 by @avishniakov in #3060
• add docs on k8s per pod settings by @wjayesh in #3066
• Remove Python 3.8 support by @strickvl in #3034
• register_artifact function by @avishniakov in #3053
• Fix bad link in docs by @avishniakov in #3069
• Fix model linkage for the lazy loading scenarios by @avishniakov in #3054
• Updating template versions after the Python 3.8 changes by @bcdurak in #3070
• Add UUID materializer by @htahir1 in #3073
• Fix pipeline and model URLs for ZenML Pro on-prem deployments by @stefannica in #3083
• Update bentoml integration to 1.3.5 and add containerization by @wjayesh in #3045
• Fix mlflow linting by @schustmi in #3085
• Add docs for factory generation of artifact names by @strickvl in #3084
• Remove unnecessary metadata fields in UUID materializer test by @htahir1 in #3088
• Client-side computation of cached steps by @schustmi in #3068
• Fix Kaniko image builder service account passing by @schustmi in #3081
• Bugfix in GitLab Code Repository integration by @4gt-104 in #3076
• Add docs on common stacks best practices by @strickvl in #3092
• [docs] Update stacks page and add azure 1-click from dashboard docs by @wjayesh in #3082
• Local development how-to section by @strickvl in #3090
• [docs] best practices for upgrading zenml server by @wjayesh in #3087
• Fix S3 ArtifactStore auth issue by @avishniakov in #3086
• Reduce migration testing runtime by @avishniakov in #3078
• [docs] Dedicated docs on how to skip building an image on pipeline run by @wjayesh in #3079
• Fix filtering by tag for pipeline runs by @schustmi in #3097
• Remove deprecated features: zenml deploy and zenml <stack-component> deploy by @stefannica in #3089
• Do not tag model via Model class on creation by @avishniakov in #3098
• Sagemaker add pipeline tags by @htahir1 in #3080
• [docs] Add custom Dataset classes and Materializers in ZenML by @htahir1 in #3091
• Delete Scarf related scripts and workflow files by @htahir1 in #3103
• Add more detailed docs for ZenML Pro by @wjayesh in #3065
• Add missing code hash filter in client method by @schustmi in #3094
• Remove lineage graph and legacy dashboard support by @schustmi in #3064
• Add unittest to cover gitlab CR regex. by @4gt-104 in #3102
• Automating the release process using Github workflows by @bcdurak in #3101
• Bugfix for release automation by @bcdurak in #3107
• Bugfix for new version in the release automation by @bcdurak in #3108
• using the right parent image name by @bcdurak in #3109
• Making the new release automation scripts executable by @bcdurak in #3110
• Fixing the env variables for the release automation by @bcdurak in #3111
• Adding the right Github configuration before using the gh CLI to fetch the version by @bcdurak in #3112
• Fixing the outputs of the first step in the release automation by @bcdurak in #3113
• Handling github auth and release notes for release automation by @bcdurak in #3114
• Fixing the cloudbuild call for release automation by @bcdurak in #3116
• Fixing the update tenant call in the script by @bcdurak in #3118
• Release automation with the new redeploy logic by @bcdurak in #3120
• Fixing the automation triggers for other branches by @bcdurak in #3125
• Update link for llm-complete-guide repository.- Updated link to poi… by @htahir1 in #3128
• Fixing the migration testing for the release branches by @bcdurak in #3127
• Update pipeline deletion docs by @strickvl in #3123
• Disabling the cache for the quickstart tests by @bcdurak in #3133
• Update Argilla integration for v2.x SDK by @sdiazlor in #2915
• Using pip instead of gh CLI in the migration tests by @bcdurak in #3136
• Adapting tags to work with older versions of Sagemaker by @bcdurak in #3135
• Manual trigger for the release_finalize workflow by @bcdurak in #3137
• Fixing the prepare trigger for the release automation by @bcdurak in #3138

New Contributors

• @4gt-104 made their first contribution in #3076
• @sdiazlor made their first contribution in #2915

Full Changelog: 0.67.0...0.68.0