CHANGELOG.md

Change Log

3.4.0 (2025-01-09)

View commits

3.4.0 (2025-01-09)

View commits

Merged pull requests:

• Use Confluent MockSchemaRegistry #269 (@philipp94831)
• Upgrade large-message to 2.9 #268 (@philipp94831)

3.3.0 (2025-01-08)

View commits

Merged pull requests:

• Upgrade to Kafka 3.8 #267 (@philipp94831)

3.2.0 (2025-01-07)

View commits

Merged pull requests:

• Upgrade to Kafka 3.7 #266 (@philipp94831)
• Replace kafka-junit with testcontainers #264 (@philipp94831)
• Fix version in README #262 (@philipp94831)

3.1.0 (2024-12-05)

View commits

Merged pull requests:

• Retain type of Streams/ProducerApp in CLI module #261 (@philipp94831)

3.0.4 (2024-12-03)

View commits

Merged pull requests:

• Do not configure replicas if autoscaling is enabled #260 (@philipp94831)

3.0.3 (2024-11-29)

View commits

Merged pull requests:

• Fix producer app name template #259 (@raminqaf)

3.0.2 (2024-10-29)

View commits

Merged pull requests:

• Add annotations to all Kubernetes resources #258 (@raminqaf)

3.0.1 (2024-08-20)

View commits

Closed issues:

• Change default productive to false #151
• CloseOptions supports timeout #189
• Rename \-\-brokers to \-\-bootstrap\-server #232
• Remove avro dependency #235
• Rename extra-input/output topic to named-input/output topic #231
• Remove guava dependency #236
• Remove dependency on log4j #162
• Separate rclone chart from streams-bootstrap #229
• Investigate the necessity of nameOverride #185
• Distinguish pod and deployment annotations in Helm charts #171
• Migrate from Azure Pipelines to GitHub Actions #139

Merged pull requests:

• Create v3 #250 (@philipp94831)
• Pre-bump version to 3.0.1-SNAPSHOT #257 (@philipp94831)
• Add hook to prepare running of app #256 (@philipp94831)
• Validate persistence mandatory chart values (persistence.size) #255 (@raminqaf)
• Validate autoscaling mandatory fields when it is enabled #254 (@raminqaf)
• Fix Sonarqube issues #253 (@philipp94831)
• Rename Streams section in Helm chart docs #252 (@philipp94831)
• Make CleanUpRunner closeable #247 (@philipp94831)
• Rename TestTopologyFactory #249 (@philipp94831)
• Replace Guava usages #246 (@philipp94831)
• Remove unnecessary schema registry configurations in tests #248 (@philipp94831)
• Add CLI parameter to specify streams application ID #243 (@philipp94831)
• Rename \-\-brokers to \-\-bootstrap\-servers #242 (@philipp94831)
• Rename extra topics to labeled topics #240 (@philipp94831)
• Rename streams section to kafka in Helm charts #241 (@philipp94831)
• Add default serialization config to apps #239 (@philipp94831)
• Remove log4j dependency and debug parameter #238 (@philipp94831)
• Add HostInfo to ImprovedStreamsConfig #230 (@raminqaf)
• Use KAFKA_ as prefix for environment Kafka config #209 (@philipp94831)
• Remove guava dependency #237 (@philipp94831)
• Separate CLI and application #207 (@philipp94831)
• feat: remove rclone #234 (@DerTiedemann)
• fix: add gh actions badge #233 (@DerTiedemann)

2.23.0 (2024-06-12)

View commits

Merged pull requests:

• Add dynamic application.server config to streams app chart #214 (@philipp94831)

2.22.1 (2024-06-11)

View commits

Merged pull requests:

• Use same variable names for clean up job as for main chart #228 (@philipp94831)

2.22.0 (2024-06-10)

View commits

Merged pull requests:

• Support mounting of external secrets #225 (@torbsto)

2.21.0 (2024-05-24)

View commits

Merged pull requests:

• Make affinity in charts more generic #224 (@yannick-roeder)

2.20.1 (2024-05-23)

View commits

Merged pull requests:

• Roll deployments on config map updates #227 (@philipp94831)
• Fix content of config maps #226 (@philipp94831)

2.20.0 (2024-05-23)

View commits

Merged pull requests:

• Mount arbitrary files to apps #223 (@philipp94831)

2.19.2 (2024-05-14)

View commits

Merged pull requests:

• Fix indentation of Rclone annotations #222 (@philipp94831)
• Bump CI template version to fix Helm lint #221 (@yannick-roeder)

2.19.1 (2024-05-13)

View commits

Merged pull requests:

• Fix default values for podLabels and podAnnotations #220 (@philipp94831)

2.19.0 (2024-05-13)

View commits

Merged pull requests:

• Remove awsRole from helm charts #219 (@philipp94831)
• Add Values.podLabels and Values.podAnnotations #217 (@vostres)
• Fix naming of helm releases #216 (@philipp94831)

2.18.1 (2024-05-08)

View commits

Merged pull requests:

• Shorten log4j-config-volume to log4j #218 (@raminqaf)
• Add terminationGracePeriodSeconds #212 (@vostres)
• Prevent closing of reused admin clients #213 (@philipp94831)
• Upgrade fluent kafka streams to 2.13.1 #215 (@philipp94831)
• Publish snapshots for Helm charts #211 (@philipp94831)
• Update CI templates #210 (@philipp94831)

2.18.0 (2024-04-17)

View commits

Merged pull requests:

• Bump fluent-kafka-streams-tests #208 (@JakobEdding)

2.17.0 (2024-03-01)

View commits

Merged pull requests:

• Upgrade to Kafka 3.6 #206 (@philipp94831)
• Migrate azure pipeline to GH-action #191 (@MichaelKora)
• Improve CI caching #205 (@philipp94831)
• Upgrade to Gradle 8.4 #204 (@philipp94831)

2.16.1 (2024-02-02)

View commits

Closed issues:

• Fix secretRefs for cleanup job #202

Merged pull requests:

• Fix secretRefs for cleanup jobs #203 (@mkcode92)
• Fix Jacoco for Java 17 #201 (@philipp94831)

2.16.0 (2024-01-26)

View commits

Merged pull requests:

• Add missing default values for internal topics #200 (@philipp94831)
• Simplify configuration of internal topics for autoscaling #198 (@philipp94831)
• Build in CI using JDK 17 #199 (@philipp94831)

2.15.3 (2024-01-09)

View commits

Merged pull requests:

• Update dependencies #196 (@philipp94831)

2.15.2 (2024-01-08)

View commits

Merged pull requests:

• Update dependencies #195 (@philipp94831)

2.15.1 (2023-12-20)

View commits

Merged pull requests:

• Add slf4j dependency #194 (@philipp94831)

2.15.0 (2023-11-10)

View commits

Merged pull requests:

• Upgrade to Kafka 3.5 #192 (@philipp94831)

2.14.0 (2023-10-30)

View commits

Merged pull requests:

• Add modules for easy usage with Fluent Kafka Streams Tests and Large Message SerDe #188 (@philipp94831)

2.13.0 (2023-09-21)

View commits

Merged pull requests:

• Make Schema Registry URL optional #187 (@raminqaf)

Fixed bugs:

• Cleanup job fails if topic doesn't have associated schema #186

2.12.1 (2023-08-15)

View commits

Closed issues:

• Upgrade to Kafka 3.4 #172

Merged pull requests:

• Switch to official Rclone image #176 (@philipp94831)
• Upgrade to Kafka 3.4 #184 (@torbsto)

2.12.0 (2023-05-10)

View commits

Merged pull requests:

• Create method to register topic clean up hooks #183 (@philipp94831)
• Fail streams-app templating if no topics are provided for ScaledObject triggers #182 (@JakobEdding)

2.11.2 (2023-04-19)

View commits

Merged pull requests:

• Make brokers and schema registry url optional chart configs #181 (@philipp94831)

2.11.1 (2023-04-17)

View commits

Fixed bugs:

• Fix service selector to match pod labels #180 (@raminqaf)

2.11.0 (2023-04-14)

View commits

Merged pull requests:

• Reference existing secrets from helm charts #179 (@philipp94831)
• Support deployment of bootstrap application with an HTTP server #178 (@torbsto)
• Support additional triggers for KEDA autoscaling #177 (@torbsto)

2.10.0 (2023-03-15)

View commits

Merged pull requests:

• Producer App can be deployed as k8s Deployment #175 (@mkcode92)
• Remove duplicate of changelog config #174 (@yannick-roeder)

2.9.0 (2023-02-21)

View commits

Merged pull requests:

• Fix casing of consumerGroup value #173 (@disrupted)

2.8.0 (2023-01-24)

View commits

Merged pull requests:

• Upgrade to Kafka 3.3.2 #169 (@philipp94831)

2.7.0 (2023-01-03)

View commits

Closed issues:

• Support new KafkaStreams close options #163

Merged pull requests:

• Update dependencies #168 (@philipp94831)
• Improve backwards compatibility with 2.5 and older versions #167 (@philipp94831)
• Send leave group requests in case of streams application shutdown #165 (@philipp94831)
• Fix Helm lint #166 (@philipp94831)
• Use environment config when creating admin client #164 (@philipp94831)

2.6.0 (2022-12-29)

View commits

Closed issues:

• Missing CLI arguments in clean job don't crash the app #105

Merged pull requests:

• Use new CommandLine API #161 (@philipp94831)
• Use the latest version of the workflow #160 (@MichaelKora)

2.5.0 (2022-12-22)

View commits

Merged pull requests:

• Upgrade to Kafka 3.3 #157 (@philipp94831)

2.4.2 (2022-12-21)

View commits

Closed issues:

• Release helm charts to GitHub Pages #140
• Add optional ServiceAccount to Pod specs #154

Merged pull requests:

• Lint and release helm charts #158 (@MichaelKora)
• Remove tgz files and index.yaml #159 (@MichaelKora)
• Add ServiceAccountName option to Helm charts #156 (@yannick-roeder)
• Configure access token for changelog generation #155 (@philipp94831)

2.4.1 (2022-11-29)

View commits

Merged pull requests:

• Clean schema registry subjects of named foreign key joins #153 (@philipp94831)
• Do not configure empty input topics #152 (@philipp94831)

Fixed bugs:

• Chart should not set inputTopics to empty string if none are specified #149

2.4.0 (2022-11-16)

View commits

Merged pull requests:

• Support multiple extra input topics #150 (@philipp94831)

2.3.1 (2022-10-12)

View commits

Merged pull requests:

• Fix discovery of repartition topics #148 (@philipp94831)
• Make input topic accessors public #147 (@philipp94831)

2.3.0 (2022-07-13)

View commits

Merged pull requests:

• Upgrade to Kafka 3.2 and Confluent 7.2 #146 (@philipp94831)

2.2.0 (2022-06-23)

View commits

Merged pull requests:

• Upgrade to Kafka 3.1 and Confluent 7.1 #145 (@philipp94831)

2.1.2 (2022-06-08)

View commits

Merged pull requests:

• Update documentation #144 (@philipp94831)
• Remove default schedule for producer jobs #143 (@philipp94831)
• Fix key-value pairs indentation inside pod.yaml file #141 (@irux)
• Update CronJob apiVersion due to deprecation #142 (@JakobEdding)

2.1.1 (2022-03-02)

View commits

Closed issues:

• Clean up should check if topic deletion was successful #134

Merged pull requests:

• Add more configurations to jobs and cron jobs #136 (@philipp94831)
• Verify deletion of topics #135 (@philipp94831)

2.1.0 (2022-02-21)

View commits

Merged pull requests:

• Add input pattern config #132 (@philipp94831)
• Add imagePullSecrets to streams-app helm chart #133 (@raminqaf)
• Fix clean for pattern subscriptions #131 (@philipp94831)
• Add option for streams state persistence #130 (@philipp94831)

2.0.0 (2022-01-13)

View commits

Closed issues:

• Upgrade Kafka Streams to 3.0 #126

Merged pull requests:

• Update to Kafka 3.0.0 #129 (@philipp94831)

1.9.3 (2021-12-20)

View commits

Merged pull requests:

• Update log4j to 2.17.0 #128 (@philipp94831)

1.9.2 (2021-12-16)

View commits

Merged pull requests:

• Update log4j to 2.16.0 #127 (@philipp94831)

1.9.1 (2021-12-10)

View commits

Merged pull requests:

• Update log4j to 2.15.0 #125 (@philipp94831)

1.9.0 (2021-09-14)

View commits

Closed issues:

• Restart failed clean up jobs rather than creating new pods #94
• Fix vulnerable dependencies #114

Merged pull requests:

• Upgrade to Kafka 2.8 #124 (@philipp94831)
• Add workaround for rclone config permissions #123 (@yannick-roeder)
• Fix rclone volume permissions #122 (@yannick-roeder)
• Restart failed cleanup jobs instead of creating new pods #120 (@yannick-roeder)
• Update openbridge/ob_bulkstash version and allow arguments #121 (@aqchen)
• Downgrad Gradle to 6.7.1 #119 (@philipp94831)

1.8.0 (2021-09-03)

View commits

Merged pull requests:

• Migrate to Log4j 2 #115 (@philipp94831)
• Suspend rclone cron jobs #117 (@philipp94831)
• Auto-scale stateful sets #116 (@philipp94831)

1.7.1 (2021-08-20)

View commits

Closed issues:

• Clean schemas of pseudo topics created when performing a foreign key join #112
• Extend the Readme #99

Merged pull requests:

• Clean schemas of pseudo topics created when performing a foreign key join #113 (@philipp94831)
• Support KEDA idleReplicaCount #109 (@philipp94831)
• producer chart: job/cronjob switch #110 (@mkcode92)
• Make producer-app's CronJob schedule optional #108 (@torbsto)
• Fix consumer group annotation #107 (@philipp94831)
• Fix nil warnings about empty yaml lists in value files #106 (@yannick-roeder)
• Add node affinity to helm chart #103 (@yannick-roeder)
• Reset schema of internal topics when they are deleted #104 (@torbsto)
• Extend the readme #100 (@VictorKuenstler)

Fixed bugs:

• Add serviceName for StatefulSet #102 (@disrupted)

1.7.0 (2021-02-18)

View commits

Closed issues:

• Update to Kafka 2.6/Confluent 6.0 #96
• Add support for tolerations in streams helm charts #86

Merged pull requests:

• Update Kafka to 2.7 #98 (@philipp94831)
• Add priority class to helm charts #97 (@yannick-roeder)
• Add consumer group annotation from autoscaling #95 (@VictorKuenstler)

1.6.0 (2021-01-25)

View commits

Merged pull requests:

• Rename to streams-bootstrap #93 (@philipp94831)

1.5.5 (2021-01-21)

View commits

Merged pull requests:

• Expose AdminClient from CleanUpRunner #92 (@philipp94831)
• Upgrade to helm 3 #91 (@yannick-roeder)
• Reduce helm chart size #90 (@philipp94831)

1.5.4 (2020-12-18)

View commits

Merged pull requests:

• Add producer application #89 (@philipp94831)
• Add tolerations to deployment #87 (@yannick-roeder)

1.5.3 (2020-11-24)

View commits

Merged pull requests:

• Do not delete consumer group if it does not exist #85 (@philipp94831)

1.5.2 (2020-11-23)

View commits

Merged pull requests:

• Delete consumer groups on delete output #84 (@philipp94831)
• Fix comment for autoscaling #83 (@yannick-roeder)
• Add scaled object for autoscaling with keda #82 (@yannick-roeder)

1.5.1 (2020-11-03)

View commits

Merged pull requests:

• Add extra input topics to streams app #81 (@philipp94831)
• Remove container suffix from container name #73 (@yannick-roeder)

Fixed bugs:

• Container can not be launched if name exceeds 63 characters #69

1.5.0 (2020-09-29)

View commits

Merged pull requests:

• Add labels, annotations and configurable name to rclone CronJob #80 (@philipp94831)
• Update Kafka to 2.5.1 and Confluent to 5.5.1 #79 (@philipp94831)

1.4.6 (2020-08-26)

View commits

Merged pull requests:

• Close resources after streams client #78 (@philipp94831)

1.4.5 (2020-08-06)

View commits

Closed issues:

• Use inheritance for streams helm charts #71

Merged pull requests:

• Make helm chart backwards compatible #77 (@philipp94831)
• Specify additional output topics via default CLI #75 (@philipp94831)
• Trim rclone job name #76 (@philipp94831)
• Add annotations to clean up job #74 (@philipp94831)
• Remove duplicated streams chart #72 (@philipp94831)
• Fix yaml file for kubernetes > 1.16 #70 (@VictorKuenstler)
• Make JAVA_TOOL_OPTIONS configurable #68 (@philipp94831)

1.4.4 (2020-04-14)

View commits

Merged pull requests:

• Close resources on streams creation error #67 (@philipp94831)
• Only run streams resetter for existing topics #66 (@philipp94831)
• Use Kubernetes secrets for password parameters #64 (@yannick-roeder)
• Merge streams chart for statefulset and deployment #65 (@philipp94831)

1.4.3 (2020-04-02)

View commits

Implemented enhancements:

• Override log4j.properties when using provided #62 (@b-feldmann)

Merged pull requests:

• Revert PicoCli api usage to exit application properly #63 (@philipp94831)

1.4.2 (2020-03-24)

View commits

Merged pull requests:

• Run streams resetter for external source topics and intermediate topics #61 (@philipp94831)
• Add helm chart to deploy streams app as statefulset with static group membership #60 (@philipp94831)

1.4.1 (2020-03-13)

View commits

Merged pull requests:

• Allow specification of boolean flags without any value #59 (@philipp94831)
• Add topic client #58 (@philipp94831)
• Do not exit application #57 (@philipp94831)

1.4.0 (2020-03-10)

View commits

Merged pull requests:

• Remove DeadLetter #56 (@philipp94831)
• Move error handlers to com.bakdata.kafka:error-handling #55 (@philipp94831)

1.3.4 (2020-03-10)

View commits

Merged pull requests:

• Parse streams config from separate environment variables #54 (@philipp94831)

1.3.3 (2020-02-27)

View commits

Merged pull requests:

• Exit application with proper code #53 (@philipp94831)

1.3.2 (2020-02-24)

View commits

Merged pull requests:

• Check for errors when running streams resetter #52 (@philipp94831)

1.3.1 (2020-01-30)

View commits

Merged pull requests:

• Add hook for Streams state transitions #51 (@philipp94831)
• Fix parsing of environment parameters with ENV_PREFIX in name #50 (@philipp94831)

1.3.0 (2020-01-29)

View commits

Merged pull requests:

• Update Kafka to 2.4.0 #49 (@philipp94831)

1.2.1 (2020-01-10)

View commits

Merged pull requests:

• Configure StreamsResetter with application properties #48 (@torbsto)
• Support schema registry authentication #47 (@philipp94831)

1.2.0 (2020-01-08)

View commits

Implemented enhancements:

• Call close after application clean up #46 (@philipp94831)
• Apply overridden Kafka config before CLI passed configuration #44

Merged pull requests:

• Prioritize Kafka Streams Config passed via CLI over overridden properties #45 (@philipp94831)
• Add custom annotations #43 (@SvenLehmann)
• Improve values.yaml structure #42 (@SvenLehmann)
• Allow custom labels for jobs and deployments #41 (@SvenLehmann)
• Reset internal topics #40 (@torbsto)

1.1.13 (2019-11-15)

View commits

Merged pull requests:

• Classify all Kafka errors as recoverable bar some exceptions #38 (@philipp94831)
• Make PodAffinity rule configurable #37 (@SvenLehmann)
• Add flat value transformers for error handling #36 (@philipp94831)

1.1.12 (2019-11-07)

View commits

Merged pull requests:

• Fix schema registry clean up #35 (@philipp94831)

1.1.11 (2019-11-05)

View commits

Merged pull requests:

• Improve code quality #34 (@philipp94831)
• Add common error handlers #33 (@philipp94831)

1.1.10 (2019-10-31)

View commits

Merged pull requests:

• Delete schemas during clean up #32 (@torbsto)
• Add a helm chart for running Streams App as a job #31 (@SvenLehmann)

1.1.9 (2019-10-25)

View commits

Merged pull requests:

• Delete error topic if requested #30 (@SvenLehmann)

1.1.8 (2019-10-17)

View commits

Merged pull requests:

• Expose delete topic method to subclasses #29 (@philipp94831)

1.1.7 (2019-10-16)

View commits

Merged pull requests:

• Change reprocessing to clean up #26 (@torbsto)

1.1.6 (2019-10-11)

View commits

Merged pull requests:

• Add utility method for single input topic #28 (@torbsto)

1.1.5 (2019-10-08)

View commits

Merged pull requests:

• Allow multiple input topics and add an error topic #27 (@torbsto)

1.1.4 (2019-09-11)

View commits

Closed issues:

• External stream configuration parameter cannot handle primitive data types other than String #23

Merged pull requests:

• Add hook for registering an uncaught exception handler #25 (@philipp94831)
• Add rclone chart #24 (@lawben)
• Allow kafka streams configuration with external parameter #22 (@fapaul)
• Add JMX prometheus #21 (@lawben)

1.1.3 (2019-08-01)

View commits

Implemented enhancements:

• Add CLI parameter to allow reprocessing of data #14 (@SvenLehmann)

Merged pull requests:

• Release new version of kafka-streams chart #20 (@fapaul)
• Change clean up visibility #19 (@fapaul)
• Change reset visibility #18 (@fapaul)
• Initialize topolgy stream before cleanup #17 (@fapaul)
• Add cleanup possibility on processor startup #15 (@fapaul)
• Fix duplicate in app name #16 (@lawben)
• Change log level on debug for child app in different package #13 (@fapaul)
• Add unique AppID method #12 (@lawben)
• Log initial configuration on debug #11 (@fapaul)

1.1.2 (2019-06-27)

View commits

Merged pull requests:

• Add support for AWS roles and pod resources #10 (@SvenLehmann)
• Update readme with latest release version number #9 (@fapaul)

1.1.1 (2019-05-24)

View commits

Merged pull requests:

• Add log4j binding for sl4j #8 (@fapaul)
• Set arity for boolean options to 1 to match environment key value pairs #7 (@fapaul)
• Comply default stream application name schema with kube dns #6 (@fapaul)

1.1.0 (2019-05-14)

View commits

Merged pull requests:

• Remove default custom environment variables ingestion from values.yaml #5 (@lawben)
• Add default log properties #4 (@lawben)
• Bakdata Kafka Streams Helm Repository #3 (@fapaul)

1.0.1 (2019-03-27)

View commits

1.0.0 (2019-03-13)

View commits

Closed issues:

• Extract common Kafka Streams parts #1

Merged pull requests:

• Kafka streams application #2 (@fapaul)