Merge pull request #6 from Fanatics/producer-issue

Producer issue
Fanatics · May 10, 2022 · 52877c5 · 52877c5
2 parents e705ca9 + 8aa2e03
commit 52877c5
Show file tree

Hide file tree

Showing 62 changed files with 3,704 additions and 930 deletions.
diff --git a/.github/workflows/go.yml b/.github/workflows/go.yml
@@ -6,7 +6,7 @@ jobs:
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        go-version: [1.13, 1.14, 1.15, 1.16, 1.17]
+        go-version: [1.15, 1.16, 1.17]
     steps:
       - name: clean docker cache
         run: |

diff --git a/.github/workflows/project.yml b/.github/workflows/project.yml
@@ -6,7 +6,7 @@ jobs:
     runs-on: ubuntu-latest
     strategy:
       matrix:
-        go-version: [1.13, 1.14, 1.15, 1.16, 1.17]
+        go-version: [1.15, 1.16, 1.17]
     steps:
       - name: Set up Go
         uses: actions/setup-go@v1

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -23,6 +23,87 @@
 
 All notable changes to this project will be documented in this file.
 
+[0.8.1] 2022-03-08
+
+## What's Changed
+* Upgrade beefsack/go-rate by @shubham1172 in https://github.com/apache/pulsar-client-go/pull/735
+* Upgrade prometheus client to 1.11.1 by @nicoloboschi in https://github.com/apache/pulsar-client-go/pull/738
+
+## New Contributors
+@shubham1172 made their first contribution in https://github.com/apache/pulsar-client-go/pull/735
+@nicoloboschi made their first contribution in https://github.com/apache/pulsar-client-go/pull/738
+
+[0.8.0] 2022-02-16
+
+## What's Changed
+* Update release docs with missing information by @cckellogg in https://github.com/apache/pulsar-client-go/pull/656
+* Update change log for 0.7.0 release by @cckellogg in https://github.com/apache/pulsar-client-go/pull/655
+* Update version to 0.7.0 by @cckellogg in https://github.com/apache/pulsar-client-go/pull/654
+* fix issue 650,different handle sequence value by @baomingyu in https://github.com/apache/pulsar-client-go/pull/651
+* Support nack backoff policy for SDK by @wolfstudy in https://github.com/apache/pulsar-client-go/pull/660
+* Remove unused dependency in `oauth2` module by @reugn in https://github.com/apache/pulsar-client-go/pull/661
+* [Issue 662] Fix race in connection.go waitUntilReady() by @bschofield in https://github.com/apache/pulsar-client-go/pull/663
+* Update dependencies by @reugn in https://github.com/apache/pulsar-client-go/pull/665
+* [Issue 652] Quick fixes to the documentation for the main building blocks of the library by @reugn in https://github.com/apache/pulsar-client-go/pull/667
+* Add subscription properties for ConsumerOptions by @wolfstudy in https://github.com/apache/pulsar-client-go/pull/671
+* Add new bug-resistant build constraints by @reugn in https://github.com/apache/pulsar-client-go/pull/670
+* Handle the parameters parsing error in NewProvider by @reugn in https://github.com/apache/pulsar-client-go/pull/673
+* Update email template of release docs by @izumo27 in https://github.com/apache/pulsar-client-go/pull/674
+* Add properties filed for batch container by @wolfstudy in https://github.com/apache/pulsar-client-go/pull/683
+* [Issue 513] Correct apparent logic error in batchContainer's hasSpace() method by @bschofield in https://github.com/apache/pulsar-client-go/pull/678
+* Upgrade DataDog/zstd to v1.5.0 by @dferstay in https://github.com/apache/pulsar-client-go/pull/690
+* fix:add order key to message by @leizhiyuan in https://github.com/apache/pulsar-client-go/pull/688
+* Set default go version to 1.13 in CI related files by @reugn in https://github.com/apache/pulsar-client-go/pull/692
+* [Partition Consumer] Provide lock-free access to compression providers by @dferstay in https://github.com/apache/pulsar-client-go/pull/689
+* Use a separate gorutine to handle the logic of reconnect by @wolfstudy in https://github.com/apache/pulsar-client-go/pull/691
+* [DefaultRouter] add a parallel bench test by @dferstay in https://github.com/apache/pulsar-client-go/pull/693
+* Revert "Use a separate gorutine to handle the logic of reconnect" by @wolfstudy in https://github.com/apache/pulsar-client-go/pull/700
+* Fix data race while accessing connection in partitionProducer by @wolfstudy in https://github.com/apache/pulsar-client-go/pull/701
+* Fix stuck when reconnect broker by @wolfstudy in https://github.com/apache/pulsar-client-go/pull/703
+* Fix slice bounds out of range for readSingleMessage by @wolfstudy in https://github.com/apache/pulsar-client-go/pull/709
+* Encryption failure test case fix by @GPrabhudas in https://github.com/apache/pulsar-client-go/pull/708
+* [DefaultRouter] fix unnecessary system clock reads due to races accessing router state by @dferstay in https://github.com/apache/pulsar-client-go/pull/694
+* Fix negative WaitGroup counter issue by @wolfstudy in https://github.com/apache/pulsar-client-go/pull/712
+* [issue 675] oauth2 use golang-jwt address CVE-2020-26160 by @zzzming in https://github.com/apache/pulsar-client-go/pull/713
+* readme: add note about how to build and test by @pgier in https://github.com/apache/pulsar-client-go/pull/714
+* Bump oauth2 package version to the latest in master by @iorvd in https://github.com/apache/pulsar-client-go/pull/715
+* Fix closed connection leak by @billowqiu in https://github.com/apache/pulsar-client-go/pull/716
+* [Bugfix] producer runEventsLoop for reconnect early exit by @billowqiu in https://github.com/apache/pulsar-client-go/pull/721
+* [issue 679][oauth2] Fix macos compiler warnings by @pgier in https://github.com/apache/pulsar-client-go/pull/719
+* [optimize] Stop batchFlushTicker when Disable batching by @Shoothzj in https://github.com/apache/pulsar-client-go/pull/720
+* Markdown error fix by @Shoothzj in https://github.com/apache/pulsar-client-go/pull/722
+
+## New Contributors
+* @bschofield made their first contribution in https://github.com/apache/pulsar-client-go/pull/663
+* @izumo27 made their first contribution in https://github.com/apache/pulsar-client-go/pull/674
+* @pgier made their first contribution in https://github.com/apache/pulsar-client-go/pull/714
+* @iorvd made their first contribution in https://github.com/apache/pulsar-client-go/pull/715
+* @billowqiu made their first contribution in https://github.com/apache/pulsar-client-go/pull/716
+
+[0.7.0] 2021-10-31
+
+## Feature
+* Encryption support for producer, see [PR-560](https://github.com/apache/pulsar-client-go/pull/560)
+* Decrytion support for consumer, see [PR-612](https://github.com/apache/pulsar-client-go/pull/612)
+* User-defined metric cardinality, see [PR-604](https://github.com/apache/pulsar-client-go/pull/604)
+* Better support for Azure AD OAuth 2.0, see [PR-630](https://github.com/apache/pulsar-client-go/pull/630), [PR-633](https://github.com/apache/pulsar-client-go/pull/633), [PR-634](https://github.com/apache/pulsar-client-go/pull/634)
+* Removed testing for go versions 1.11 and 1.12, see [PR-632](https://github.com/apache/pulsar-client-go/pull/632)
+* Add epoch to create producer to prevent a duplicate producer when broker is not available., see [PR-582] (https://github.com/apache/pulsar-client-go/pull/582)
+
+## Improve
+* Fix batch size limit validation, see [PR-528](https://github.com/apache/pulsar-client-go/pull/528)
+* Fix logic of command for sendError, see [PR-622](https://github.com/apache/pulsar-client-go/pull/622)
+* Drain connection requests channel without closing, see [PR-645](https://github.com/apache/pulsar-client-go/pull/645)
+* Fix ConsumersOpened counter not incremented when use multitopic or regexp consumer, see [PR-619](https://github.com/apache/pulsar-client-go/pull/619)
+* Fix reconnection logic when topic is deleted, see [PR-627](https://github.com/apache/pulsar-client-go/pull/627)
+* Fix panic when scale down partitions, see [PR-601](https://github.com/apache/pulsar-client-go/pull/601)
+* Fix missing metrics for topics by registration of existing collector, see [PR-600](https://github.com/apache/pulsar-client-go/pull/600)
+* Fix producer panic by oldProducers, see [PR-598](https://github.com/apache/pulsar-client-go/pull/598)
+* Fail pending messages when topic is terminated, see [PR-588](https://github.com/apache/pulsar-client-go/pull/588)
+* Fix handle send error panic, see [PR-576](https://github.com/apache/pulsar-client-go/pull/576)
+
+
+
 [0.6.0] 2021-07-21
 
 ## Feature

diff --git a/Dockerfile b/Dockerfile
@@ -17,8 +17,8 @@
 # under the License.
 #
 
-ARG GO_VERSION=golang:1.12
-FROM apachepulsar/pulsar:latest as pulsar
+ARG GO_VERSION=golang:1.15
+FROM apachepulsar/pulsar:2.8.2 as pulsar
 FROM $GO_VERSION as go
 
 RUN apt-get update && apt-get install -y openjdk-11-jre-headless ca-certificates

diff --git a/README.md b/README.md
@@ -36,7 +36,7 @@ CGo based library.
 
 ## Requirements
 
-- Go 1.11+
+- Go 1.15+
 
 ## Status
 
@@ -72,8 +72,9 @@ defer producer.Close()
 
 if err != nil {
     fmt.Println("Failed to publish message", err)
+} else {
+    fmt.Println("Published message")
 }
-fmt.Println("Published message")
 ```
 
 Create a Consumer:
@@ -133,6 +134,16 @@ for reader.HasNext() {
 }
 ```
 
+## Build and Test
+
+Build the sources:
+
+    go build ./pulsar
+
+Run the unit tests:
+
+    ./docker-ci.sh
+
 ## Contributing
 
 Contributions are welcomed and greatly appreciated. See [CONTRIBUTING.md](CONTRIBUTING.md) for details on submitting patches and the contribution workflow.
@@ -141,8 +152,8 @@ Contributions are welcomed and greatly appreciated. See [CONTRIBUTING.md](CONTRI
 
 ##### Mailing lists
 
-| Name                                                                          | Scope                           |                                                                 |                                                                     |                                                                              |
-|:------------------------------------------------------------------------------|:--------------------------------|:----------------------------------------------------------------|:--------------------------------------------------------------------|:-----------------------------------------------------------------------------|
+| Name                                                      | Scope                           |                                                       |                                                           |                                                                    |
+|:----------------------------------------------------------|:--------------------------------|:------------------------------------------------------|:----------------------------------------------------------|:-------------------------------------------------------------------|
 | [[email protected]](mailto:[email protected]) | User-related discussions        | [Subscribe](mailto:[email protected]) | [Unsubscribe](mailto:[email protected]) | [Archives](http://mail-archives.apache.org/mod_mbox/pulsar-users/) |
 | [[email protected]](mailto:[email protected])     | Development-related discussions | [Subscribe](mailto:[email protected])   | [Unsubscribe](mailto:[email protected])   | [Archives](http://mail-archives.apache.org/mod_mbox/pulsar-dev/)   |
 

diff --git a/VERSION b/VERSION
@@ -1,3 +1,3 @@
 // This version number refers to the currently released version number
 // Please fix the version when release.
-v0.6.0
+v0.8.0
diff --git a/docker-ci.sh b/docker-ci.sh
@@ -25,7 +25,7 @@ cd ${SRC_DIR}
 
 IMAGE_NAME=pulsar-client-go-test:latest
 
-GO_VERSION=${1:-1.12}
+GO_VERSION=${1:-1.16}
 docker rmi --force ${IMAGE_NAME} || true
 docker rmi --force apachepulsar/pulsar:latest || true
 docker build -t ${IMAGE_NAME} --build-arg GO_VERSION="golang:${GO_VERSION}" .

diff --git a/docs/release-process.md b/docs/release-process.md
@@ -12,6 +12,9 @@ In general, you need to perform the following steps:
 8. Update the release note.
 9. Announce the release.
 
+### Requirements
+- [GPG keys to sign release artifacts](https://github.com/apache/pulsar/wiki/Create-GPG-keys-to-sign-release-artifacts)
+
 ## Steps in detail
 
 1. Create a release branch.
@@ -65,9 +68,20 @@ $ gpg -b --armor apache-pulsar-client-go-0.X.0-src.tar.gz
 $ shasum -a 512 apache-pulsar-client-go-0.X.0-src.tar.gz > apache-pulsar-client-go-0.X.0-src.tar.gz.sha512 
 ```
 
+Checkout repo for uploading artifacts
+```
+$ svn co https://dist.apache.org/repos/dist/dev/pulsar pulsar-dist-dev
+$ cd pulsar-dist-dev
 ```
-$ mkdir pulsar-client-go-0.X.0-candidate-1
+
+Create a candidate directory at the root repo
+```
+$ svn mkdir pulsar-client-go-0.X.0-candidate-1
 $ cd pulsar-client-go-0.X.0-candidate-1
+```
+
+Copy the signed artifacts into the candiate directory and commit
+```
 $ cp ../apache-pulsar-client-go-0.X.0-* .
 $ svn add *
 $ svn ci -m 'Staging artifacts and signature for Pulsar Client Go release 0.X.0-candidate-1'
@@ -100,7 +114,7 @@ It fixes the following issues:
 https://github.com/apache/pulsar-client-go/milestone/1?closed=1
 
 Pulsar Client Go's KEYS file contains PGP keys we used to sign this release:
-https://dist.apache.org/repos/dist/release/pulsar/KEYS
+https://dist.apache.org/repos/dist/dev/pulsar/KEYS
 
 Please download these packages and review this release candidate:
 - Review release notes
@@ -110,7 +124,7 @@ README.md to build and run the pulsar-client-go.
 The vote will be open for at least 72 hours. It is adopted by majority approval, with at least 3 PMC affirmative votes.
 
 Source file:
-https://dist.apache.org/repos/dist/dev/pulsar/pulsar-client-go/apache-pulsar-client-go-0.X.0-src.tar.gz
+https://dist.apache.org/repos/dist/dev/pulsar/pulsar-client-go-0.X.0-candidate-1/
 
 The tag to be voted upon:
 v0.X.0
@@ -142,8 +156,9 @@ Remove the old releases (if any). We only need the latest release there, older r
 ```
 
 # Get the list of releases
-
-svn ls https://dist.apache.org/repos/dist/release/pulsar/pulsar-client-go/
+```
+svn ls https://dist.apache.org/repos/dist/release/pulsar | grep client-go
+```
 
 # Delete each release (except for the last one)
 

diff --git a/go.mod b/go.mod
@@ -1,33 +1,29 @@
 module github.com/apache/pulsar-client-go
 
-go 1.13
+go 1.15
 
 require (
-	github.com/AthenZ/athenz v1.10.15
-	github.com/DataDog/zstd v1.4.6-0.20210211175136-c6db21d202f4
-	github.com/apache/pulsar-client-go/oauth2 v0.0.0-20201120111947-b8bd55bc02bd
-	github.com/beefsack/go-rate v0.0.0-20180408011153-efa7637bb9b6
+	github.com/AthenZ/athenz v1.10.39
+	github.com/DataDog/zstd v1.5.0
+	github.com/apache/pulsar-client-go/oauth2 v0.0.0-20220120090717-25e59572242e
+	github.com/beefsack/go-rate v0.0.0-20220214233405-116f4ca011a0
 	github.com/bmizerany/perks v0.0.0-20141205001514-d9a9656a3a4b
 	github.com/davecgh/go-spew v1.1.1
 	github.com/gogo/protobuf v1.3.2
-	github.com/golang/protobuf v1.4.2
+	github.com/golang/protobuf v1.5.2
 	github.com/google/uuid v1.1.2
-	github.com/inconshreveable/mousetrap v1.0.0 // indirect
-	github.com/klauspost/compress v1.10.8
+	github.com/klauspost/compress v1.14.4
 	github.com/linkedin/goavro/v2 v2.9.8
-	github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
-	github.com/modern-go/reflect2 v1.0.1 // indirect
 	github.com/opentracing/opentracing-go v1.2.0
 	github.com/pierrec/lz4 v2.0.5+incompatible
 	github.com/pkg/errors v0.9.1
-	github.com/prometheus/client_golang v1.7.1
-	github.com/sirupsen/logrus v1.4.2
+	github.com/prometheus/client_golang v1.11.1
+	github.com/sirupsen/logrus v1.6.0
 	github.com/spaolacci/murmur3 v1.1.0
-	github.com/spf13/cobra v0.0.3
-	github.com/spf13/pflag v1.0.3 // indirect
-	github.com/stretchr/testify v1.5.1
+	github.com/spf13/cobra v1.2.1
+	github.com/stretchr/testify v1.7.0
 	go.uber.org/atomic v1.7.0
-	golang.org/x/oauth2 v0.0.0-20200107190931-bf48bf16ab8d
+	golang.org/x/oauth2 v0.0.0-20210402161424-2e8d93401602
 )
 
 replace github.com/apache/pulsar-client-go/oauth2 => ./oauth2