diff --git a/README.md b/README.md index 4e70e115..0fd0408e 100644 --- a/README.md +++ b/README.md @@ -1,4 +1,4 @@ -# Aiven's OpenSearch® Connector for Apache Kafka® +# Aiven's OpenSearch® Sink Connector for Apache Kafka® [![Build Status](https://github.com/aiven/opensearch-connector-for-apache-kafka/actions/workflows/master_push_workflow.yml/badge.svg)](https://github.com/aiven/opensearch-connector-for-apache-kafka/actions) @@ -11,7 +11,14 @@ The code was forked and all classes were renamed. # Documentation -## How to install +## Install and configure required software: +1. Install and Run Kafka (https://kafka.apache.org/quickstart) +2. Install and Run Kafka Connect (https://kafka.apache.org/documentation/#connect_running) +3. Install and Run OpenSearch (https://opensearch.org/docs/latest/install-and-configure/install-opensearch/index/) + - If OpenSearch is running on SSL, make sure to copy the credentials and copy them to woker properties as mentioned in Quickstart guide below +4. Install Aiven's OpenSearch® Sink Connector (see below) + +## How to install Aiven's OpenSearch® Sink Connector 1. Connector plugins are packaged in zip/tar format to be released 2. Users download plugins from GitHub releases or build binaries from source @@ -75,6 +82,18 @@ curl http://localhost:8083/connector-plugins | jq . [OpenSearch® Sink Connector Configuration Options](docs/opensearch-sink-connector-config-options.rst) +## QuickStart guide + +1. Ensure that the required software is installed and running +2. Add OpenSearch Sink connector to Kafka Connect: Follow the 'How to install' instructions to add the OpenSearch sink connector to Kafka Connect. Example worker config is located here https://github.com/Aiven-Open/opensearch-connector-for-apache-kafka/blob/main/config/quickstart-opensearch.properties +3. Verify plugin installation : Visit http://localhost:8083/connectors to confirm that the OpenSearch sink connector is listed +4. Check ACLs (If Enabled): If ACLs are enabled on Kafka, ensure there are no authorization exceptions for the topic and group resources. Example of adding acls : https://kafka.apache.org/documentation/#security_authz_examples +5. Produce Events: Produce JSON-formatted events to the Kafka topic specified in the worker properties. +6. Index Creation: An index will be created in OpenSearch with the same name as the Kafka topic. +7. Create Index Pattern: Create an index pattern in OpenSearch. +8. Discover Events: Events produced to the Kafka topic can now be discovered in OpenSearch. +9. Trouble shooting: If there are any deserialization errors in the connector logs, try setting schema.ignore to true. + # Contribute [Source Code](https://github.com/aiven/aiven-kafka-connect-opensearch) diff --git a/config/quickstart-elasticsearch.properties b/config/quickstart-opensearch.properties similarity index 100% rename from config/quickstart-elasticsearch.properties rename to config/quickstart-opensearch.properties