Deploy the Kafka connector for real-time reporting (RTR)

This page covers deploying the Kafka sink connector that consumes RTR topics and writes transformed data into reporting tables.

1. The helm chart for Kafka connector should be available under charts/kafka-connect-sink.

2. Validate the image repository and tag:

   image: confluentinc/cp-kafka-connect
   tag: <release-version-tag> e.g v1.0.1
   

3. Update values.yaml with the RDB_modern hostname, username, password, and Kafka bootstrap server names:

   sqlServerConnector:
     config:
       connection.url: jdbc:sqlserver://nbs-db.EXAMPLE_FIXME.nbspreview.com:1433;databaseName=rdb;encrypt=true;trustServerCertificate=true;,
       connection.user: EXAMPLE_FIXME,
       connection.password: EXAMPLE_FIXME,
   kafka:
     bootstrapServers: "EXAMPLE_FIXME"
   

4. Install the pod:

   helm install -f ./kafka-connect-sink/values.yaml cp-kafka-connect-server ./kafka-connect-sink/
   

5. Verify the pod is running:

   kubectl get pods
   

6. Validate the service:

   • This is an internal service with no ingress. Validation should be part of RTR Pipeline Validation.
   • If the service has trouble connecting to the database, run this command to reset the ConfigMap:

   kubectl delete configmap cp-kafka-connect-sqlserver-connect