Deploy the Kafka connector for real-time reporting (RTR)

This page walks through deploying the Kafka sink connector using the ‘kafka-connect-sink’ Helm chart from the NEDSS-Helm repository for NBS version 7.13. The connector consumes RTR topics and writes transformed data into reporting tables.

On this page

  1. Prerequisites
  2. Deploy the Kafka connector using Helm
  3. Troubleshoot the Kafka connector
  4. Next steps

Prerequisites

This page assumes you’ve completed Before you begin for the microservices phase and each RTR deployment page before this one, in order. The page immediately before this one is Debezium.

Have your database credentials and Kafka endpoint values available. See the Helm values reference if you need help determining any values.

Deploy the Kafka connector using Helm

Complete the following steps to deploy the Kafka connector Helm chart from the charts/kafka-connect-sink/ directory of your cloned NEDSS-Helm repository:

  1. In the kafka-connect-sink chart directory, open the values file for your cloud provider:
    • AWS: kafka-connect-sink/values.yaml
    • Azure: kafka-connect-sink/values-azure.yaml

  2. Search the values file for EXAMPLE and fill in your environment-specific values. See the Helm values reference for help determining values.

  3. Install the Helm chart:

    • AWS:

       helm install -f ./kafka-connect-sink/values.yaml cp-kafka-connect-server ./kafka-connect-sink/

    • Azure:

       helm install -f ./kafka-connect-sink/values-azure.yaml cp-kafka-connect-server ./kafka-connect-sink/

  4. Verify the pod is running:

    kubectl get deployment cp-kafka-connect-server

After the Kafka connector deploys successfully, continue to Java service.

Troubleshoot the Kafka connector

If the service has trouble connecting to the database, run the following command to reset the ConfigMap:

kubectl delete configmap cp-kafka-connect-sqlserver-connect

If issues persist, email nbs@cdc.gov.

Next steps

Continue to Java service.