You can configure Rancher to send cluster or project logs to a Kafka server.

Configuring Kafka Logging

Prerequisite: You must have a Kafka server configured.

  1. Browse to the cluster or project that you want to log. {{% accordion id="cluster" label="To Configure Cluster Logging:" %}} If you're a cluster owner or member who works in operations or security, configure cluster logging.

  2. From the Global view, open the cluster that you want to configure logging for.

  3. From the main menu, select Tools > Logging. {{% /accordion %}} {{% accordion id="project" label="To Configure Project Logging:" %}} If you're a project owner or member who works on an application, configure project logging.

  4. From the Global view, open the project that you want to configure logging for.

  5. From the main menu, select Resources > Logging. {{% /accordion %}}

  6. Select Kafka.

  7. Complete the Kafka Configuration form.

    1. From Endpoint Type, select the type of Kafka server you are using: Zookeeper or Broker.

    2. From the Endpoint field, enter the IP address and port for your Kafka server.

      By default, Kafka uses port 9092.

    3. From the Topic field, enter the name of a Kafka topic that your Kubernetes cluster submits logs to.

  8. Complete the Additional Logging Configuration form.

    1. Optional: Use the Add Field button to add custom log fields to your logging configuration. These fields are key value pairs (such as foo=bar) that you can use to filter the logs from another system.

    2. Enter a Flush Interval. This value determines how often Fluentd flushes logs to the logging server. Intervals are measured in seconds.

  9. Click Save.

Result: Rancher is now configured to send logs to Kafka. View your Kafka stream to view logs for your cluster and containers.