Introduction

Our FluentD + Elasticsearch + Kibana setup can automatically parse log statements if the application prints each log line as a JSON string. This is very useful for log statements that usually output many lines and require a lot of energy to sift through. The most common example of a case like this would be a typical stacktrace.

Kibana is automatically configured to handle Logstash message logging format and this is what FluentD does. For Java there is a Logback encoder that can generate the logging in this format and the goal of this setup is to leverage that.

Setup dependencies

compile 'net.logstash.logback:logstash-logback-encoder:4.7'
compile 'org.codehaus.janino:janino:3.0.6'

logstash-logback-encoder does all the magic. Janino is necessary to enable conditional logging (more on that in a moment).

Logback configuration

By default, Spring Boot tries to use Logback if that is configured in the classpath. If you have any logback*.xml files in the src/main/resources folder, remove them. The bulk of the functionality we need is enabled by the logback-spring.xml file:

<?xml version="1.0" encoding="UTF-8"?>
<configuration>
    <include resource="org/springframework/boot/logging/logback/defaults.xml"/>
    <appender name="CONSOLE" class="ch.qos.logback.core.ConsoleAppender">
        <if condition='property("LOG_OUTPUT").equalsIgnoreCase("json")'>
            <then>
                <encoder class="net.logstash.logback.encoder.LogstashEncoder"/>
            </then>
            <else>
                <encoder>
                    <pattern>${CONSOLE_LOG_PATTERN}</pattern>
                    <charset>utf8</charset>
                </encoder>
            </else>
        </if>
    </appender>
    <root level="INFO">
        <appender-ref ref="CONSOLE"/>
    </root>
</configuration>

This if expression is enabled by Janino (GitHub). The property LOG_OUTPUT can be declared as a system property or environment variable.

Deployment configuration

Since the Logback config file needs the environment variable LOG_OUTPUT, all you need to do is declare this environment variable in your Kubernetes replication controller or deployment YAML.

env:
  - name: LOG_OUTPUT
    value: 'json'

Bonus

This configuration frees your console logging for development only. To make your logging a bit more readable you can enable ANSI colors, thus:

application.properties
spring.output.ansi.enabled = always

or application.yaml

spring:
  output:
    ansi:
      enabled: always