Collector¶
Kieker supports for different programming languages and probes to send monitoring data via TCP to a remote host. In distributed systems, this can imply that multiple probes send data simultaneously. The Kieker Collector allows to collect this data from different sources and stores them in one kieker log, a database or relays it to analysis software.
The Collector supports any ISourceCompositeStage compatible event
source provided by Kieker tools, including TCP binary streams. As sinks,
the collector supports all Kieker writers based on the DataSinkStage from
Kieker. The Collector is configured via a configuration file. The
configuration file consists of three parts one for basic Kieker settings, one
for the source and one for the sink.
Currently supported event sources (readers):
architecture-receive-events-via-tcp
architecture-receive-events-from-log-files
architecture-receive-events-via-http
Short |
Long Option |
Required |
Description |
|---|---|---|---|
-c |
–configuration |
true |
Configuration file |
Example configurations snippets¶
The following snippets can be combined to configure the collector application. A complete configuration must contain one general settings block, one input configuration and one output configuration.
General setup parameter for Kieker.
# Kieker settings
## The name of the Kieker instance.
kieker.monitoring.name=KIEKER
## Auto detect hostname for the writer
kieker.monitoring.hostname=
## Output metadata record
kieker.monitoring.metadata=true
Receive data via TCP (binary encoding). This can be used to receive binary logging data from Kieker in Java and C/C++/Fortran.
# TCP servcer for multiple connections
kieker.tools.source=kieker.tools.source.MultipleConnectionTcpSourceCompositeStage
kieker.tools.source.MultipleConnectionTcpSourceCompositeStage.port=9876
kieker.tools.source.MultipleConnectionTcpSourceCompositeStage.capacity=8192
Read another Kieker log. This can be useful to inspect binary logs, replay logs, make text based logs more compact and even compress them in the process.
# File reader
kieker.tools.source=kieker.tools.source.LogsReaderCompositeStage
kieker.tools.source.LogsReaderCompositeStage.logDirectories=$INPUT_DIR
## Buffer size
kieker.tools.source.LogsReaderCompositeStage.bufferSize = 8192
Store the output in a Kieker log.
# Define output
## Data sink stage (FileWriter)
kieker.monitoring.writer=kieker.monitoring.writer.filesystem.FileWriter
## FileWriter settings
## output path
kieker.monitoring.writer.filesystem.FileWriter.customStoragePath=$OUTPUT_DATA_DIR/
kieker.monitoring.writer.filesystem.FileWriter.charsetName=UTF-8
## Number of entries per file
kieker.monitoring.writer.filesystem.FileWriter.maxEntriesInFile=25000
## Limit of the log file size; -1 no limit
kieker.monitoring.writer.filesystem.FileWriter.maxLogSize=-1
## Limit number of log files; -1 no limit
kieker.monitoring.writer.filesystem.FileWriter.maxLogFiles=-1
## Map files are written as text files
kieker.monitoring.writer.filesystem.FileWriter.mapFileHandler=kieker.monitoring.writer.filesystem.TextMapFileHandler
## Flush map file after each record
kieker.monitoring.writer.filesystem.TextMapFileHandler.flush=true
## Do not compress the map file
kieker.monitoring.writer.filesystem.TextMapFileHandler.compression=kieker.monitoring.writer.compression.NoneCompressionFilter
## Log file pool handler
kieker.monitoring.writer.filesystem.FileWriter.logFilePoolHandler=kieker.monitoring.writer.filesystem.RotatingLogFilePoolHandler
## Text log for record data
kieker.monitoring.writer.filesystem.FileWriter.logStreamHandler=kieker.monitoring.writer.filesystem.TextLogStreamHandler
## Do not compress the log file
kieker.monitoring.writer.filesystem.TextLogStreamHandler.compression=kieker.monitoring.writer.compression.NoneCompressionFilter
## Flush log data after every record
kieker.monitoring.writer.filesystem.FileWriter.flush=true
## buffer size. The log buffer size must be big enough to hold the biggest record
kieker.monitoring.writer.filesystem.FileWriter.bufferSize=81920
Instead of generating text log files, you may use the
BinaryLogStreamHandler to produce binary output. Also can specify a
compression algorithm for the log and map files, or use a totally
different Kieker writer. For more details on the writer see
architecture-java-file-writer.