1
0
mirror of https://github.com/meineerde/rackstash.git synced 2025-10-17 14:01:01 +00:00

190 Commits

Author SHA1 Message Date
b6d15a8da7 Extract resolving of severity values to Rackstash.severity 2017-10-05 21:17:32 +02:00
4ee34e5d4e Insert an ellipsis if messages are removed in TruncateMessage filter
If messages are removed by the final cut step (but not when applying the
selectors), we insert an ellipsis, by default a Message with the
contents of "[...]\n", at the place where we removed messages. This
helps the investigating human who reads the logs to understand that
there were logs messages removed from the event.
2017-10-05 01:05:55 +02:00
e7fd3677ca Ensure more consistent code formatting with Rubocop 2017-10-03 22:52:49 +02:00
d157e53129 Add logger adapter to write log events to an external logger 2017-10-03 20:25:30 +02:00
0941b24b63 Optionally prefix all message lines with extraced fields in Message encoder
This can be used to achieve similar results as
ActiveSupport::TaggedLogger. With our implementation, the "tags" are
prepended to each line (instead of each message) and we add the same
tags to all message lines.

If required, you can still use ActiveSupport::TaggedLogger with
Rackstash to add tags at the time of adding the log message.
2017-09-30 00:01:25 +02:00
3584244bc1 Return the normalized unchanged message from the Message encoder 2017-09-29 23:40:02 +02:00
59c94a8ecc Insert newline comment below magic comments in ruby file header 2017-09-29 17:15:07 +02:00
402f754736 Test TrucateMessage filter with Proc selectors and custom callables 2017-09-23 13:45:08 +02:00
f17692ff73 Add Update filter to set new values to existing event keys 2017-09-23 00:50:46 +02:00
c383e2b3dc Add Replace filter to set new values on event fields 2017-09-23 00:26:03 +02:00
a827769bb3 Add Rename filter to rename event fields 2017-09-23 00:14:56 +02:00
1711b0d589 Resolve Ruby warning in spec for Rackstash::Encoders::Logstash
This resolves the following warning:

> spec/rackstash/encoders/logstash_spec.rb:18: warning: ambiguous first
> argument; put parentheses or a space even after `/' operator
2017-09-21 22:35:35 +02:00
2f0755c967 Optionally retain scope when evaluating procs in collections
This allows to receive the scope as the argument to a proc-value instead
of always executing the proc in the scope's context.
2017-09-21 22:30:52 +02:00
bcc481bfbe Test that a Logger can be created without defining flows 2017-09-21 22:29:39 +02:00
a31f07f7c2 Yield the last created flow on Logger.new
This allows to configure the last (and often only) flow on creation of
the Logger without having to manually create the object. We can thus use
the following shortcut to e.g. set a custom encoder:

    Rackstash::Logger.new(STDOUT) do
      encoder Rackstash::Encoder::Message.new
    end
2017-09-21 16:18:46 +02:00
74242caf0b There can be only one... UNDEFINED
By ensuring that we only ever create a single UNDEFINED object from the
UndefinedClass, it behaves more like nil
2017-09-07 21:18:03 +02:00
32e755396b Splat flows/adapters passed to Rackstash::Logger#initialize
This ensures that we are not unduely altering (i.e. flattening) adapter
specifications when creating the flows.
2017-08-31 21:59:48 +02:00
cadc235795 Add Helpers::Time module to access a monotonic clock 2017-08-30 01:01:15 +02:00
8b75013472 Test that the Lograge encoder skips messages 2017-08-24 23:17:20 +02:00
b101d7356a Add lograge encoder
This encoder formats logs similar to the key-value formatter of the
lograge gem.
2017-08-24 23:07:17 +02:00
40aa74c5c0 Always add a @timestamp field in the logstash encoder, even if it was removed by a filter 2017-08-24 23:07:17 +02:00
7b06d053ca Create an error event in the Flow exactly as a Buffer would
That way, we ensure that the error_flow receives an expected event and
doesn;t have to deal with different data formats which might result in
additional (then quietly hidden) errors.
2017-08-24 23:07:17 +02:00
96aaaa0348 Set Encoder helper methods as private
That way, they don't pollute the public interface of the encoders
including the helpers modules.
2017-08-24 23:07:17 +02:00
3078bccafd Pass the raw Buffer timestamp through the event Hash
This allows filters and finally the encoder to use this without having
to first convert the timestamp (back) to a more suitable format.
2017-08-24 23:07:17 +02:00
99728842aa Pass the raw messages array in the event Hash to the encoder
The encoder is then responsible to format it as it pleases. Commonly,
encoders can use Rackstash::Encoders::Helpers::Message#normalize_message
to create a single combined message string.

If the encoder is not interested in using the message, it can jsut get
rid if it without incuring any overhead.
2017-08-22 00:02:10 +02:00
78466f9439 Set the Buffer's timestamp when setting a tag 2017-08-17 00:49:10 +02:00
021dbc256b Flush silent Buffers by default
With the change to allow ading fields to the Buffer with a Logger
interface in the previous commit, we should also flush those Buffers
with fields set that way by default.
2017-08-17 00:34:47 +02:00
013d0f7d92 Auto-flush the buffer when adding fields using Logger#add
With this interface, the user adds fields the same way they would add
messages. The implicit assumption here is that the buffer is handled the
same way. Thus, if the current buffer is non-buffering, we will
automatically flush it to the sink and clear it, just the same way as we
would have done it for a message.
2017-08-17 00:34:47 +02:00
cca00408e2 With allow_silent, also check the presence of an explicit timestamp for Buffer#pending? 2017-08-16 20:54:55 +02:00
d8300a125e Rename Buffer#allow_empty? to Buffer#allow_silent?
This new name better reflects what this option actually does: it allows
to flush a Buffer even if there were just added fields or tags. A tryely
empty Buffer, i.e., one which has neither added messages, fields, or
tags is never flushed.
2017-08-16 20:46:28 +02:00
0a5e0a4aab Add TruncateMessage filter to restrict the size of the logged message 2017-08-16 00:43:09 +02:00
ee1f5480c4 Remove useless assignments in message_spec 2017-08-15 23:45:54 +02:00
fa174bba9d Freeze the raw object along the wrapper for all fields 2017-08-15 18:36:33 +02:00
57f1197299 Do not alter the message in JSON encoder
If required, users can setup filters for that. We shouldnot assume any
overly specific use-cases here.
2017-08-14 23:25:59 +02:00
0a1fe46ea5 Add filter to remove all ANSI color codes from the event message 2017-08-14 23:18:19 +02:00
10993d2b1a Serialize Rational numbers to a String
With this, we are more compatible with the common JSON encoders,
including the one on RUby core.
2017-08-14 14:10:48 +02:00
ab595d031b Add Logstash encoder
This encoder is very similar to the JSON encoder. It handles a few
additional specifics for Logstash to be suotable for Logstash's json
input.
2017-08-14 13:33:52 +02:00
2fbdf3c1f5 Don't create the @version fields in events by default
This field is only relevant when actually sending JSON to Logstash.
Other formats don't need it. It is thus more desireable to only include
this field in a specific logstash JSON encoder.
2017-08-14 13:27:20 +02:00
536bb9a087 Ensure that the Formatter freezes the strings in all cases
Previously, the returned String would been frozen only in Rubies
understanding the frozen_literals magic comment.
2017-08-12 14:05:20 +02:00
168e62b63c Remove useless newlines 2017-08-12 14:02:03 +02:00
3a1f3686c0 Add Message#strip methods to strip whitespace from messages
These mehods work very similar to their String equivalents.
2017-08-08 22:26:34 +02:00
e999746f1e Add Message#gsub and Message#sub methods
These allow to create new Message objects with an updated message,
similar to `gsub` and `sub` of the core String.
2017-08-08 22:15:21 +02:00
eb93d70468 Add Message#copy_with to create an adjusted copy of an exising message
You can overwrite any (or none) of the existing fields to create a new
frozen copy of a message object.
2017-08-08 22:15:13 +02:00
011249bb75 Add Flows#first and Flows#last 2017-08-07 22:49:39 +02:00
eab3d16ec5 Fix minor code formatting issues 2017-08-05 23:53:01 +02:00
d362b7efaa Make Flow#unshift_filter the primary alias before filter_prepend to be more consistent 2017-08-05 23:29:54 +02:00
581b8b1819 Allow to insert a new filter defined by a symbol or String into a FilterChain 2017-08-05 23:29:54 +02:00
f0c65524a7 Build a new Filter from a String or Symbol 2017-08-05 23:29:54 +02:00
77675f65a6 Add Logger#push_buffer and Logger#pop_buffer for manual Buffer management
Most of the time, these methods should be used. They are only required
for special cases where the Buffer needs to be flushed later than when
its poped. In this case, special precautions need to be made to ensure
that the Buffer is always reliably poped and flushed to avoid loosing
logs.
2017-08-05 00:05:41 +02:00
a987e92250 Add BufferStack#pop to remove the top-most Buffer without flushing it
This can be used for certain advanced constructs where the Buffer needs
to be manually flushed later, after it was already removed from the
stack.
2017-08-04 14:49:10 +02:00