1
0
mirror of https://github.com/meineerde/rackstash.git synced 2025-10-17 14:01:01 +00:00

383 Commits

Author SHA1 Message Date
8c9b564f54 Allow to modify a filter chain instance by specifying registered filter names as a Symbol 2018-02-10 21:30:42 +01:00
42d120561b Clarify the error message when trying to fetch invalid specs from a ClassRegistry 2018-02-10 14:12:11 +01:00
2d04ccb6c9 Add ClassRegistry#fetch and change ClassRegistry#[] to return nil if no class was found 2018-02-10 14:08:20 +01:00
21e0c5a228 Rename the filename to paht in file adapter 2018-02-06 20:32:31 +01:00
4ee7a3f967 Simplify field serialization of Lograge encoder 2018-01-30 23:07:21 +01:00
58b58cfe4a Test sane behavior when trying to set fields or tags causing exception in Rack::Middleware 2018-01-30 18:37:25 +01:00
ce12e26bba Simplify the Rack middleware to write request log immediately
Previously, we would only setup a `::Rack::BodyProxy` to eventually log
the request once the response is deivered to the client. We did this
mainly in order to have a more "accurate" duration in the logs, similar
to what the `::Rack::CommonLogger` middleware does.

This has some significant disadvantages though:

* If an exception is raised by an outer middleware after we returned,
the response is never delivered and the log is lost.
* If a timeout occured (and e.g. a Unicorn worker is killed or a
`Rack::Timeout` stroke) before we return the full response, the log is
lost.
* If the client goes away before the request is finished, the app server
might decide no just throw away the response.

Besides addressing these issues, logging directly after the request
leaves our middleware scope makes the whole process much easier to
reason about.
2018-01-30 18:36:47 +01:00
1c461bf8da Specify exact behavior for @version and @timestamp fields for Logstash encoder 2018-01-27 00:48:51 +01:00
1d16d366a0 Test JSON encoder behavior with a missing timestamp 2018-01-27 00:47:25 +01:00
952ecd5c73 Ensure that the JSON encoder correctly serializes the event timestamp 2018-01-27 00:36:03 +01:00
d6cc062b93 Omit an empty message when encoding an event as JSON 2018-01-27 00:36:03 +01:00
db57d21a45 Correct name of initializer parameter in documentation for Encoder::Message 2018-01-27 00:36:03 +01:00
7f6e7e86c6 Add tests to ensure the the Message encoder can deal with uncommon messages 2018-01-27 00:36:03 +01:00
e805d69d84 Use low-level IO#syswrite instead of IO#write in File adapter
Since

1. we are using binmode and do not expect any conversation of newlines
   or encodings to take place, and
2. we are only writing to the open file when we have squired the mutex
   of the adapter instance

we can safely use the syswrite method to write logs. In the end, this
uses the same low-level syscall as `IO#write` but with less overhead.
2018-01-26 18:02:09 +01:00
4a012e28cc Clarify documentation of File adapter semantics 2018-01-26 18:00:37 +01:00
18abd892ba Test that Adapter::File.from_uri only accepts file URIs 2018-01-25 11:14:27 +01:00
6d4106fcb7 Add Select and Remove filters to filter which fields get passed to the encoder 2018-01-25 10:52:42 +01:00
70f9b1b735 Ensure that field keys configured in Filters are valid UTF-8 2018-01-25 01:14:06 +01:00
2a9d0f4b74 Simplify the tests for embedded IPs in the AnonymizeIPMask filter again
We still want to avoid the deprecated call to `IPAddr#ipv4_compat?`.
However, since we are always masking of at least one bit of any given IP
address anyways, we can ignore the specific tests for '::' and '::1' in
that method. The result of masking off these addresses will always be
'::' regardless of how many bits we are masking off anyways.
2018-01-24 21:17:56 +01:00
1687a817b7 Do not require 'complex'. It's not needed and deprecated 2018-01-24 20:22:34 +01:00
dda72c2f84 Work around deprecations on Ruby 2.5.0 2018-01-24 20:18:50 +01:00
5962b84f99 Reference the JSON Lines standard in Encoder::JSON documentation 2018-01-24 15:10:45 +01:00
99d2f489dd Update Copyright year 2018-01-24 00:21:04 +01:00
577c065900 Allow to define a File adapter using a file:// URI 2018-01-23 23:54:48 +01:00
39f12047cb Add helper method to parse URI parameter from an adapter URI 2018-01-23 23:53:31 +01:00
c194b00b34 Improve code formatting and fix Rubocop issues 2018-01-23 23:21:58 +01:00
3a8371e1b8 Fix comment typo in Rackstash::Logger 2018-01-23 23:19:45 +01:00
38a5c669a3 Allow to register complex URI schemes for Adapters
Most notably, we want to use the plus character in URI schemes. By only
checking the first character of the scheme registration, we can better
fullfil our contract and better distinguish between schemes and class
names of registered Adapters.
2018-01-22 01:11:27 +01:00
e8bf2c31bd Raise ArgumentError when trying to resolve an unknown adapter URI
If we were to fall through, undefined schemes would always end up as a
file adapter. This is generally undesired since it hides the fact that
we have not found a suitable adapter. Most of the time, this will be a
configuration error which should be reported early.

If a user still wants to create a file adapter with a filename that
looks like a URI, they can create a Rackstash::Adapter::File object
manually.
2018-01-22 01:05:06 +01:00
bdeb0534c9 Allow to set the Flow#raise_on_error setting with the same getter/setter protocol as all the other values 2018-01-12 23:34:17 +01:00
96f102875f Allow to set Flow#raise_on_error? to aid in debugging flows 2018-01-12 21:03:52 +01:00
0605fc433d Document the error_flow parameter in Flow#initialize 2018-01-12 20:38:57 +01:00
2425fdea09 Bump Rubies in Travis CI and add Ruby 2.5.0 2018-01-03 21:03:34 +01:00
7c424edfe0 Remove debug code from specs 2018-01-03 20:40:51 +01:00
65d75523c3 Ensure only valid classes can be registered as filters and encoders 2018-01-03 20:40:31 +01:00
fe44186da4 Add filter to mask of IP addresses in event fields 2018-01-03 20:39:24 +01:00
cf7587cde8 Use the registered filter names to define them in examples 2017-12-28 22:08:34 +01:00
795a8e4e2d Clarify documentation on how to define filters 2017-12-28 22:07:41 +01:00
e5c83d9ac5 Add documentation about Filter#register and Encoder#register 2017-12-21 01:42:01 +01:00
9ed350cb8d Allow fields to hold force-inspected BasicObjects 2017-12-21 01:19:10 +01:00
6eaeaedaf8 Register encoders using a ClassRegistry 2017-12-20 23:58:32 +01:00
de6b60925a Allow to define conditionals on built filters
Now you can define optional filters which only run if some condition is
true or false. This can be used to e.g. update fields depending on some
tags being present in the event.
2017-12-20 23:40:20 +01:00
a475a5c6fc Use the ClassRegistry to register filters
This removes the requirement that filters need to be defined in the
Rackstash::Filters namespace in order to be usable with a short name.
Filters can be registered with an arbitrary name.

As before, raw blocks / procs can be defined as ad-hoc filters.
2017-12-20 23:40:20 +01:00
01593a9b8b Add Rackstash::ClassRegistry to register filters and encoders
Using the ClassRegistry, we can register short names for filter and
encoder classes so that they can be specified in a shorter and more
readable way during creation of the logger.
2017-12-20 23:06:25 +01:00
1b2dd09000 Add FilterChain#push as an alias to FilterChain#<< 2017-12-20 21:39:55 +01:00
0782f31d45 Test that FilterChain#append returns the original filter chain 2017-12-20 21:39:15 +01:00
6bfe8ecb25 Provide prepend alias for FilterChain@unshift
Ruby 2.5 introduces this alias for all arrays.
2017-12-20 21:33:49 +01:00
43a303b09d Include Enumerable module into Flows 2017-12-20 21:27:12 +01:00
ad4a40413b Add example to documentation of DropIf filter 2017-12-20 18:42:21 +01:00
40e151ecb8 Rename the SkipEvent filter to DropIf to make the intention clearer 2017-12-20 18:41:40 +01:00