[DONE] Feature: Control what users to log/not log

New Features Ideas
1

Right now logging is very spammy as it logs every attempt for all users. When we have authentication for beats and logstash, the log is huge.

A way to disable log for certain users would be very useful, as we could log only the end user or failing users and ignore the remaining users that are known to work. Those that need audit logs can always not disable any user.

1 Like
2

Yeah I was also thinking logs need more attention and configurability, at least per user. BTW it’s probable that you can filter logs using regex with l4j2 in the meanwhile :slight_smile:

3

it was hard, as there are no examples how to work with regexfilter in properties files, but i manage to configure it!
So here is the way to log to a file all except some users (logstash, beat and kibana for me)

#Plugin readonly rest logging file definition
appender.access_log_rolling.type = RollingFile
appender.access_log_rolling.name = access_log_rolling
appender.access_log_rolling.fileName = ${sys:es.logs}_access.log
appender.access_log_rolling.layout.pattern = [%d{ISO8601}][%-5p][%-25c] %marker%.-10000m%n
appender.access_log_rolling.layout.type = PatternLayout
appender.access_log_rolling.filePattern = ${sys:es.logs}_access-%d{yyyy-MM-dd}.log.gz
appender.access_log_rolling.policies.type = Policies
appender.access_log_rolling.policies.time.type = TimeBasedTriggeringPolicy
appender.access_log_rolling.policies.time.interval = 1
appender.access_log_rolling.policies.time.modulate = true

logger.access_log_rolling.name = org.elasticsearch.plugin.readonlyrest.acl
logger.access_log_rolling.level = info
logger.access_log_rolling.appenderRef.access_log_rolling.ref = access_log_rolling
logger.access_log_rolling.additivity = false

# exclude kibana, beat and logstash users as they generate too much noise
logger.access_log_rolling.filter.regex.type = RegexFilter
logger.access_log_rolling.filter.regex.regex = .*USR:(kibana|beat|logstash),.*
logger.access_log_rolling.filter.regex.onMatch = DENY
logger.access_log_rolling.filter.regex.onMisMatch = ACCEPT

:grinning:

1 Like
4

HAHA dude you’re the best :+1: :rocket::joy:

Thanks & will add to docs :writing_hand:️

5

Now there’s the verbosity rule that lets you skip logging allowed requests if they match a certain block (useful for Kibana deamon, which is always polling).

6

@sscarduzio Is that verbosity global or can we set it per rule?

7

it is per rule-block. See example in README.md

...
 - name: "::KIBANA-SRV::"
      auth_key: kibana:kibana
      verbosity: error # don't log successful request
      type: allow
- name "..."
 #other rules
8

Ohh, sorry i didn’t notice it in the README.md! That is perfect! thanks! :smiley:

ps: ES is already in version 5.3.1, so a new build is always welcome :wink:

9

Yeah I didn’t have the time to release 1.15.0, but it’s there and it works with 5.3.{1,2}

https://readonlyrest-data.s3-eu-west-1.amazonaws.com/build/1.15.0/readonlyrest-1.15.0_es5.3.2.zip?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAJEKIPNTOTIVGQ4EQ/20170502/eu-west-1/s3/aws4_request&X-Amz-Date=20170502T140518Z&X-Amz-Expires=604800&X-Amz-SignedHeaders=host&X-Amz-Signature=26c6d7cea05af61505567f149daafecd48ab189038145637e968c044d9c941ec

This is a build for 1.15.0 for 5.3.2, will have more time to update the website after this week.