Logging Ror on dedicated file on es 6.2.1

Users Feedback
1

While implementing Ror on our new cluster 6.2.1 , we have trouble to make the log4j part running :
This below come from the doc.

Trick: log requests to different files
Here is a l4j2.properties snippet for ES 5.x that logs all the received requests as a new line in a separate file:

#Plugin readonly rest separate access logging file definition
appender.access_log_rolling.type = RollingFile
appender.access_log_rolling.name = access_log_rolling
appender.access_log_rolling.fileName = ${sys:es.logs}_access.log
appender.access_log_rolling.layout.pattern = [%d{ISO8601}][%-5p][%-25c] %marker%.-10000m%n
appender.access_log_rolling.layout.type = PatternLayout
appender.access_log_rolling.filePattern = ${sys:es.logs}_access-%d{yyyy-MM-dd}.log
appender.access_log_rolling.policies.type = Policies
appender.access_log_rolling.policies.time.type = TimeBasedTriggeringPolicy
appender.access_log_rolling.policies.time.interval = 1
appender.access_log_rolling.policies.time.modulate = true

logger.access_log_rolling.name = org.elasticsearch.plugin.readonlyrest.acl
logger.access_log_rolling.level = info
logger.access_log_rolling.appenderRef.access_log_rolling.ref = access_log_rolling
logger.access_log_rolling.additivity = false

# exclude kibana, beat and logstash users as they generate too much noise
logger.access_log_rolling.filter.regex.type = RegexFilter
logger.access_log_rolling.filter.regex.regex = .*USR:(kibana|beat|logstash),.*
logger.access_log_rolling.filter.regex.onMatch = DENY
logger.access_log_rolling.filter.regex.onMisMatch = ACCEPT

We replaced by
logger.access_log_rolling.name = org.elasticsearch.plugin.readonlyrest.acl
by
logger.access_log_rolling.name = tech.beshu

but :
as the additivity setting works fine, ( no more logs from RoR in es log file), we are not able to get the RoR logs written in our
appender.access_log_rolling.fileName = ${sys:es.logs}_access.log

the file is created, but nothing is written in.

did you change something ?

PS : on another hand, audit_log_collector runs fine.

Rgrds,

Fred

2

HI Fred, thanks for the feedback!

“did you change anything” from when? Between 2.x and 5.x the ES logging framework moved to log4j2, so a lot has changed.

About the “too much noise” section, you now can set a “verbosity: error” rule to silence the “ALLOWED” log lines, while preserving the “FORBIDDEN” ones which lets you detect attempts to brute forcing those credentials. Please note: “verbosity” rule affects audit log collector in the same way.

3

Well I speak about the example for 5.x, which was using org.elasticsearch.plugin.readonlyrest.acl ,

And since you are using tech.beshu.ror now.

I copy/pasted the example above from elasticsearch.md, but still have no luck to see RoR logs in its dedicated file.

In fact, with the config above, i get no more RoR logs in files.

But I still have them in audit collector indice. Then i think something wrong with my l4j2 block above.

Do you see something’s wrong in it ?

4

Hi all,

I found that went wrong and it is related to the line
appender.access_log_rolling.fileName = ${sys:es.logs}_access.log

It has to be replaced by
appender.access_log_rolling.fileName = ${sys:es.logs.base_path}${sys:file.separator}${sys:es.logs.cluster_name}_access.log

# ROR
#Plugin readonly rest separate access logging file definition
appender.access_log_rolling.type = RollingFile
appender.access_log_rolling.name = access_log_rolling
appender.access_log_rolling.fileName = ${sys:es.logs.base_path}${sys:file.separator}${sys:es.logs.cluster_name}_access.log
appender.access_log_rolling.layout.pattern = [%d{ISO8601}][%-5p][%-25c] %marker%.-10000m%n
appender.access_log_rolling.layout.type = PatternLayout
appender.access_log_rolling.filePattern = ${sys:es.logs}_access-%d{yyyy-MM-dd}.log
appender.access_log_rolling.policies.type = Policies
appender.access_log_rolling.policies.time.type = TimeBasedTriggeringPolicy
appender.access_log_rolling.policies.time.interval = 1
appender.access_log_rolling.policies.time.modulate = true
 
logger.access_log_rolling.name = tech.beshu
logger.access_log_rolling.level = info
logger.access_log_rolling.appenderRef.access_log_rolling.ref = access_log_rolling
logger.access_log_rolling.additivity = false
 
# exclude kibana, beat and logstash users as they generate too much noise
logger.access_log_rolling.filter.regex.type = RegexFilter
logger.access_log_rolling.filter.regex.regex = (?s).*USR:(kibana|beat|logstash|monitor),.*
logger.access_log_rolling.filter.regex.onMatch = DENY
logger.access_log_rolling.filter.regex.onMisMatch = ACCEPT
1 Like
5

modified a bit the regex pattern, as logs acl come as multiline