Hi all,
I will implement in Pre-Production the 1.16.14.pre1 version next week , and will give feedbacks about behavior on 2.x infrastructure.
regarding our 5.x infrastructure, unfortunately the owner is unavailable for now, we will have to wait.
KR
Fred
Okay, here are feedbacks using 1.16.14
A. installing and retro compatibility OK with old rules methodes (1.13.2 and below) : OK
B. LDAP test : OK Yay Ponies !! - I had to change some approach on my side, I will share later
C. Logging compatibility test : _OK _
D. Auditing compatibility test : See Issue 2 and Issue 4
E. test case below : OK
F. readonlyrest config not showed anymore in /_nodes/_all since hosted in separated file. : OK
Issues:
[Issue 2] Audit_collector: true :
Currently in logging.xml :
es.logger.level: DEBUG
I get some errors in logs, between good messages
[2017-11-14 15:33:22,268][DEBUG][cluster.service ] [ELKG_TEST] cluster state update task [put-mapping [ror_audit_evt]] failed
`MapperParsingException[Field name [{“error_message”:null,“headers”:[“Accept-Encoding”,“Authorization”,“Connection”,“Content-Length”,“Content-Type”,“Host”,“User-Agent”],“acl_history”:"[[ELKG admin to kibana->[auth_key->false]], [Kibana Server (we trust this server side component, full access granted via HTTP authentication)->[auth_key->false]], [ELKG admin to general->[auth_key->false]], [Logstash can write and create its own indices->[indices->true, auth_key->true, actions->true]]]",“origin”:“127.0.0.1”,“final_state”:“ALLOWED”,“task_id”:null,“type”:“BulkRequest”,“req_method”:“POST”,“path”:"/_bulk",“indices”:[“log_lu_ei_tel_telephony-2017.11.14”],"@timestamp":“2017-11-14T14:33:21Z”,“content_len_kb”:3,“error_type”:null,“processingMillis”:0,“action”:“indices:data/write/bulk”,“id”:“249889551-1618737383”,“content_len”:3844,“user”:“logstash_BE_TEST”}] cannot contain ‘.’]``
`at org.elasticsearch.index.mapper.object.ObjectMapper$TypeParser.parseProperties(ObjectMapper.java:277)
at org.elasticsearch.index.mapper.object.ObjectMapper$TypeParser.parseObjectOrDocumentTypeProperties(ObjectMapper.java:222)
at org.elasticsearch.index.mapper.object.RootObjectMapper$TypeParser.parse(RootObjectMapper.java:139)
at org.elasticsearch.index.mapper.DocumentMapperParser.parse(DocumentMapperParser.java:118)
at org.elasticsearch.index.mapper.DocumentMapperParser.parse(DocumentMapperParser.java:99)
at org.elasticsearch.index.mapper.MapperService.parse(MapperService.java:549)
at org.elasticsearch.cluster.metadata.MetaDataMappingService$PutMappingExecutor.applyRequest(MetaDataMappingService.java:257)
at org.elasticsearch.cluster.metadata.MetaDataMappingService$PutMappingExecutor.execute(MetaDataMappingService.java:230)
at org.elasticsearch.cluster.service.InternalClusterService.runTasksForExecutor(InternalClusterService.java:480)
at org.elasticsearch.cluster.service.InternalClusterService$UpdateTask.run(InternalClusterService.java:784)
at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:231)
at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:194)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source) `
example here an extract if es.logger.level : INFO
[2017-11-14 15:42:45,379][ERROR][tech.beshu.ror.es.AuditSinkImpl] 1x: MapperParsingException[Field name [{"error_message":null,"headers":["Accept","Accept-Encoding","Accept-Language","Authorization","Connection","Host","Referer","User-Agent"],"acl_history":"[[ELKG admin to kibana->[kibana_access->false, auth_key->true]], [ELKG admin to general->[auth_key->true, actions->true]]]","origin":"127.0.0.1","final_state":"ALLOWED","task_id":null,"type":"ClusterStateRequest","req_method":"GET","path":"/_cluster/settings","indices":[],"@timestamp":"2017-11-14T14:42:44Z","content_len_kb":0,"error_type":null,"processingMillis":0,"action":"cluster:monitor/state","id":"1249810828-1992724869","content_len":0,"user":"Adm_elkg_test"}] cannot contain '.']
[2017-11-14 15:42:46,947][INFO ][tech.beshu.ror.acl.ACL ] e[36mALLOWED by { name: 'Kibana Server (we trust this server side component, full access granted via HTTP authentication)', policy: ALLOW} req={ ID:214237159-1826245360, TYP:NodesInfoRequest, CGR:N/A, USR:elkg_kibana_test, BRS:false, ACT:cluster:monitor/nodes/info, OA:127.0.0.1, IDX:<N/A>, MET:GET, PTH:/_nodes, CNT:<N/A>, HDR:Authorization,Connection,Host, HIS:[ELKG admin to general->[auth_key->false]], [Kibana Server (we trust this server side component, full access granted via HTTP authentication)->[hosts->true, auth_key->true]], [ELKG admin to kibana->[auth_key->false]] } e[0m
[2017-11-14 15:42:46,947][INFO ][tech.beshu.ror.acl.ACL ] e[36mALLOWED by { name: 'Kibana Server (we trust this server side component, full access granted via HTTP authentication)', policy: ALLOW} req={ ID:2044458432-557819982, TYP:ClusterHealthRequest, CGR:N/A, USR:elkg_kibana_test, BRS:false, ACT:cluster:monitor/health, OA:127.0.0.1, IDX:.kibana, MET:GET, PTH:/_cluster/health/.kibana?timeout=5s, CNT:<N/A>, HDR:Authorization,Connection,Host, HIS:[ELKG admin to general->[auth_key->false]], [Kibana Server (we trust this server side component, full access granted via HTTP authentication)->[hosts->true, auth_key->true]], [ELKG admin to kibana->[auth_key->false]] } e[0m
[2017-11-14 15:42:46,947][INFO ][tech.beshu.ror.acl.ACL ] e[36mALLOWED by { name: 'Kibana Server (we trust this server side component, full access granted via HTTP authentication)', policy: ALLOW} req={ ID:843399171-2077534023, TYP:SearchRequest, CGR:N/A, USR:elkg_kibana_test, BRS:false, ACT:indices:data/read/search, OA:127.0.0.1, IDX:.kibana, MET:POST, PTH:/.kibana/config/_search, CNT:<OMITTED, LENGTH=77>, HDR:Authorization,Connection,Content-Length,Host, HIS:[Kibana Server (we trust this server side component, full access granted via HTTP authentication)->[hosts->true, auth_key->true]], [ELKG admin to general->[auth_key->false]], [ELKG admin to kibana->[auth_key->false]] } e[0m
[2017-11-14 15:42:47,384][ERROR][tech.beshu.ror.es.AuditSinkImpl] Some failures flushing the BulkProcessor:
[2017-11-14 15:42:47,384][ERROR][tech.beshu.ror.es.AuditSinkImpl] 1x: MapperParsingException[Field name [{"error_message":null,"headers":["Authorization","Connection","Content-Length","Host"],"acl_history":"[[Kibana Server (we trust this server side component, full access granted via HTTP authentication)->[hosts->true, auth_key->true]], [ELKG admin to general->[auth_key->false]], [ELKG admin to kibana->[auth_key->false]]]","origin":"127.0.0.1","final_state":"ALLOWED","task_id":null,"type":"SearchRequest","req_method":"POST","path":"/.kibana/config/_search","indices":[".kibana"],"@timestamp":"2017-11-14T14:42:46Z","content_len_kb":0,"error_type":null,"processingMillis":0,"action":"indices:data/read/search","id":"843399171-2077534023","content_len":77,"user":"elkg_kibana_test"}] cannot contain '.']
[2017-11-14 15:42:47,384][ERROR][tech.beshu.ror.es.AuditSinkImpl] 1x: MapperParsingException[Field name [{"error_message":null,"headers":["Authorization","Connection","Host"],"acl_history":"[[ELKG admin to general->[auth_key->false]], [Kibana Server (we trust this server side component, full access granted via HTTP authentication)->[hosts->true, auth_key->true]], [ELKG admin to kibana->[auth_key->false]]]","origin":"127.0.0.1","final_state":"ALLOWED","task_id":null,"type":"ClusterHealthRequest","req_method":"GET","path":"/_cluster/health/.kibana?timeout=5s","indices":[".kibana"],"@timestamp":"2017-11-14T14:42:46Z","content_len_kb":0,"error_type":null,"processingMillis":0,"action":"cluster:monitor/health","id":"2044458432-557819982","content_len":0,"user":"elkg_kibana_test"}] cannot contain '.']
[2017-11-14 15:42:47,384][ERROR][tech.beshu.ror.es.AuditSinkImpl] 1x: MapperParsingException[Field name [{"error_message":null,"headers":["Authorization","Connection","Host"],"acl_history":"[[ELKG admin to general->[auth_key->false]], [Kibana Server (we trust this server side component, full access granted via HTTP authentication)->[hosts->true, auth_key->true]], [ELKG admin to kibana->[auth_key->false]]]","origin":"127.0.0.1","final_state":"ALLOWED","task_id":null,"type":"NodesInfoRequest","req_method":"GET","path":"/_nodes","indices":[],"@timestamp":"2017-11-14T14:42:46Z","content_len_kb":0,"error_type":null,"processingMillis":0,"action":"cluster:monitor/nodes/info","id":"214237159-1826245360","content_len":0,"user":"elkg_kibana_test"}] cannot contain '.']
[Issue 3] : bah, was testing, and in kibana, basic authentication : if you enter a login name, but you let a blank password, you get a fatal error
Error: unhandled courier request error: [illegal_argument_exception] Cannot extract user name from base auth header
[Issue 4] : regarding Audit_collector - I just noticed that an indices has been created in my Es, but no data are populated in.
maybe it is related to Issue 2 .
If you see/want some other test, tell me.
Thank you very much @ld57 for testing all this!
issue1
This is a known issue in windows that I clearly need help with.
issue2
Are you sure this is a complete stack trace? Looks like the first part is missing?
Anyway looks like we are trying to serialise to JSON some object or map that contains a dot ‘.’ in a JSON field name. We need to know which one.
issue3
It’s a shit error message, actually the code just doesn’t see the separator ‘:’ and it can mean either that you sent garbage or you forgot to send the password
