[Feedback] Extended tests of 1.16.14.pre1 on ES 2.x


(Ld57) #1

Hi all,

I will implement in Pre-Production the 1.16.14.pre1 version next week , and will give feedbacks about behavior on 2.x infrastructure.

regarding our 5.x infrastructure, unfortunately the owner is unavailable for now, we will have to wait.

KR

Fred


(Ld57) #18

Okay, here are feedbacks using 1.16.14

A. installing and retro compatibility OK with old rules methodes (1.13.2 and below) : OK
B. LDAP test : OK Yay Ponies !! - I had to change some approach on my side, I will share later
C. Logging compatibility test : _OK _
D. Auditing compatibility test : See Issue 2 and Issue 4
E. test case below : OK

  • in kibana you defined an index patter like log*
  • and in ES you have indices log-apache* and log-paloalto*
  • and the user is authorized for log-paloalto* but not for log-apache*
  • and the user use kibana discover or any kibana visualisation built on index pattern log* and users see only data from log-paloalto* (without credential request, transparent - in fact it is a behavior test )

F. readonlyrest config not showed anymore in /_nodes/_all since hosted in separated file. : OK



Issues:


[Issue 2] Audit_collector: true :
Currently in logging.xml :
es.logger.level: DEBUG

I get some errors in logs, between good messages

[2017-11-14 15:33:22,268][DEBUG][cluster.service ] [ELKG_TEST] cluster state update task [put-mapping [ror_audit_evt]] failed
`MapperParsingException[Field name [{“error_message”:null,“headers”:[“Accept-Encoding”,“Authorization”,“Connection”,“Content-Length”,“Content-Type”,“Host”,“User-Agent”],“acl_history”:"[[ELKG admin to kibana->[auth_key->false]], [Kibana Server (we trust this server side component, full access granted via HTTP authentication)->[auth_key->false]], [ELKG admin to general->[auth_key->false]], [Logstash can write and create its own indices->[indices->true, auth_key->true, actions->true]]]",“origin”:“127.0.0.1”,“final_state”:“ALLOWED”,“task_id”:null,“type”:“BulkRequest”,“req_method”:“POST”,“path”:"/_bulk",“indices”:[“log_lu_ei_tel_telephony-2017.11.14”],"@timestamp":“2017-11-14T14:33:21Z”,“content_len_kb”:3,“error_type”:null,“processingMillis”:0,“action”:“indices:data/write/bulk”,“id”:“249889551-1618737383”,“content_len”:3844,“user”:“logstash_BE_TEST”}] cannot contain ‘.’]``

`at org.elasticsearch.index.mapper.object.ObjectMapper$TypeParser.parseProperties(ObjectMapper.java:277)
at org.elasticsearch.index.mapper.object.ObjectMapper$TypeParser.parseObjectOrDocumentTypeProperties(ObjectMapper.java:222)
at org.elasticsearch.index.mapper.object.RootObjectMapper$TypeParser.parse(RootObjectMapper.java:139)
at org.elasticsearch.index.mapper.DocumentMapperParser.parse(DocumentMapperParser.java:118)
at org.elasticsearch.index.mapper.DocumentMapperParser.parse(DocumentMapperParser.java:99)
at org.elasticsearch.index.mapper.MapperService.parse(MapperService.java:549)
at org.elasticsearch.cluster.metadata.MetaDataMappingService$PutMappingExecutor.applyRequest(MetaDataMappingService.java:257)
at org.elasticsearch.cluster.metadata.MetaDataMappingService$PutMappingExecutor.execute(MetaDataMappingService.java:230)
at org.elasticsearch.cluster.service.InternalClusterService.runTasksForExecutor(InternalClusterService.java:480)
at org.elasticsearch.cluster.service.InternalClusterService$UpdateTask.run(InternalClusterService.java:784)
at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.runAndClean(PrioritizedEsThreadPoolExecutor.java:231)
at org.elasticsearch.common.util.concurrent.PrioritizedEsThreadPoolExecutor$TieBreakingPrioritizedRunnable.run(PrioritizedEsThreadPoolExecutor.java:194)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)  `

example here an extract if es.logger.level : INFO
[2017-11-14 15:42:45,379][ERROR][tech.beshu.ror.es.AuditSinkImpl] 1x: MapperParsingException[Field name [{"error_message":null,"headers":["Accept","Accept-Encoding","Accept-Language","Authorization","Connection","Host","Referer","User-Agent"],"acl_history":"[[ELKG admin to kibana->[kibana_access->false, auth_key->true]], [ELKG admin to general->[auth_key->true, actions->true]]]","origin":"127.0.0.1","final_state":"ALLOWED","task_id":null,"type":"ClusterStateRequest","req_method":"GET","path":"/_cluster/settings","indices":[],"@timestamp":"2017-11-14T14:42:44Z","content_len_kb":0,"error_type":null,"processingMillis":0,"action":"cluster:monitor/state","id":"1249810828-1992724869","content_len":0,"user":"Adm_elkg_test"}] cannot contain '.']
[2017-11-14 15:42:46,947][INFO ][tech.beshu.ror.acl.ACL ] e[36mALLOWED by { name: 'Kibana Server (we trust this server side component, full access granted via HTTP authentication)', policy: ALLOW} req={ ID:214237159-1826245360, TYP:NodesInfoRequest, CGR:N/A, USR:elkg_kibana_test, BRS:false, ACT:cluster:monitor/nodes/info, OA:127.0.0.1, IDX:<N/A>, MET:GET, PTH:/_nodes, CNT:<N/A>, HDR:Authorization,Connection,Host, HIS:[ELKG admin to general->[auth_key->false]], [Kibana Server (we trust this server side component, full access granted via HTTP authentication)->[hosts->true, auth_key->true]], [ELKG admin to kibana->[auth_key->false]] } e[0m
[2017-11-14 15:42:46,947][INFO ][tech.beshu.ror.acl.ACL ] e[36mALLOWED by { name: 'Kibana Server (we trust this server side component, full access granted via HTTP authentication)', policy: ALLOW} req={ ID:2044458432-557819982, TYP:ClusterHealthRequest, CGR:N/A, USR:elkg_kibana_test, BRS:false, ACT:cluster:monitor/health, OA:127.0.0.1, IDX:.kibana, MET:GET, PTH:/_cluster/health/.kibana?timeout=5s, CNT:<N/A>, HDR:Authorization,Connection,Host, HIS:[ELKG admin to general->[auth_key->false]], [Kibana Server (we trust this server side component, full access granted via HTTP authentication)->[hosts->true, auth_key->true]], [ELKG admin to kibana->[auth_key->false]] } e[0m
[2017-11-14 15:42:46,947][INFO ][tech.beshu.ror.acl.ACL ] e[36mALLOWED by { name: 'Kibana Server (we trust this server side component, full access granted via HTTP authentication)', policy: ALLOW} req={ ID:843399171-2077534023, TYP:SearchRequest, CGR:N/A, USR:elkg_kibana_test, BRS:false, ACT:indices:data/read/search, OA:127.0.0.1, IDX:.kibana, MET:POST, PTH:/.kibana/config/_search, CNT:<OMITTED, LENGTH=77>, HDR:Authorization,Connection,Content-Length,Host, HIS:[Kibana Server (we trust this server side component, full access granted via HTTP authentication)->[hosts->true, auth_key->true]], [ELKG admin to general->[auth_key->false]], [ELKG admin to kibana->[auth_key->false]] } e[0m
[2017-11-14 15:42:47,384][ERROR][tech.beshu.ror.es.AuditSinkImpl] Some failures flushing the BulkProcessor:
[2017-11-14 15:42:47,384][ERROR][tech.beshu.ror.es.AuditSinkImpl] 1x: MapperParsingException[Field name [{"error_message":null,"headers":["Authorization","Connection","Content-Length","Host"],"acl_history":"[[Kibana Server (we trust this server side component, full access granted via HTTP authentication)->[hosts->true, auth_key->true]], [ELKG admin to general->[auth_key->false]], [ELKG admin to kibana->[auth_key->false]]]","origin":"127.0.0.1","final_state":"ALLOWED","task_id":null,"type":"SearchRequest","req_method":"POST","path":"/.kibana/config/_search","indices":[".kibana"],"@timestamp":"2017-11-14T14:42:46Z","content_len_kb":0,"error_type":null,"processingMillis":0,"action":"indices:data/read/search","id":"843399171-2077534023","content_len":77,"user":"elkg_kibana_test"}] cannot contain '.']
[2017-11-14 15:42:47,384][ERROR][tech.beshu.ror.es.AuditSinkImpl] 1x: MapperParsingException[Field name [{"error_message":null,"headers":["Authorization","Connection","Host"],"acl_history":"[[ELKG admin to general->[auth_key->false]], [Kibana Server (we trust this server side component, full access granted via HTTP authentication)->[hosts->true, auth_key->true]], [ELKG admin to kibana->[auth_key->false]]]","origin":"127.0.0.1","final_state":"ALLOWED","task_id":null,"type":"ClusterHealthRequest","req_method":"GET","path":"/_cluster/health/.kibana?timeout=5s","indices":[".kibana"],"@timestamp":"2017-11-14T14:42:46Z","content_len_kb":0,"error_type":null,"processingMillis":0,"action":"cluster:monitor/health","id":"2044458432-557819982","content_len":0,"user":"elkg_kibana_test"}] cannot contain '.']
[2017-11-14 15:42:47,384][ERROR][tech.beshu.ror.es.AuditSinkImpl] 1x: MapperParsingException[Field name [{"error_message":null,"headers":["Authorization","Connection","Host"],"acl_history":"[[ELKG admin to general->[auth_key->false]], [Kibana Server (we trust this server side component, full access granted via HTTP authentication)->[hosts->true, auth_key->true]], [ELKG admin to kibana->[auth_key->false]]]","origin":"127.0.0.1","final_state":"ALLOWED","task_id":null,"type":"NodesInfoRequest","req_method":"GET","path":"/_nodes","indices":[],"@timestamp":"2017-11-14T14:42:46Z","content_len_kb":0,"error_type":null,"processingMillis":0,"action":"cluster:monitor/nodes/info","id":"214237159-1826245360","content_len":0,"user":"elkg_kibana_test"}] cannot contain '.']


[Issue 3] : bah, was testing, and in kibana, basic authentication : if you enter a login name, but you let a blank password, you get a fatal error
Error: unhandled courier request error: [illegal_argument_exception] Cannot extract user name from base auth header


[Issue 4] : regarding Audit_collector - I just noticed that an indices has been created in my Es, but no data are populated in.

maybe it is related to Issue 2 .



If you see/want some other test, tell me.


(Simone Scarduzio) #19

Thank you very much @ld57 for testing all this!

issue1
This is a known issue in windows that I clearly need help with.

issue2

Are you sure this is a complete stack trace? Looks like the first part is missing?
Anyway looks like we are trying to serialise to JSON some object or map that contains a dot ‘.’ in a JSON field name. We need to know which one.

issue3
It’s a shit error message, actually the code just doesn’t see the separator ‘:’ and it can mean either that you sent garbage or you forgot to send the password


(Ld57) #20

the error stack comes just after the message
… ] [ELKG_TEST] cluster state update task [put-mapping [ror_audit_evt]] failed

like this