ReadonlyREST Audit logs

(Simone Scarduzio) #1

Hi everyone,
As you know, ReadonlyREST has the capability to inspect incoming HTTP requests and extrapolate a lot of useful information from each of them i.e.

  • What indices is this request targeting
  • What action?
  • What IP does it come from?
  • What’s the identity of the requestor?

Until now we just printed log lines like this:

ALLOWED by '{ block=::RW::, match=true }' req={ ID:988237926--1331598758#34982, TYP:SearchRequest, USR:simone, BRS:true, ACT:indices:data/read/search, OA:, IDX:readonlyrest_audit-2017-06-29, MET:GET, PTH:/readonlyrest_audit-2017-06-29/_search, CNT:<OMITTED, LENGTH=0>, HDR:Accept,Authorization,content-length,Host,User-Agent, HIS:[::LOGSTASH::->[auth_key->false]], [::RO::->[auth_key->false]], [kibana->[auth_key->false]], [::RW::->[kibana_access->true, indices->true, kibana_hide_apps->true, auth_key->true]] } 
But what if we dumped all this information as JSON inside an ES index? Data points can be something like this:
    "error_message": null,
    "headers": [
    "acl_history": "[[::LOGSTASH::->[auth_key->false]], [::RW::->[kibana_access->true, indices->true, kibana_hide_apps->true, auth_key->true]], [kibana->[auth_key->false]], [::RO::->[auth_key->false]]]",
    "origin": "",
    "final_state": "ALLOWED",
    "task_id": 1158,
    "type": "SearchRequest",
    "req_method": "GET",
    "path": "/readonlyrest_audit-2017-06-29/_search?pretty",
    "indices": [
    "@timestamp": "2017-06-30T09:41:58Z",
    "content_len_kb": 0,
    "error_type": null,
    "processingMillis": 0,
    "action": "indices:data/read/search",
    "matched_block": "::RW::",
    "id": "933409190-292622897#1158",
    "content_len": 0,
    "user": "simone"

Once this stuff went on Kibana, I came up with these test  visualizations, first  that came to mind really. Pretty sure we can break down this data in an even more interesting way though.

<img src="/uploads/db7709/original/1X/d8ca3a4828cdec888114ce7831f437bdf6fe9a27.png" width="690" height="383">

In clockwise order from left to right: 

1.  A donut chart of whose requests take more time to ES to process.
2. Average request processing time
3. A donut chart of what are the most queried indices
4. A double line graph to correlate cumulative request payload length  to response time. I.e. in a time frame where there is many large bulk inserts, we expect to see higher processing times.