Kibana rotation of readonlyrest_kbn.sessions_index_name: ".readonlyrest_kbn_sessions" index – 8.5.2 and ror 1.47.0

1

Dear All, I’m currently using elasticsearch and kibana in 8.5.2 version with both ror plugins in 1.47.0

Walking through my current production cluster, I see that my index .readonlyrest_kbn_sessions is now larger then 52Gb and has 24360548 docs indexed.

Is there any additional parameter I can add in my kibana.ymlfile to automatically manage the size or rotation ?

readonlyrest_kbn.sessions_index_name: ".readonlyrest_kbn_sessions"

According to the documentation page: (For Kibana | ReadonlyREST)[Kibana 7.8.x and older | ReadonlyREST] I don’t find any paramters to do so.

As a workaround, I setup a dedicated ilm

  1. create an index template that match .readonlyrest_kbn_sessions including an ilm policy
  2. delete the current index: .readonlyrest_kbn_sessions
  3. let the data stream magic operate
DELETE .readonlyrest_kbn_sessions
PUT _index_template/readonlyrest_kbn
{
  "index_patterns": [
    ".readonlyrest_kbn_sessions*"
  ],
  "template": {
    "settings": {
      "index": {
        "lifecycle": {
          "name": "ilm-default",
          "rollover_alias": ".readonlyrest_kbn_sessions"
        },
        "number_of_shards": "1",
        "number_of_replicas": "1"
      }
    }
  },
  "composed_of": []
}

GET .readonlyrest_kbn_sessions
GET _data_stream/.readonlyrest_kbn_sessions
GET .ds-.readonlyrest_kbn_sessions-2023.02.27-000001/_ilm/explain

It works, but I don’t see any document in my data stream :confused:

# GET _cat/indices/.readonlyrest_kbn_sessions/?v&h=index,store.size 200 OK
index                                            store.size
.ds-.readonlyrest_kbn_sessions-2023.02.27-000001       450b

# GET .readonlyrest_kbn_sessions/_ilm/explain 200 OK
{
  "indices": {
    ".ds-.readonlyrest_kbn_sessions-2023.02.27-000001": {
      "index": ".ds-.readonlyrest_kbn_sessions-2023.02.27-000001",
      "managed": true,
      "policy": "ilm-default",
      "index_creation_date_millis": 1677510630018,
      "time_since_index_creation": "5.14m",
      "lifecycle_date_millis": 1677510630018,
      "age": "5.14m",
      "phase": "hot",
      "phase_time_millis": 1677510630266,
      "action": "rollover",
      "action_time_millis": 1677510630468,
      "step": "check-rollover-ready",
      "step_time_millis": 1677510630468,
      "phase_execution": {
        "policy": "ilm-default",
        "phase_definition": {
          "min_age": "0ms",
          "actions": {
            "rollover": {
              "max_primary_shard_size": "20gb"
            }
          }
        },
        "version": 4,
        "modified_date_in_millis": 1677066194415
      }
    }
  }
}

Any idea ? :confused:

kr,

1 Like
2

Yes this is something we have to look at automating. Good catch!
@Dzuming would you please add this to Jira?

1 Like
3

This morning, I still have no entries in my new index (data_stream) :confused: ; Despite the fact that other users logged in.

That being said, we didn’t encountered or spotted any bad behavior for kibana open session so far ;

# GET .readonlyrest_kbn_sessions/_settings 200 OK
{
  ".ds-.readonlyrest_kbn_sessions-2023.02.28-000001": {
    "settings": {
      "index": {
        "lifecycle": {
          "name": "ilm-default",
          "rollover_alias": ".readonlyrest_kbn_sessions"
        },
        "routing": {
          "allocation": {
            "include": {
              "_tier_preference": "data_hot"
            }
          }
        },
        "hidden": "true",
        "number_of_shards": "1",
        "provided_name": ".ds-.readonlyrest_kbn_sessions-2023.02.28-000001",
        "creation_date": "1677572819783",
        "number_of_replicas": "1",
        "uuid": "vjz87cJ4Qgm6Eh8uDtKzLw",
        "version": {
          "created": "8050299"
        }
      }
    }
  }
}
# GET .readonlyrest_kbn_sessions/_count 200 OK
{
  "count": 0,
  "_shards": {
    "total": 1,
    "successful": 1,
    "skipped": 0,
    "failed": 0
  }
}
# GET /_data_stream/.readonlyrest_kbn_sessions/_stats?human=true 200 OK
{
  "_shards": {
    "total": 2,
    "successful": 2,
    "failed": 0
  },
  "data_stream_count": 1,
  "backing_indices": 1,
  "total_store_size": "450b",
  "total_store_size_bytes": 450,
  "data_streams": [
    {
      "data_stream": ".readonlyrest_kbn_sessions",
      "backing_indices": 1,
      "store_size": "450b",
      "store_size_bytes": 450,
      "maximum_timestamp": 0
    }
  ]
}
4

Hello @orsius

Can we close this topic? We already have an automatic session cleanup mechanism implemented some time ago.

5

Hi, yes sure you can close this one.

1 Like