kbnIndex check failed with an unusual error : Invalid index name, already exists as alias

Hello,

I have some slowdown issues with my Kibana setup lately and, when we are having our slowdowns, I am seeing this a lot in our Kibana logs.

I don’t know if it’s related, caused by the issue or causes the issue.

Can someone enlighten me, please?

{
  "type": "log",
  "@timestamp": "2020-10-01T20:35:54Z",
  "tags": [
    "warning",
    "readonlyrest_kbn:ensureIndexExists"
  ],
  "pid": 1,
  "message": "error writing default space to .kibana_test {}"
}

AND:

{
  "type": "log",
  "@timestamp": "2020-10-01T20:43:52Z",
  "tags": [
    "error",
    "readonlyrest_kbn:ensureIndexExists"
  ],
  "pid": 1,
  "message": "kbnIndex check failed with an unusual error: {\"msg\":\"[invalid_index_name_exception] Invalid index name [.kibana_test], already exists as alias, with { index_uuid=\\\"_na_\\\" & index=\\\".kibana_test\\\" }\",\"path\":\"/.kibana_test\",\"query\":{},\"body\":\"REMOVED!!\",\"statusCode\":400,\"response\":\"{\\\"error\\\":{\\\"root_cause\\\":[{\\\"type\\\":\\\"invalid_index_name_exception\\\",\\\"reason\\\":\\\"Invalid index name [.kibana_test], already exists as alias\\\",\\\"index_uuid\\\":\\\"_na_\\\",\\\"index\\\":\\\".kibana_test\\\"}],\\\"type\\\":\\\"invalid_index_name_exception\\\",\\\"reason\\\":\\\"Invalid index name [.kibana_test], already exists as alias\\\",\\\"index_uuid\\\":\\\"_na_\\\",\\\"index\\\":\\\".kibana_test\\\"},\\\"status\\\":400}\"}"
}

I’m seeing that .kibana_test is indeed an alias:

{
  ".kibana_test_2" : {
    "aliases" : {
      ".kibana_test" : { }
    },
}

Thanks!

Every time a non-default tenancy is used (a user accesses a tenancy with a non-default kibana index), ROR checks the index exists, and if spaces feature is enabled, it tries to write the default space document in that index (without even bothering checking if it exists, which may be an overlook from our side).

When the shards are not ready, or too slow responding, the index checker can fail, and all the above described routine fails.

If this is not the first time the user accesses the tenancy, no problem.
But if it’s the first time a user accesses the tenancy and the shard is slow, the index or the default space might fail to be created.