I actually see that it happens every time kibana starts. So I get a new .kibana_N+1 every single time I start kibana.