Kibana spins in redirect loop

I have updated the rules to contain a filter. I can log into ‘admin’ fine. But ‘john’ and ‘jane’ sends kibana into a redirect loop. After I get in this state I have restart kibana to clear it up.

I know the rules has a typo for one the ‘jane’ filter. It was in both when this first happened so I fixed just ‘john’ to see if I could login, but that did not work so I’m guessing the filter typo is a decoy.

Without the filter I was able to login to all users, including ‘john’ and ‘jane’. Will investigate further.

    readonlyrest:
      prompt_for_basic_auth: false
#      audit_collector: true
#      audit_serializer: tech.beshu.ror.requestcontext.DefaultAuditLogSerializer

      users:
        - username: john
          auth_key: john:john
          groups: [male]

        - username: jane
          auth_key: jane:jane
          groups: [female]

      access_control_rules:
        - name: CONSUL-SRV
          auth_key: elastic:elastic

        - name: KIBANA-SRV
          auth_key: kibana:kibana

        - name: ADMIN
          auth_key: admin:admin
          kibana_access: admin

        - name: MALE
          groups: [male]
          kibana_access: ro
          indices: [ ".kibana", "kibana_sample_data_ecommerce"]
          filter: '{"bool": { "must": { "match": { "customer_gender": "MALE" }}}}'
          kibana_hide_apps: "{{ kibana_ror_plant_hide_apps }}"

        - name: FEMALE
          groups: [female]
          kibana_access: ro
          indices: [ ".kibana", "kibana_sample_data_ecommerce"]
          filter: '{"bool": { "must": { "match": { " t customer_gender": "FEMALE" }}}}'
          kibana_hide_apps: "{{ kibana_ror_plant_hide_apps }}"

Here are the kibana logs

{"type":"response","@timestamp":"2018-12-19T00:35:40Z","tags":[],"pid":24176,"method":"get","statusCode":302,"req":{"url":"/","method":"get","headers":{"host":"s-ror-es-1.use1.systeminsights.com:5601","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:60.0) Gecko/20100101 Firefox/60.0","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate","referer":"http://s-ror-es-1.use1.systeminsights.com:5601/login","dnt":"1","connection":"keep-alive","upgrade-insecure-requests":"1","x-ror-current-group":"male"},"remoteAddress":"10.0.192.247","userAgent":"10.0.192.247","referer":"http://s-ror-es-1.use1.systeminsights.com:5601/login"},"res":{"statusCode":302,"responseTime":7,"contentLength":9},"message":"GET / 302 7ms - 9.0B"}
{"type":"log","@timestamp":"2018-12-19T00:35:41Z","tags":["spaces","error"],"pid":24176,"message":"Unable to navigate to space \"default\", redirecting to Space Selector. Error: Saved object [space/default] not found"}
{"type":"response","@timestamp":"2018-12-19T00:35:41Z","tags":[],"pid":24176,"method":"get","statusCode":302,"req":{"url":"/app/kibana","method":"get","headers":{"host":"s-ror-es-1.use1.systeminsights.com:5601","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:60.0) Gecko/20100101 Firefox/60.0","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate","referer":"http://s-ror-es-1.use1.systeminsights.com:5601/login","dnt":"1","connection":"keep-alive","upgrade-insecure-requests":"1","cache-control":"max-age=0"},"remoteAddress":"10.0.192.247","userAgent":"10.0.192.247","referer":"http://s-ror-es-1.use1.systeminsights.com:5601/login"},"res":{"statusCode":302,"responseTime":6,"contentLength":9},"message":"GET /app/kibana 302 6ms - 9.0B"}
{"type":"response","@timestamp":"2018-12-19T00:35:41Z","tags":[],"pid":24176,"method":"get","statusCode":302,"req":{"url":"/","method":"get","headers":{"host":"s-ror-es-1.use1.systeminsights.com:5601","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:60.0) Gecko/20100101 Firefox/60.0","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate","referer":"http://s-ror-es-1.use1.systeminsights.com:5601/login","dnt":"1","connection":"keep-alive","upgrade-insecure-requests":"1","cache-control":"max-age=0","x-ror-current-group":"male"},"remoteAddress":"10.0.192.247","userAgent":"10.0.192.247","referer":"http://s-ror-es-1.use1.systeminsights.com:5601/login"},"res":{"statusCode":302,"responseTime":6,"contentLength":9},"message":"GET / 302 6ms - 9.0B"}
{"type":"log","@timestamp":"2018-12-19T00:35:41Z","tags":["spaces","error"],"pid":24176,"message":"Unable to navigate to space \"default\", redirecting to Space Selector. Error: Saved object [space/default] not found"}
{"type":"response","@timestamp":"2018-12-19T00:35:41Z","tags":[],"pid":24176,"method":"get","statusCode":302,"req":{"url":"/app/kibana","method":"get","headers":{"host":"s-ror-es-1.use1.systeminsights.com:5601","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:60.0) Gecko/20100101 Firefox/60.0","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate","referer":"http://s-ror-es-1.use1.systeminsights.com:5601/login","dnt":"1","connection":"keep-alive","upgrade-insecure-requests":"1","cache-control":"max-age=0"},"remoteAddress":"10.0.192.247","userAgent":"10.0.192.247","referer":"http://s-ror-es-1.use1.systeminsights.com:5601/login"},"res":{"statusCode":302,"responseTime":6,"contentLength":9},"message":"GET /app/kibana 302 6ms - 9.0B"}
{"type":"response","@timestamp":"2018-12-19T00:35:41Z","tags":[],"pid":24176,"method":"get","statusCode":302,"req":{"url":"/","method":"get","headers":{"host":"s-ror-es-1.use1.systeminsights.com:5601","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:60.0) Gecko/20100101 Firefox/60.0","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate","referer":"http://s-ror-es-1.use1.systeminsights.com:5601/login","dnt":"1","connection":"keep-alive","upgrade-insecure-requests":"1","cache-control":"max-age=0","x-ror-current-group":"male"},"remoteAddress":"10.0.192.247","userAgent":"10.0.192.247","referer":"http://s-ror-es-1.use1.systeminsights.com:5601/login"},"res":{"statusCode":302,"responseTime":7,"contentLength":9},"message":"GET / 302 7ms - 9.0B"}
{"type":"log","@timestamp":"2018-12-19T00:35:41Z","tags":["spaces","error"],"pid":24176,"message":"Unable to navigate to space \"default\", redirecting to Space Selector. Error: Saved object [space/default] not found"}
{"type":"response","@timestamp":"2018-12-19T00:35:41Z","tags":[],"pid":24176,"method":"get","statusCode":302,"req":{"url":"/app/kibana","method":"get","headers":{"host":"s-ror-es-1.use1.systeminsights.com:5601","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:60.0) Gecko/20100101 Firefox/60.0","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate","referer":"http://s-ror-es-1.use1.systeminsights.com:5601/login","dnt":"1","connection":"keep-alive","upgrade-insecure-requests":"1","cache-control":"max-age=0"},"remoteAddress":"10.0.192.247","userAgent":"10.0.192.247","referer":"http://s-ror-es-1.use1.systeminsights.com:5601/login"},"res":{"statusCode":302,"responseTime":5,"contentLength":9},"message":"GET /app/kibana 302 5ms - 9.0B"}
{"type":"response","@timestamp":"2018-12-19T00:35:42Z","tags":[],"pid":24176,"method":"get","statusCode":302,"req":{"url":"/","method":"get","headers":{"host":"s-ror-es-1.use1.systeminsights.com:5601","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:60.0) Gecko/20100101 Firefox/60.0","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate","referer":"http://s-ror-es-1.use1.systeminsights.com:5601/login","dnt":"1","connection":"keep-alive","upgrade-insecure-requests":"1","cache-control":"max-age=0","x-ror-current-group":"male"},"remoteAddress":"10.0.192.247","userAgent":"10.0.192.247","referer":"http://s-ror-es-1.use1.systeminsights.com:5601/login"},"res":{"statusCode":302,"responseTime":9,"contentLength":9},"message":"GET / 302 9ms - 9.0B"}
{"type":"log","@timestamp":"2018-12-19T00:35:42Z","tags":["spaces","error"],"pid":24176,"message":"Unable to navigate to space \"default\", redirecting to Space Selector. Error: Saved object [space/default] not found"}
{"type":"response","@timestamp":"2018-12-19T00:35:42Z","tags":[],"pid":24176,"method":"get","statusCode":302,"req":{"url":"/app/kibana","method":"get","headers":{"host":"s-ror-es-1.use1.systeminsights.com:5601","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:60.0) Gecko/20100101 Firefox/60.0","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate","referer":"http://s-ror-es-1.use1.systeminsights.com:5601/login","dnt":"1","connection":"keep-alive","upgrade-insecure-requests":"1","cache-control":"max-age=0"},"remoteAddress":"10.0.192.247","userAgent":"10.0.192.247","referer":"http://s-ror-es-1.use1.systeminsights.com:5601/login"},"res":{"statusCode":302,"responseTime":5,"contentLength":9},"message":"GET /app/kibana 302 5ms - 9.0B"}
{"type":"response","@timestamp":"2018-12-19T00:35:42Z","tags":[],"pid":24176,"method":"get","statusCode":302,"req":{"url":"/","method":"get","headers":{"host":"s-ror-es-1.use1.systeminsights.com:5601","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:60.0) Gecko/20100101 Firefox/60.0","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate","referer":"http://s-ror-es-1.use1.systeminsights.com:5601/login","dnt":"1","connection":"keep-alive","upgrade-insecure-requests":"1","cache-control":"max-age=0","x-ror-current-group":"male"},"remoteAddress":"10.0.192.247","userAgent":"10.0.192.247","referer":"http://s-ror-es-1.use1.systeminsights.com:5601/login"},"res":{"statusCode":302,"responseTime":6,"contentLength":9},"message":"GET / 302 6ms - 9.0B"}
{"type":"log","@timestamp":"2018-12-19T00:35:42Z","tags":["spaces","error"],"pid":24176,"message":"Unable to navigate to space \"default\", redirecting to Space Selector. Error: Saved object [space/default] not found"}
{"type":"response","@timestamp":"2018-12-19T00:35:42Z","tags":[],"pid":24176,"method":"get","statusCode":302,"req":{"url":"/app/kibana","method":"get","headers":{"host":"s-ror-es-1.use1.systeminsights.com:5601","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:60.0) Gecko/20100101 Firefox/60.0","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate","referer":"http://s-ror-es-1.use1.systeminsights.com:5601/login","dnt":"1","connection":"keep-alive","upgrade-insecure-requests":"1","cache-control":"max-age=0"},"remoteAddress":"10.0.192.247","userAgent":"10.0.192.247","referer":"http://s-ror-es-1.use1.systeminsights.com:5601/login"},"res":{"statusCode":302,"responseTime":5,"contentLength":9},"message":"GET /app/kibana 302 5ms - 9.0B"}
{"type":"response","@timestamp":"2018-12-19T00:35:43Z","tags":[],"pid":24176,"method":"get","statusCode":302,"req":{"url":"/","method":"get","headers":{"host":"s-ror-es-1.use1.systeminsights.com:5601","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:60.0) Gecko/20100101 Firefox/60.0","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate","referer":"http://s-ror-es-1.use1.systeminsights.com:5601/login","dnt":"1","connection":"keep-alive","upgrade-insecure-requests":"1","cache-control":"max-age=0","x-ror-current-group":"male"},"remoteAddress":"10.0.192.247","userAgent":"10.0.192.247","referer":"http://s-ror-es-1.use1.systeminsights.com:5601/login"},"res":{"statusCode":302,"responseTime":7,"contentLength":9},"message":"GET / 302 7ms - 9.0B"}
{"type":"log","@timestamp":"2018-12-19T00:35:43Z","tags":["spaces","error"],"pid":24176,"message":"Unable to navigate to space \"default\", redirecting to Space Selector. Error: Saved object [space/default] not found"}
{"type":"response","@timestamp":"2018-12-19T00:35:43Z","tags":[],"pid":24176,"method":"get","statusCode":302,"req":{"url":"/app/kibana","method":"get","headers":{"host":"s-ror-es-1.use1.systeminsights.com:5601","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:60.0) Gecko/20100101 Firefox/60.0","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate","referer":"http://s-ror-es-1.use1.systeminsights.com:5601/login","dnt":"1","connection":"keep-alive","upgrade-insecure-requests":"1","cache-control":"max-age=0"},"remoteAddress":"10.0.192.247","userAgent":"10.0.192.247","referer":"http://s-ror-es-1.use1.systeminsights.com:5601/login"},"res":{"statusCode":302,"responseTime":6,"contentLength":9},"message":"GET /app/kibana 302 6ms - 9.0B"}
{"type":"response","@timestamp":"2018-12-19T00:35:43Z","tags":[],"pid":24176,"method":"get","statusCode":302,"req":{"url":"/","method":"get","headers":{"host":"s-ror-es-1.use1.systeminsights.com:5601","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:60.0) Gecko/20100101 Firefox/60.0","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate","referer":"http://s-ror-es-1.use1.systeminsights.com:5601/login","dnt":"1","connection":"keep-alive","upgrade-insecure-requests":"1","cache-control":"max-age=0","x-ror-current-group":"male"},"remoteAddress":"10.0.192.247","userAgent":"10.0.192.247","referer":"http://s-ror-es-1.use1.systeminsights.com:5601/login"},"res":{"statusCode":302,"responseTime":6,"contentLength":9},"message":"GET / 302 6ms - 9.0B"}
{"type":"log","@timestamp":"2018-12-19T00:35:43Z","tags":["spaces","error"],"pid":24176,"message":"Unable to navigate to space \"default\", redirecting to Space Selector. Error: Saved object [space/default] not found"}
{"type":"response","@timestamp":"2018-12-19T00:35:43Z","tags":[],"pid":24176,"method":"get","statusCode":302,"req":{"url":"/app/kibana","method":"get","headers":{"host":"s-ror-es-1.use1.systeminsights.com:5601","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:60.0) Gecko/20100101 Firefox/60.0","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate","referer":"http://s-ror-es-1.use1.systeminsights.com:5601/login","dnt":"1","connection":"keep-alive","upgrade-insecure-requests":"1","cache-control":"max-age=0"},"remoteAddress":"10.0.192.247","userAgent":"10.0.192.247","referer":"http://s-ror-es-1.use1.systeminsights.com:5601/login"},"res":{"statusCode":302,"responseTime":5,"contentLength":9},"message":"GET /app/kibana 302 5ms - 9.0B"}
{"type":"response","@timestamp":"2018-12-19T00:35:43Z","tags":[],"pid":24176,"method":"get","statusCode":302,"req":{"url":"/","method":"get","headers":{"host":"s-ror-es-1.use1.systeminsights.com:5601","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:60.0) Gecko/20100101 Firefox/60.0","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate","referer":"http://s-ror-es-1.use1.systeminsights.com:5601/login","dnt":"1","connection":"keep-alive","upgrade-insecure-requests":"1","cache-control":"max-age=0","x-ror-current-group":"male"},"remoteAddress":"10.0.192.247","userAgent":"10.0.192.247","referer":"http://s-ror-es-1.use1.systeminsights.com:5601/login"},"res":{"statusCode":302,"responseTime":10,"contentLength":9},"message":"GET / 302 10ms - 9.0B"}
{"type":"log","@timestamp":"2018-12-19T00:35:44Z","tags":["spaces","error"],"pid":24176,"message":"Unable to navigate to space \"default\", redirecting to Space Selector. Error: Saved object [space/default] not found"}
{"type":"response","@timestamp":"2018-12-19T00:35:44Z","tags":[],"pid":24176,"method":"get","statusCode":302,"req":{"url":"/app/kibana","method":"get","headers":{"host":"s-ror-es-1.use1.systeminsights.com:5601","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:60.0) Gecko/20100101 Firefox/60.0","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate","referer":"http://s-ror-es-1.use1.systeminsights.com:5601/login","dnt":"1","connection":"keep-alive","upgrade-insecure-requests":"1","cache-control":"max-age=0"},"remoteAddress":"10.0.192.247","userAgent":"10.0.192.247","referer":"http://s-ror-es-1.use1.systeminsights.com:5601/login"},"res":{"statusCode":302,"responseTime":5,"contentLength":9},"message":"GET /app/kibana 302 5ms - 9.0B"}
{"type":"response","@timestamp":"2018-12-19T00:35:44Z","tags":[],"pid":24176,"method":"get","statusCode":302,"req":{"url":"/","method":"get","headers":{"host":"s-ror-es-1.use1.systeminsights.com:5601","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:60.0) Gecko/20100101 Firefox/60.0","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate","referer":"http://s-ror-es-1.use1.systeminsights.com:5601/login","dnt":"1","connection":"keep-alive","upgrade-insecure-requests":"1","cache-control":"max-age=0","x-ror-current-group":"male"},"remoteAddress":"10.0.192.247","userAgent":"10.0.192.247","referer":"http://s-ror-es-1.use1.systeminsights.com:5601/login"},"res":{"statusCode":302,"responseTime":6,"contentLength":9},"message":"GET / 302 6ms - 9.0B"}
{"type":"log","@timestamp":"2018-12-19T00:35:44Z","tags":["spaces","error"],"pid":24176,"message":"Unable to navigate to space \"default\", redirecting to Space Selector. Error: Saved object [space/default] not found"}
{"type":"response","@timestamp":"2018-12-19T00:35:44Z","tags":[],"pid":24176,"method":"get","statusCode":302,"req":{"url":"/app/kibana","method":"get","headers":{"host":"s-ror-es-1.use1.systeminsights.com:5601","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:60.0) Gecko/20100101 Firefox/60.0","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate","referer":"http://s-ror-es-1.use1.systeminsights.com:5601/login","dnt":"1","connection":"keep-alive","upgrade-insecure-requests":"1","cache-control":"max-age=0"},"remoteAddress":"10.0.192.247","userAgent":"10.0.192.247","referer":"http://s-ror-es-1.use1.systeminsights.com:5601/login"},"res":{"statusCode":302,"responseTime":5,"contentLength":9},"message":"GET /app/kibana 302 5ms - 9.0B"}
{"type":"response","@timestamp":"2018-12-19T00:35:44Z","tags":[],"pid":24176,"method":"get","statusCode":302,"req":{"url":"/","method":"get","headers":{"host":"s-ror-es-1.use1.systeminsights.com:5601","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:60.0) Gecko/20100101 Firefox/60.0","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate","referer":"http://s-ror-es-1.use1.systeminsights.com:5601/login","dnt":"1","connection":"keep-alive","upgrade-insecure-requests":"1","cache-control":"max-age=0","x-ror-current-group":"male"},"remoteAddress":"10.0.192.247","userAgent":"10.0.192.247","referer":"http://s-ror-es-1.use1.systeminsights.com:5601/login"},"res":{"statusCode":302,"responseTime":5,"contentLength":9},"message":"GET / 302 5ms - 9.0B"}
{"type":"log","@timestamp":"2018-12-19T00:35:44Z","tags":["spaces","error"],"pid":24176,"message":"Unable to navigate to space \"default\", redirecting to Space Selector. Error: Saved object [space/default] not found"}
{"type":"response","@timestamp":"2018-12-19T00:35:44Z","tags":[],"pid":24176,"method":"get","statusCode":302,"req":{"url":"/app/kibana","method":"get","headers":{"host":"s-ror-es-1.use1.systeminsights.com:5601","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:60.0) Gecko/20100101 Firefox/60.0","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate","referer":"http://s-ror-es-1.use1.systeminsights.com:5601/login","dnt":"1","connection":"keep-alive","upgrade-insecure-requests":"1","cache-control":"max-age=0"},"remoteAddress":"10.0.192.247","userAgent":"10.0.192.247","referer":"http://s-ror-es-1.use1.systeminsights.com:5601/login"},"res":{"statusCode":302,"responseTime":5,"contentLength":9},"message":"GET /app/kibana 302 5ms - 9.0B"}
{"type":"response","@timestamp":"2018-12-19T00:35:45Z","tags":[],"pid":24176,"method":"get","statusCode":302,"req":{"url":"/","method":"get","headers":{"host":"s-ror-es-1.use1.systeminsights.com:5601","user-agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:60.0) Gecko/20100101 Firefox/60.0","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8","accept-language":"en-US,en;q=0.5","accept-encoding":"gzip, deflate","referer":"http://s-ror-es-1.use1.systeminsights.com:5601/login","dnt":"1","connection":"keep-alive","upgrade-insecure-requests":"1","cache-control":"max-age=0","x-ror-current-group":"male"},"remoteAddress":"10.0.192.247","userAgent":"10.0.192.247","referer":"http://s-ror-es-1.use1.systeminsights.com:5601/login"},"res":{"statusCode":302,"responseTime":5,"contentLength":9},"message":"GET / 302 5ms - 9.0B"}

A query via api works as expected. The following returns only the customer_gender:MALE documents.

curl --user 'john:john' -X GET "10.11.136.187:9200/kibana_sample_data_ecommerce/_search" -H 'Content-Type: application/json' -d'
{
    "query": {
        "match_all": {}
    }
}
'

Removed all kibana_hide_apps, still loops.

Not sure I saw this in the error logs snippet above, but looks like a good clue

{"type":"log","@timestamp":"2018-12-19T00:53:44Z","tags":["spaces","error"],"pid":24251,"message":"Unable to navigate to space \"default\", redirecting to Space Selector. Error: Saved object [space/default] not found"}

Here is a full log from kibana startup to redirect loop

Ok, can’t upload a log file… https://gist.github.com/memelet/6958c26a25478b70b35c20c9e2ef2169

I removed the filter for ‘john’ while ‘john’ was logged into kibana. Bounced kibana. Via discover the filter had the correct effect. Logged out, but when trying to log back in got the loop.

Its hard to see why the filter would have this effect.

es: 6.5.3 / readonlyrest-1.16.31_es6.5.3.zip
kibana: 6.5.3 / readonlyrest_kbn_enterprise-1.16.31-20181215_es6.5.3.zip

This happens because “filter” and “fields” are read-only rules. You should clone the ACL blocks that in your example contain “filter”, the first one with the filter rule to intercept the read requests, the second one, without it.

See the “IMPORTANT” notice in the docs.

Damn, I read that. Sorry.

But what index was trying to be written to? kibana_access:ro and nothing was trying to write to kibana_sample_data_ecommerce.

So you mean this?

        - name: MALE
          groups: [male]
          kibana_access: ro
          indices: [ ".kibana", "kibana_sample_data_ecommerce"]
          filter: '{"bool": { "must": { "match": { "customer_gender": "MALE" }}}}'

        - name: MALE-2
          groups: [male]
          kibana_access: ro
          indices: [ ".kibana", "kibana_sample_data_ecommerce"]

        - name: FEMALE
          groups: [female]
          kibana_access: ro
          indices: [ ".kibana", "kibana_sample_data_ecommerce"]
          filter: '{"bool": { "must": { "match": { "customer_gender": "FEMALE" }}}}'

        - name: FEMALE-2
          groups: [female]
          kibana_access: ro
          indices: [ ".kibana", "kibana_sample_data_ecommerce"]

If you want to allow write requests…

Because I do NOT want to allow write requests, except maybe in a user-specific kibana index when I get that far.

Yes but the blocks name need to be unique

still redirect loop with those rules

Explore the ES audit logs and see some line saying “FORBIDDEN”

Well one FORBIDDEN is for

ACT:cluster:monitor/main MET:GET PTH:/ IDX:<N/A>

Will I have to just add rules for these kinds of requests?

But besides, if a user does not have access to index they won’t be able to even login to kibana? I must be missing something here.

And there also the bit that the only way to recover from this is to restart the kibana process. This will be a problem in production.

It’s just that on the first “FORBIDDEN” the user is logged out. Normally being logged out does not trigger a redirect loop because it deletes the authentication cookie. Are you using a reverse proxy instead of the ROR login form?

No, running kibana right next to es. I am getting the ror login form.

Ok, the ACT:cluster:monitor/main is a decoy. Its my consul health check. I have not updated its password. I’ll stop that…

So when I login with ‘john’ I get only ALLOWED, but still redirect spin.

I cleared all cookies. Still spin. But the cookie does get set even with the redirects.