Unable to load beat dashboards on kibana

Users Feedback
1

The ROR is fullly configured with Elasticsearch and Kibana. I can use it. But i am unable to load dasboards

PS C:\Program Files\WinlogBeat> .\winlogbeat.exe setup --dashboards
Loading dashboards (Kibana must be running and reachable)
Exiting: Failed to import dashboard: Failed to load directory C:\Program Files\WinlogBeat\kibana/7/dashboard:
  error loading C:\Program Files\WinlogBeat\kibana\7\dashboard\Winlogbeat-overview.json: fail to execute the HTTP POST request: Post /api/kibana/dashboards/import: stopped after 10 redirects. Response:
PS C:\Program Files\WinlogBeat>

PS: Kibana auth has been set in winlogbeat.yml

ROR config

readonlyrest:
        access_control_rules:
                - name: "Require HTTP Basic Auth"
                  type: allow
                  auth_key: user:password
2

Look at the ES logs, find the log line saying “FORBIDDEN”, there will tell you what went wrong (see “HIS:” field). Paste it here, so we can help.

3

I think its an error of kibana…

4

Hmmm yeah, now I get what’s going on. This is a pain in the ass when tools like Logstash and similar want to talk to Kibana API directly and create the dashboards. We have a configuration in our Kibana plugins that will (temporarily) allow them to connect without authentication.

Please add this to kibana.yml, restart Kibana, make it create the dashboards, remove it, and restart Kibana.

readonlyrest_kbn.whitelistedPaths: ["*"]
5

I am getting

{
  "statusCode": 500,
  "error": "Internal Server Error",
  "message": "An internal server error occurred"
}

And this in kibana journal logs

earch":null,"query":{},"pathname":"/api/status","path":"/api/status","href":"/api/status"},"message":"Invalid regular expression: /*|/status|/api/status/: Nothing to repeat"}
Apr 28 02:27:56 ubuntu-s-2vcpu-4gb-blr1-01 kibana[14874]: {"type":"response","@timestamp":"2020-04-27T20:57:56Z","tags":[],"pid":14874,"method":"get","statusCode":500,"req":{"url":"/api/status","method":"get","headers":{"host":"localhost:5601","user-agent":"Go-http-client/1.1","accept-encoding":"gzip"},"remoteAddress":"127.0.0.1","userAgent":"127.0.0.1"},"res":{"statusCode":500,"responseTime":56,"contentLength":9},"message":"GET /api/status 500 56ms - 9.0B"}
Apr 28 02:28:06 ubuntu-s-2vcpu-4gb-blr1-01 kibana[14874]: {"type":"log","@timestamp":"2020-04-27T20:58:06Z","tags":["error","readonlyrest_kbn:onPreResponse"],"pid":14874,"message":"got an error [500] Internal Server Error for path /api/status"}
Apr 28 02:28:06 ubuntu-s-2vcpu-4gb-blr1-01 kibana[14874]: {"type":"log","@timestamp":"2020-04-27T20:58:06Z","tags":["error","readonlyrest_kbn:onPreResponse"],"pid":14874,"message":"ES just returned an error stack trace error, will return the useful error."}
Apr 28 02:28:06 ubuntu-s-2vcpu-4gb-blr1-01 metricbeat[14629]: 2020-04-28T02:28:06.812+0530        INFO        module/wrapper.go:252        Error fetching data for metricset kibana.status: HTTP error 500 in : 500 Internal Server Error
Apr 28 02:28:06 ubuntu-s-2vcpu-4gb-blr1-01 kibana[14874]: Debug: internal, implementation, error
Apr 28 02:28:06 ubuntu-s-2vcpu-4gb-blr1-01 kibana[14874]:     SyntaxError: Invalid regular expression: /*|/status|/api/status/: Nothing to repeat
Apr 28 02:28:06 ubuntu-s-2vcpu-4gb-blr1-01 kibana[14874]:     at new RegExp (<anonymous>)
Apr 28 02:28:06 ubuntu-s-2vcpu-4gb-blr1-01 kibana[14874]:     at Object.isWhitelistedPath (/usr/share/kibana/plugins/readonlyrest_kbn/server/routes/lib/constants.js:169:74)
Apr 28 02:28:06 ubuntu-s-2vcpu-4gb-blr1-01 kibana[14874]:     at server.ext (/usr/share/kibana/plugins/readonlyrest_kbn/server/routes/lib/auth.js:224:19)
Apr 28 02:28:06 ubuntu-s-2vcpu-4gb-blr1-01 kibana[14874]:     at module.exports.internals.Manager.execute (/usr/share/kibana/node_modules/hapi/lib/toolkit.js:35:106)
Apr 28 02:28:06 ubuntu-s-2vcpu-4gb-blr1-01 kibana[14874]:     at Request._invoke (/usr/share/kibana/node_modules/hapi/lib/request.js:293:55)
Apr 28 02:28:06 ubuntu-s-2vcpu-4gb-blr1-01 kibana[14874]:     at process._tickCallback (internal/process/next_tick.js:68:7)
Apr 28 02:28:06 ubuntu-s-2vcpu-4gb-blr1-01 kibana[14874]: {"type":"error","@timestamp":"2020-04-27T20:58:06Z","tags":[],"pid":14874,"level":"error","error":{"message":"Invalid regular expression: /*|/status|/api/status/: Nothing to repeat","name":"SyntaxError","stack":"SyntaxError: Invalid regular expression: /*|/status|/api/status/: Nothing to repeat\n    at new RegExp (<anonymous>)\n    at Object.isWhitelistedPath (/usr/share/kibana/plugins/readonlyrest_kbn/server/routes/lib/constants.js:169:74)\n    at server.ext (/usr/share/kibana/plugins/readonlyrest_kbn/server/routes/lib/auth.js:224:19)\n    at module.exports.internals.Manager.execute (/usr/share/kibana/node_modules/hapi/lib/toolkit.js:35:106)\n    at Request._invoke (/usr/share/kibana/node_modules/hapi/lib/request.js:293:55)\n    at process._tickCallback (internal/process/next_tick.js:68:7)"},"url":{"protocol":null,"slashes":null,"auth":null,"host":null,"port":null,"hostname":null,"hash":null,"search":null,"query":{},"pathname":"/api/status","path":"/api/status","href":"/api/status"},"message":"Invalid regular expression: /*|/status|/api/status/: Nothing to repeat"}
Apr 28 02:28:06 ubuntu-s-2vcpu-4gb-blr1-01 kibana[14874]: {"type":"response","@timestamp":"2020-04-27T20:58:06Z","tags":[],"pid":14874,"method":"get","statusCode":500,"req":{"url":"/api/status","method":"get","headers":{"host":"localhost:5601","user-agent":"Go-http-client/1.1","accept-encoding":"gzip"},"remoteAddress":"127.0.0.1","userAgent":"127.0.0.1"},"res":{"statusCode":500,"responseTime":47,"contentLength":9},"message":"GET /api/status 500 47ms - 9.0B"}
6

@sscarduzio i tried to fix it from readonlyrest-docs/kibana.md at master · beshu-tech/readonlyrest-docs · GitHub by chaning * to ./*

Now i am getting

{
  "statusCode": 400,
  "error": "Bad Request",
  "message": "Error: Unauthorized"
}

and no error in journalctl

7

ok apparently the problem is that whitelistedPaths lets you in kibana, but if you try to do something that persists to Elasticsearch, it doesn’t let you inherit Kibana server credentials and ES bounces you.

I suggest to uninstall readonlyrest from Kibana and reinstall it after you are done. We’ll will fix that bug ASAP.

8

I mean Kibana’s plugin

9

The correct procedure is given on documentation, after i followed it, i have created dashboard, but the new problem is coming while opening dashboard

[esaggs] > "field" is a required parameter

and

Error fetching fields for index pattern winlogbeat-* (ID: winlogbeat-*)

Wrapper@http://206.189.128.192:5601/bundles/commons.bundle.js:3:4453680
HttpFetchError@http://206.189.128.192:5601/bundles/commons.bundle.js:3:4455462
fetchResponse$@http://206.189.128.192:5601/bundles/commons.bundle.js:3:4451135
s@http://206.189.128.192:5601/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:338:774550
l/i._invoke</<@http://206.189.128.192:5601/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:338:774304
v/</e[t]@http://206.189.128.192:5601/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:338:774907
s@http://206.189.128.192:5601/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:338:774550
t@http://206.189.128.192:5601/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:338:775046
t/<@http://206.189.128.192:5601/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:338:775196

But index exists

image
image1102Ă—336 121 KB

It says this error in visualization

10

OK now you should really look in ES logs. Looks like the ACL is blocking a call that retrieves the fields.

11

I found these logs related to winlogbeat

Apr 27 21:16:15 ubuntu-s-2vcpu-4gb-blr1-01 kibana[15358]: {"type":"response","@timestamp":"2020-04-27T21:16:15Z","tags":[],"pid":15358,"method":"post","statusCode":200,"req":{"url":"/elasticsearch/winlogbeat-*/_search?rest_total_hits_as_int=true&ignore_unavailable=true&ignore_throttled=true&preference=1588022181935&timeout=30000ms","method":"post","headers":{"host":"206.189.128.192:5601","user-agent":"Mozilla/5.0 (X11; Linux x86_64; rv:75.0) Gecko/20100101 Firefox/75.0","accept":"application/json, text/plain, */*","accept-language":"en-US","accept-encoding":"gzip, deflate","content-type":"application/json","kbn-version":"7.6.2","content-length":"952","origin":"http://206.189.128.192:5601","connection":"keep-alive","referer":"http://206.189.128.192:5601/app/kibana","x-ror-kibana-request-path":"/elasticsearch/winlogbeat-*/_search","x-ror-kibana-request-method":"post"},"remoteAddress":"61.0.5.151","userAgent":"61.0.5.151","referer":"http://206.189.128.192:5601/app/kibana"},"res":{"statusCode":200,"responseTime":18,"contentLength":9},"message":"POST /elasticsearch/winlogbeat-*/_search?rest_total_hits_as_int=true&ignore_unavailable=true&ignore_throttled=true&preference=1588022181935&timeout=30000ms 200 18ms - 9.0B"}
Apr 27 21:16:15 ubuntu-s-2vcpu-4gb-blr1-01 kibana[15358]: {"type":"response","@timestamp":"2020-04-27T21:16:15Z","tags":[],"pid":15358,"method":"post","statusCode":200,"req":{"url":"/elasticsearch/winlogbeat-*/_search?rest_total_hits_as_int=true&ignore_unavailable=true&ignore_throttled=true&preference=1588022181935&timeout=30000ms","method":"post","headers":{"host":"206.189.128.192:5601","user-agent":"Mozilla/5.0 (X11; Linux x86_64; rv:75.0) Gecko/20100101 Firefox/75.0","accept":"application/json, text/plain, */*","accept-language":"en-US","accept-encoding":"gzip, deflate","content-type":"application/json","kbn-version":"7.6.2","content-length":"1028","origin":"http://206.189.128.192:5601","connection":"keep-alive","referer":"http://206.189.128.192:5601/app/kibana","x-ror-kibana-request-path":"/elasticsearch/winlogbeat-*/_search","x-ror-kibana-request-method":"post"},"remoteAddress":"61.0.5.151","userAgent":"61.0.5.151","referer":"http://206.189.128.192:5601/app/kibana"},"res":{"statusCode":200,"responseTime":20,"contentLength":9},"message":"POST /elasticsearch/winlogbeat-*/_search?rest_total_hits_as_int=true&ignore_unavailable=true&ignore_throttled=true&preference=1588022181935&timeout=30000ms 200 20ms - 9.0B"}
Apr 27 21:19:52 ubuntu-s-2vcpu-4gb-blr1-01 kibana[15358]: {"type":"response","@timestamp":"2020-04-27T21:19:52Z","tags":[],"pid":15358,"method":"post","statusCode":200,"req":{"url":"/elasticsearch/winlogbeat-*/_search?rest_total_hits_as_int=true&ignore_unavailable=true&ignore_throttled=true&preference=1588022181935&timeout=30000ms","method":"post","headers":{"host":"206.189.128.192:5601","user-agent":"Mozilla/5.0 (X11; Linux x86_64; rv:75.0) Gecko/20100101 Firefox/75.0","accept":"application/json, text/plain, */*","accept-language":"en-US","accept-encoding":"gzip, deflate","content-type":"application/json","kbn-version":"7.6.2","content-length":"952","origin":"http://206.189.128.192:5601","connection":"keep-alive","referer":"http://206.189.128.192:5601/app/kibana","x-ror-kibana-request-path":"/elasticsearch/winlogbeat-*/_search","x-ror-kibana-request-method":"post"},"remoteAddress":"61.0.5.151","userAgent":"61.0.5.151","referer":"http://206.189.128.192:5601/app/kibana"},"res":{"statusCode":200,"responseTime":927,"contentLength":9},"message":"POST /elasticsearch/winlogbeat-*/_search?rest_total_hits_as_int=true&ignore_unavailable=true&ignore_throttled=true&preference=1588022181935&timeout=30000ms 200 927ms - 9.0B"}
Apr 27 21:19:52 ubuntu-s-2vcpu-4gb-blr1-01 kibana[15358]: {"type":"response","@timestamp":"2020-04-27T21:19:52Z","tags":[],"pid":15358,"method":"post","statusCode":200,"req":{"url":"/elasticsearch/winlogbeat-*/_search?rest_total_hits_as_int=true&ignore_unavailable=true&ignore_throttled=true&preference=1588022181935&timeout=30000ms","method":"post","headers":{"host":"206.189.128.192:5601","user-agent":"Mozilla/5.0 (X11; Linux x86_64; rv:75.0) Gecko/20100101 Firefox/75.0","accept":"application/json, text/plain, */*","accept-language":"en-US","accept-encoding":"gzip, deflate","content-type":"application/json","kbn-version":"7.6.2","content-length":"1028","origin":"http://206.189.128.192:5601","connection":"keep-alive","referer":"http://206.189.128.192:5601/app/kibana","x-ror-kibana-request-path":"/elasticsearch/winlogbeat-*/_search","x-ror-kibana-request-method":"post"},"remoteAddress":"61.0.5.151","userAgent":"61.0.5.151","referer":"http://206.189.128.192:5601/app/kibana"},"res":{"statusCode":200,"responseTime":867,"contentLength":9},"message":"POST /elasticsearch/winlogbeat-*/_search?rest_total_hits_as_int=true&ignore_unavailable=true&ignore_throttled=true&preference=1588022181935&timeout=30000ms 200 867ms - 9.0B"}
12

Or could you tell me where i can find these logs, i am a bit new to elk stack

13

I tried this, still the same issue is coming even though applied that fix. Is there any way to temporarily disable ROR?

14

What fix did you try? Just uninstall the kibana plugin, let winlogbeat create its dashboards, and install it again. It’s just two commands.

15

This one, never mind, i have loaded the dashboarded

1 Like