{"statusCode":500,"error":"Internal Server Error","message":"An internal server error occurred"}

narasimmavarman · February 8, 2019, 12:10pm

Hi
While upgrade kibana version into 6.5.4 I face this issue “{“statusCode”:500,“error”:“Internal Server Error”,“message”:“An internal server error occurred”}” can you please help me to resolve this issue

sscarduzio · February 8, 2019, 12:45pm

Hi @narasimmavarman,

Very difficult to help you without a broader context.

Update from what to what?
What ROR version?
Are you an Enteprise user?
Do you use our Kibana plugins?
When does the error occur?
Did you update the ES before updating Kibana?
Do you have the same ROR plugin versions in both ES and Kibana?
Is there any error in ES logs?

narasimmavarman · February 12, 2019, 11:09am

Hi
I’m currently using kibana - 6.2.3 and elasticsearch - 6.2.3, I need to upgrade both into 6.5.4 version. While upgrading I got this issue. From elasticsearch point its working fine, when its come to kibana point its showing error.

Curl Logs:

curl -vvv ‘http://localhost:5601/login’
-H ‘kbn-xsrf: 6.5.4’
-H ‘kbn-version: 6.5.4’
-H ‘Content-Type: application/x-www-form-urlencoded; charset=UTF-8’
-H ‘Accept: application/json, text/javascript, /; q=0.01’
-H ‘Connection: keep-alive’
–data ‘username=admin&password=admin’* Trying 127.0.0.1…
TCP_NODELAY set
Connected to localhost (127.0.0.1) port 5605 (#0)
POST /login HTTP/1.1
Host: localhost:5605
User-Agent: curl/7.60.0
kbn-xsrf: 6.5.4
kbn-version: 6.5.4
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
Accept: application/json, text/javascript, /; q=0.01
Connection: keep-alive
Content-Length: 37
upload completely sent off: 37 out of 37 bytes
< HTTP/1.1 500 Internal Server Error
< content-type: application/json; charset=utf-8
< set-cookie: rorCookie=Fe26.28005627fe3039cf8434c15b5b3fd1f129c74ef9371d5aaf62a4ef52c9e0684daHhaQliTgGjDF13ha0Keo5gX56vNonqxkzu-DBFBLsZTC2fI-_f8zO7bN3HHVmTkc917MvVFhO3nYpePb6JagFgXiJ3J1-RDCI2Qf46imE6Uyk_c3CNgubIh1tvLcsRDohriXTd2dZjPsUluSrzFmBjAaH1H_0fwrNgHuWlCVBTl_UWXMBlletUreVUks4bK2-uQmvzGmPSvu0E84ge0GOfJHlaZ0F8p7GdoSuPPeUciwpHeybpPmf7GBTywS1XQ4wFx-A4FhU2a-lPWxzqdKz8MaqSLPIaQ-TWEKvLmPbOD2hWk_0LS5ovqccqfOsCR5MxyNwiE39thHGTnuH_lO0fV6aj-cqndOfgLG8kWjkWREGeNaUF9bzvAKFA14opSk8099ef41eab7f78a8a7ce6c118f5e9efd65d61283aad50b3b6532e37498831be1*VJgyGaKT_A-iTW0WLQxnqEePs0lLTtQzV2L5bSqIc_0; Max-Age=345600; Expires=Sat, 16 Feb 2019 06:49:51 GMT; HttpOnly; SameSite=Strict; Path=/
< cache-control: no-cache
< content-length: 96
< connection: close
< Date: Tue, 12 Feb 2019 06:49:51 GMT
< Closing connection 0
{“statusCode”:500,“error”:“Internal Server Error”,“message”:“An internal server error occurred”}

Kibana Logs:

error  [08:21:26.875]  Error: Uncaught error: reply interface called twice
    at Object.exports.assert (/home/desa/kibana-6.5.4-linux-x86_64/node_modules/hoek/lib/index.js:740:11)
    at Function.internals.response (/home/desa/kibana-6.5.4-linux-x86_64/node_modules/hapi/lib/reply.js:133:10)
    at reply (/home/desa/kibana-6.5.4-linux-x86_64/node_modules/hapi/lib/reply.js:70:22)
    at a.error.b (/home/desa/kibana-6.5.4-linux-x86_64/plugins/readonlyrest_kbn/server/routes/lib/auth.js:57:25)
    at a.server.app.cache.set.g (/home/desa/kibana-6.5.4-linux-x86_64/plugins/readonlyrest_kbn/server/routes/lib/auth.js:18:24)
    at _cache.set (/home/desa/kibana-6.5.4-linux-x86_64/node_modules/hapi/node_modules/catbox/lib/policy.js:280:16)
    at module.exports.internals.Client.internals.Client.set (/home/desa/kibana-6.5.4-linux-x86_64/node_modules/hapi/node_modules/catbox/lib/client.js:113:16)
    at module.exports.internals.Policy.internals.Policy.set (/home/desa/kibana-6.5.4-linux-x86_64/node_modules/hapi/node_modules/catbox/lib/policy.js:274:17)
    at writeIdentity (/home/desa/kibana-6.5.4-linux-x86_64/plugins/readonlyrest_kbn/server/routes/lib/auth.js:11:28)
    at /home/desa/kibana-6.5.4-linux-x86_64/plugins/readonlyrest_kbn/server/routes/lib/auth.js:54:129
    at _wreck2.default.read (/home/desa/kibana-6.5.4-linux-x86_64/plugins/readonlyrest_kbn/server/routes/lib/identityManager.js:119:13)
    at finish (/home/desa/kibana-6.5.4-linux-x86_64/plugins/readonlyrest_kbn/node_modules/wreck/lib/index.js:369:16)
    at wrapped (/home/desa/kibana-6.5.4-linux-x86_64/plugins/readonlyrest_kbn/node_modules/hoek/lib/index.js:879:20)
    at module.exports.internals.Recorder.onReaderFinish (/home/desa/kibana-6.5.4-linux-x86_64/plugins/readonlyrest_kbn/node_modules/wreck/lib/index.js:415:16)
    at Object.onceWrapper (events.js:313:30)
    at emitNone (events.js:111:20)
    at module.exports.internals.Recorder.emit (events.js:208:7)
    at finishMaybe (_stream_writable.js:613:14)
    at afterWrite (_stream_writable.js:464:3)
    at _combinedTickCallback (internal/process/next_tick.js:145:20)
    at process._tickDomainCallback (internal/process/next_tick.js:219:9)
Debug: internal, implementation, error 
    Error: Uncaught error: reply interface called twice
    at Object.exports.assert (/home/desa/kibana-6.5.4-linux-x86_64/node_modules/hoek/lib/index.js:740:11)
    at Function.internals.response (/home/desa/kibana-6.5.4-linux-x86_64/node_modules/hapi/lib/reply.js:133:10)
    at reply (/home/desa/kibana-6.5.4-linux-x86_64/node_modules/hapi/lib/reply.js:70:22)
    at a.error.b (/home/desa/kibana-6.5.4-linux-x86_64/plugins/readonlyrest_kbn/server/routes/lib/auth.js:57:25)
    at a.server.app.cache.set.g (/home/desa/kibana-6.5.4-linux-x86_64/plugins/readonlyrest_kbn/server/routes/lib/auth.js:18:24)
    at _cache.set (/home/desa/kibana-6.5.4-linux-x86_64/node_modules/hapi/node_modules/catbox/lib/policy.js:280:16)
    at module.exports.internals.Client.internals.Client.set (/home/desa/kibana-6.5.4-linux-x86_64/node_modules/hapi/node_modules/catbox/lib/client.js:113:16)
    at module.exports.internals.Policy.internals.Policy.set (/home/desa/kibana-6.5.4-linux-x86_64/node_modules/hapi/node_modules/catbox/lib/policy.js:274:17)
    at writeIdentity (/home/desa/kibana-6.5.4-linux-x86_64/plugins/readonlyrest_kbn/server/routes/lib/auth.js:11:28)
    at /home/desa/kibana-6.5.4-linux-x86_64/plugins/readonlyrest_kbn/server/routes/lib/auth.js:54:129
    at _wreck2.default.read (/home/desa/kibana-6.5.4-linux-x86_64/plugins/readonlyrest_kbn/server/routes/lib/identityManager.js:119:13)
    at finish (/home/desa/kibana-6.5.4-linux-x86_64/plugins/readonlyrest_kbn/node_modules/wreck/lib/index.js:369:16)
    at wrapped (/home/desa/kibana-6.5.4-linux-x86_64/plugins/readonlyrest_kbn/node_modules/hoek/lib/index.js:879:20)
    at module.exports.internals.Recorder.onReaderFinish (/home/desa/kibana-6.5.4-linux-x86_64/plugins/readonlyrest_kbn/node_modules/wreck/lib/index.js:415:16)
    at Object.onceWrapper (events.js:313:30)
    at emitNone (events.js:111:20)
    at module.exports.internals.Recorder.emit (events.js:208:7)
    at finishMaybe (_stream_writable.js:613:14)
    at afterWrite (_stream_writable.js:464:3)
    at _combinedTickCallback (internal/process/next_tick.js:145:20)
    at process._tickDomainCallback (internal/process/next_tick.js:219:9)
 error  [08:21:26.922] [warning][process] UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'headers' of undefined
    at injectXPackInfoSignature (/home/desa/kibana-6.5.4-linux-x86_64/node_modules/x-pack/plugins/xpack_main/server/lib/inject_xpack_info_signature.js:21:14)
    at <anonymous>
    at emitWarning (internal/process/promises.js:65:17)
    at emitPendingUnhandledRejections (internal/process/promises.js:109:11)
    at runMicrotasksCallback (internal/process/next_tick.js:125:9)

sscarduzio · February 12, 2019, 7:09pm

I just tried myself with the latest build of ROR 1.16.34 and I get normal results.
Tried with valid and invalid credentials. Which plugin version have you installed? What’s the name of the plugin zip file you downloaded?

narasimmavarman · February 13, 2019, 4:50am

I also downloaded the same readonlyrest-1.16.34_es6.5.4.zip file from your official website and installed, but I got an issue. Please help us to resolve this issue. we stuck on this.

log [18:30:00.000] [error][readonlyrest_kbn] error writing identity:

sscarduzio · February 14, 2019, 12:12pm

No more logs under that line? Also, please share your readonlyrest.yml!

narasimmavarman · February 14, 2019, 12:42pm

Yes, Here i have mentioned logs after error writing identity,

error  [04:57:36.577]  Error: Uncaught error: reply interface called twice
   at Object.exports.assert (/home/desa/kibana-6.5.4-linux-x86_64/node_modules/hoek/lib/index.js:740:11)
   at Function.internals.response (/home/desa/kibana-6.5.4-linux-x86_64/node_modules/hapi/lib/reply.js:133:10)
   at reply (/home/desa/kibana-6.5.4-linux-x86_64/node_modules/hapi/lib/reply.js:70:22)
   at a.error.b (/home/desa/kibana-6.5.4-linux-x86_64/plugins/readonlyrest_kbn/server/routes/lib/auth.js:57:25)
   at a.server.app.cache.set.g (/home/desa/kibana-6.5.4-linux-x86_64/plugins/readonlyrest_kbn/server/routes/lib/auth.js:18:24)
   at _cache.set (/home/desa/kibana-6.5.4-linux-x86_64/node_modules/hapi/node_modules/catbox/lib/policy.js:280:16)
   at module.exports.internals.Client.internals.Client.set (/home/desa/kibana-6.5.4-linux-x86_64/node_modules/hapi/node_modules/catbox/lib/client.js:113:16)
   at module.exports.internals.Policy.internals.Policy.set (/home/desa/kibana-6.5.4-linux-x86_64/node_modules/hapi/node_modules/catbox/lib/policy.js:274:17)
   at writeIdentity (/home/desa/kibana-6.5.4-linux-x86_64/plugins/readonlyrest_kbn/server/routes/lib/auth.js:11:28)
   at /home/desa/kibana-6.5.4-linux-x86_64/plugins/readonlyrest_kbn/server/routes/lib/auth.js:54:129
   at _wreck2.default.read (/home/desa/kibana-6.5.4-linux-x86_64/plugins/readonlyrest_kbn/server/routes/lib/identityManager.js:119:13)
   at finish (/home/desa/kibana-6.5.4-linux-x86_64/plugins/readonlyrest_kbn/node_modules/wreck/lib/index.js:369:16)
   at wrapped (/home/desa/kibana-6.5.4-linux-x86_64/plugins/readonlyrest_kbn/node_modules/hoek/lib/index.js:879:20)
   at module.exports.internals.Recorder.onReaderFinish (/home/desa/kibana-6.5.4-linux-x86_64/plugins/readonlyrest_kbn/node_modules/wreck/lib/index.js:415:16)
   at Object.onceWrapper (events.js:313:30)
   at emitNone (events.js:111:20)
   at module.exports.internals.Recorder.emit (events.js:208:7)
   at finishMaybe (_stream_writable.js:613:14)
   at afterWrite (_stream_writable.js:464:3)
   at _combinedTickCallback (internal/process/next_tick.js:145:20)
   at process._tickDomainCallback (internal/process/next_tick.js:219:9
Debug: internal, implementation, error
   Error: Uncaught error: reply interface called twice
   at Object.exports.assert (/home/desa/kibana-6.5.4-linux-x86_64/node_modules/hoek/lib/index.js:740:11)
   at Function.internals.response (/home/desa/kibana-6.5.4-linux-x86_64/node_modules/hapi/lib/reply.js:133:10)
   at reply (/home/desa/kibana-6.5.4-linux-x86_64/node_modules/hapi/lib/reply.js:70:22)
   at a.error.b (/home/desa/kibana-6.5.4-linux-x86_64/plugins/readonlyrest_kbn/server/routes/lib/auth.js:57:25)
   at a.server.app.cache.set.g (/home/desa/kibana-6.5.4-linux-x86_64/plugins/readonlyrest_kbn/server/routes/lib/auth.js:18:24)
   at _cache.set (/home/desa/kibana-6.5.4-linux-x86_64/node_modules/hapi/node_modules/catbox/lib/policy.js:280:16)
   at module.exports.internals.Client.internals.Client.set (/home/desa/kibana-6.5.4-linux-x86_64/node_modules/hapi/node_modules/catbox/lib/client.js:113:16)
   at module.exports.internals.Policy.internals.Policy.set (/home/desa/kibana-6.5.4-linux-x86_64/node_modules/hapi/node_modules/catbox/lib/policy.js:274:17)
   at writeIdentity (/home/desa/kibana-6.5.4-linux-x86_64/plugins/readonlyrest_kbn/server/routes/lib/auth.js:11:28)
   at /home/desa/kibana-6.5.4-linux-x86_64/plugins/readonlyrest_kbn/server/routes/lib/auth.js:54:129
   at _wreck2.default.read (/home/desa/kibana-6.5.4-linux-x86_64/plugins/readonlyrest_kbn/server/routes/lib/identityManager.js:119:13)
   at finish (/home/desa/kibana-6.5.4-linux-x86_64/plugins/readonlyrest_kbn/node_modules/wreck/lib/index.js:369:16)
   at wrapped (/home/desa/kibana-6.5.4-linux-x86_64/plugins/readonlyrest_kbn/node_modules/hoek/lib/index.js:879:20)
   at module.exports.internals.Recorder.onReaderFinish (/home/desa/kibana-6.5.4-linux-x86_64/plugins/readonlyrest_kbn/node_modules/wreck/lib/index.js:415:16)
   at Object.onceWrapper (events.js:313:30)
   at emitNone (events.js:111:20)
   at module.exports.internals.Recorder.emit (events.js:208:7)
   at finishMaybe (_stream_writable.js:613:14)
   at afterWrite (_stream_writable.js:464:3)
   at _combinedTickCallback (internal/process/next_tick.js:145:20)
   at process._tickDomainCallback (internal/process/next_tick.js:219:9)
error  [04:57:36.679] [warning][process] UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'headers' of undefined
   at injectXPackInfoSignature (/home/desa/kibana-6.5.4-linux-x86_64/node_modules/x-pack/plugins/xpack_main/server/lib/inject_xpack_info_signature.js:21:14)
   at <anonymous>
   at emitWarning (internal/process/promises.js:65:17)
   at emitPendingUnhandledRejections (internal/process/promises.js:109:11)
   at runMicrotasksCallback (internal/process/next_tick.js:125:9)
   at _combinedTickCallback (internal/process/next_tick.js:132:7)
   at process._tickDomainCallback (internal/process/next_tick.js:219:9)
error  [04:57:36.680] [warning][process] TypeError: Cannot read property 'headers' of undefined
   at injectXPackInfoSignature (/home/desa/kibana-6.5.4-linux-x86_64/node_modules/x-pack/plugins/xpack_main/server/lib/inject_xpack_info_signature.js:21:14)
   at <anonymous>
Message Input

readonlyrest.yml

readonlyrest:

enabled: true
prompt_for_basic_auth: false
audit_collector: true


access_control_rules:

- name: "::LOGSTASH::"
  auth_key: logstash:logstash
  actions: ["indices:data/read/*","indices:data/write/*","indices:admin/template/*","indices:admin/create"]
  indices: ["*logstash-*"]

- name: "kibana server"
  type: allow
  auth_key: kibana:kibana

Please help us to fix this issue

sscarduzio · February 14, 2019, 2:57pm

OK so to recap before trying to reproduce this: this ReadonlyREST Enterprise (or is it the PRO?) for Kibana version 1.16.34. Correct?