New ROR Kibana plugin errors 1.29.0 - 7.10.2

Users Feedback
1

hi,

I am trying to test the new ROR Kibana pluging 1.29.0 on ES 7.10.2, Windows 2012 R2. I get the below error, when i enter id/pwd on login page.

TypeError: Cannot read property '0' of undefined
    at HtmlInjector.injectCssAndJsScriptsIntoHtml (D:\Apps\kibana-7.10.2-windows-x86_64\plugins\readonlyrestkbn\proxy\preKibanaProxy\injection\htmlInjector.js:45:118)
    at ProxyBuilder.interceptResponse (D:\Apps\kibana-7.10.2-windows-x86_64\plugins\readonlyrestkbn\proxy\preKibanaProxy\proxyBuilder.js:90:35)
    at D:\Apps\kibana-7.10.2-windows-x86_64\plugins\readonlyrestkbn\node_modules\express-http-proxy\app\steps\decorateUserRes.js:69:14
    at process._tickCallback (internal/process/next_tick.js:68:7)

I did enable debugging mode logging. This is what I see on stdout.

[02:17:27:840] [debug][plugins][ReadonlyREST][LoggerFactory] Setting logLevel to: DEBUG
[02:17:27:880] [debug][plugins][ReadonlyREST][kibanaConfigInterceptor] Overriding 'server.ssl.enabled' from true to false
[02:17:27:880] [debug][plugins][ReadonlyREST][kibanaConfigInterceptor] Overriding 'server.port' from 11201 to 11211
[02:17:27:880] [debug][plugins][ReadonlyREST][kibanaConfigInterceptor] Overriding 'server.host' from myhost1.mydomain.com to 127.0.0.1
[02:17:27:880] [debug][plugins][ReadonlyREST][kibanaConfigInterceptor] Overriding 'elasticsearch.hosts' from https://myhost1.mydomain.com:11200,https://myhost2.mydomain.com:11200,https://myhost3.mydomain.com:11200 to [http://localhost:11221]
[02:17:27:881] [debug][plugins][ReadonlyREST][kibanaConfigInterceptor] Overriding 'elasticsearch.requestHeadersWhitelist' from ["x-proxy-user","authorization","x-forwarded-for","x-forwarded-server","es-security-runas-user"] to ["x-proxy-user","authorization","x-forwarded-for","x-forwarded-server","es-security-runas-user","cookie"]
[02:17:45:008] [error][plugins][ReadonlyREST][lazyUtils] Caught an error in executeWithInterval { FetchError: request to https://myhost1.mydomain.com:11200/.readonlyrest_kbn_sessions/_search failed, reason: read ECONNRESET
    at ClientRequest.<anonymous> (D:\Apps\kibana-7.10.2-windows-x86_64\plugins\readonlyrestkbn\node_modules\node-fetch\lib\index.js:1455:11)
    at ClientRequest.emit (events.js:198:13)
    at TLSSocket.socketErrorListener (_http_client.js:401:9)
    at TLSSocket.emit (events.js:198:13)
    at emitErrorNT (internal/streams/destroy.js:91:8)
    at emitErrorAndCloseNT (internal/streams/destroy.js:59:3)
    at process._tickCallback (internal/process/next_tick.js:63:19)
  message:
   'request to https://myhost1.mydomain.com:11200/.readonlyrest_kbn_sessions/_search failed, reason: read ECONNRESET',
  type: 'system',
  errno: 'ECONNRESET',
  code: 'ECONNRESET' }
[02:20:54:006] [error][plugins][ReadonlyREST][cookieManager] Error: Bad hmac value
[02:20:54:102] [error][plugins][ReadonlyREST][cookieManager] Error: Bad hmac value
[02:20:54:104] [error][plugins][ReadonlyREST][cookieManager] Error: Bad hmac value

Finally, below is my kibana log

{"type":"log","@timestamp":"2021-04-22T06:17:42Z","tags":["info","plugins-service"],"pid":35660,"message":"Plugin \"ml\" is disabled."}
{"type":"log","@timestamp":"2021-04-22T06:17:42Z","tags":["info","plugins-service"],"pid":35660,"message":"Plugin \"watcher\" is disabled."}
{"type":"log","@timestamp":"2021-04-22T06:17:42Z","tags":["info","plugins-service"],"pid":35660,"message":"Plugin \"visTypeXy\" is disabled."}
{"type":"log","@timestamp":"2021-04-22T06:17:42Z","tags":["info","plugins-service"],"pid":35660,"message":"Plugin \"auditTrail\" is disabled."}
{"type":"log","@timestamp":"2021-04-22T06:17:42Z","tags":["info","plugins-service"],"pid":35660,"message":"Plugin \"graph\" is disabled."}
{"type":"log","@timestamp":"2021-04-22T06:17:42Z","tags":["info","plugins-service"],"pid":35660,"message":"Plugin \"security\" is disabled."}
{"type":"log","@timestamp":"2021-04-22T06:17:42Z","tags":["warning","config","deprecation"],"pid":35660,"message":"\"xpack.monitoring\" is deprecated and has been replaced by \"monitoring\""}
{"type":"log","@timestamp":"2021-04-22T06:17:42Z","tags":["warning","config","deprecation"],"pid":35660,"message":"Config key [monitoring.cluster_alerts.email_notifications.email_address] will be required for email notifications to work in 8.0.\""}
{"type":"log","@timestamp":"2021-04-22T06:17:43Z","tags":["info","plugins-system"],"pid":35660,"message":"Setting up [93] plugins: [taskManager,licensing,globalSearch,globalSearchProviders,encryptedSavedObjects,code,usageCollection,xpackLegacy,telemetryCollectionManager,telemetry,telemetryCollectionXpack,kibanaUsageCollection,securityOss,newsfeed,mapsLegacy,kibanaLegacy,translations,share,legacyExport,embeddable,uiActionsEnhanced,expressions,data,home,observability,cloud,console,consoleExtensions,apmOss,searchprofiler,painlessLab,grokdebugger,management,indexPatternManagement,advancedSettings,fileUpload,savedObjects,readonlyrestkbn,dashboard,dashboardMode,visualizations,visTypeVega,visTypeTimelion,timelion,features,upgradeAssistant,snapshotRestore,maps,licenseManagement,ingestManager,indexManagement,remoteClusters,crossClusterReplication,indexLifecycleManagement,enterpriseSearch,beatsManagement,transform,ingestPipelines,dataEnhanced,visTypeTable,visTypeMarkdown,tileMap,regionMap,inputControlVis,visualize,esUiShared,charts,lens,visTypeVislib,visTypeTimeseries,rollup,visTypeTagcloud,visTypeMetric,discover,discoverEnhanced,savedObjectsManagement,spaces,reporting,lists,eventLog,actions,case,alerts,stackAlerts,triggersActionsUi,securitySolution,infra,monitoring,logstash,apm,uptime,bfetch,canvas]"}
{"type":"log","@timestamp":"2021-04-22T06:17:43Z","tags":["info","plugins","readonlyrestkbn"],"pid":35660,"message":"Setting up ReadonlyREST plugin - build info: {\"versionString\":\"free-1.29.0_es7.10.2\",\"kibanaVersion\":\"7.10.2\",\"rorEdition\":\"free\",\"rorVersion\":\"1.29.0\",\"isProduction\":true,\"isEnterprise\":false,\"isPro\":false,\"isFree\":true,\"isBuildExpired\":false}"}
{"type":"log","@timestamp":"2021-04-22T06:17:44Z","tags":["warning","plugins","reporting","config"],"pid":35660,"message":"Generating a random key for xpack.reporting.encryptionKey. To prevent sessions from being invalidated on restart, please set xpack.reporting.encryptionKey in kibana.yml"}
{"type":"log","@timestamp":"2021-04-22T06:17:44Z","tags":["info","plugins","reporting","config"],"pid":35660,"message":"Chromium sandbox provides an additional layer of protection, and is supported for Win32 OS. Automatically enabling Chromium sandbox."}
{"type":"log","@timestamp":"2021-04-22T06:17:44Z","tags":["info","plugins","monitoring","monitoring"],"pid":35660,"message":"config sourced from: production cluster"}
{"type":"log","@timestamp":"2021-04-22T06:17:44Z","tags":["info","savedobjects-service"],"pid":35660,"message":"Waiting until all Elasticsearch nodes are compatible with Kibana before starting saved objects migrations..."}
{"type":"log","@timestamp":"2021-04-22T06:17:45Z","tags":["warning","environment"],"pid":35660,"message":"Detected an unhandled Promise rejection.\nFetchError: request to https://myhost1.mydomain.com:11200/.readonlyrest_kbn_sessions/_search failed, reason: read ECONNRESET"}
{"type":"log","@timestamp":"2021-04-22T06:17:45Z","tags":["info","savedobjects-service"],"pid":35660,"message":"Starting saved objects migrations"}
{"type":"log","@timestamp":"2021-04-22T06:17:45Z","tags":["info","savedobjects-service"],"pid":35660,"message":"Creating index .kibana_task_manager_35."}
{"type":"log","@timestamp":"2021-04-22T06:17:45Z","tags":["info","savedobjects-service"],"pid":35660,"message":"Creating index .kibana_4."}
{"type":"log","@timestamp":"2021-04-22T06:17:45Z","tags":["error","elasticsearch","data"],"pid":35660,"message":"[resource_already_exists_exception]: index [.kibana_4/Tx-qP-w2RkKj6C1uxpbf8g] already exists"}
{"type":"log","@timestamp":"2021-04-22T06:17:45Z","tags":["warning","savedobjects-service"],"pid":35660,"message":"Unable to connect to Elasticsearch. Error: resource_already_exists_exception"}
{"type":"log","@timestamp":"2021-04-22T06:17:45Z","tags":["warning","savedobjects-service"],"pid":35660,"message":"Another Kibana instance appears to be migrating the index. Waiting for that migration to complete. If no other Kibana instance is attempting migrations, you can get past this message by deleting index .kibana_4 and restarting Kibana."}
{"type":"log","@timestamp":"2021-04-22T06:17:46Z","tags":["info","savedobjects-service"],"pid":35660,"message":"Migrating .kibana_task_manager_34 saved objects to .kibana_task_manager_35"}
{"type":"log","@timestamp":"2021-04-22T06:17:46Z","tags":["error","savedobjects-service"],"pid":35660,"rawDocument":{"_index":".kibana_task_manager_34","_type":"_doc","_id":"oss_telemetry-vis_telemetry","_score":1,"_source":{"task":{"taskType":"vis_telemetry","runAt":"2021-04-21T04:00:00.000Z","retryAt":"2021-04-20T04:01:01.851Z","state":"{\"runs\":17,\"stats\":{\"pie\":{\"total\":14,\"spaces_min\":14,\"spaces_max\":14,\"spaces_avg\":14},\"area\":{\"total\":44,\"spaces_min\":44,\"spaces_max\":44,\"spaces_avg\":44},\"table\":{\"total\":14,\"spaces_min\":14,\"spaces_max\":14,\"spaces_avg\":14},\"line\":{\"total\":9,\"spaces_min\":9,\"spaces_max\":9,\"spaces_avg\":9},\"markdown\":{\"total\":1,\"spaces_min\":1,\"spaces_max\":1,\"spaces_avg\":1}}}","params":"{}","ownerId":"kibana:540d829a-20db-48bb-a971-237c2a404f8c","scheduledAt":"2021-04-04T08:25:14.949Z","attempts":0,"status":"idle"},"type":"task","kibana":{"apiVersion":1,"uuid":"88fca9a4-e45d-4f30-99a3-c7e4ddbd015e","version":7020099}}},"message":"Error: Unable to migrate the corrupt Saved Object document oss_telemetry-vis_telemetry. To prevent Kibana from performing a migration on every restart, please delete or fix this document by ensuring that the namespace and type in the document's id matches the values in the namespace and type fields."}
{"type":"log","@timestamp":"2021-04-22T06:17:46Z","tags":["error","savedobjects-service"],"pid":35660,"rawDocument":{"_index":".kibana_task_manager_34","_type":"_doc","_id":"Maps-maps_telemetry","_score":1,"_source":{"type":"task","task":{"taskType":"maps_telemetry","state":"{\"stats\":{},\"runs\":0}","params":"{}","attempts":0,"scheduledAt":"2021-04-04T08:25:12.866Z","runAt":"2021-04-20T07:52:24.362Z","status":"running"},"kibana":{"uuid":"88fca9a4-e45d-4f30-99a3-c7e4ddbd015e","version":7020099,"apiVersion":1}}},"message":"Error: Unable to migrate the corrupt Saved Object document Maps-maps_telemetry. To prevent Kibana from performing a migration on every restart, please delete or fix this document by ensuring that the namespace and type in the document's id matches the values in the namespace and type fields."}
{"type":"log","@timestamp":"2021-04-22T06:17:47Z","tags":["info","savedobjects-service"],"pid":35660,"message":"Pointing alias .kibana_task_manager to .kibana_task_manager_35."}
{"type":"log","@timestamp":"2021-04-22T06:17:47Z","tags":["info","savedobjects-service"],"pid":35660,"message":"Finished in 2426ms."}

Another issue that i noticed was if the install folder path has space, then ROR is not able to read the files. So temporarily, i had to move it to another path. Previous version of plugin did not have this issue. Originally, i thought that even above error was due to space in folder path. But even after i moved the installation to a folder without space in name, I continued getting above error.

Please let me know if further details are needed.

Thanks
Ashok

2

@askids we have a fix for this, are you using the free plugin?

3

yes. Its the free version that we are using.

1 Like
4

This fix will be in the next release.

5

Thanks. Can you please also check the issue with space in install folder name. We had “Program Files” in that install path and it was throwing additional error (cannot read D:\Apps\Programs) instead of using full path. I haven’t faced this issue on older versions.

6

hi @sscarduzio,

When will the next version of ROR be available for download? We had ES upgrade to 7.10.2 planned for production next month, but because of login issue in ROR Kibana plugin, the ES upgrade is also currently on hold. So please let me know the availability date so that we can plan accordingly.

Thanks
Ashok

7

Next weekend we release 1.30.0

8

Is there a pre-release that is available for us to try out ?

9

the release is out today, check in a couple of hours.

10

Thanks. I initially thought that it was next weekend :slight_smile:

11

Did you manage to download now?

12

Yes. But unfortunately, the issue is not yet resolved. Now I am stuck with “Kibana server is not ready yet”. I am running this on Windows 2012 R2 server. Has this got anything to do with the patch that is being applied? Reason I ask that is whenever I run patch and then run unpatch, it always shows the last error. So I am wondering if patch is being applied correctly or not.

Also, this the default Kibana Distribution that comes with x-pack. This is what we have always been running.

[ROR COMPAT] Received command: unpatch
[ROR COMPAT] found patch file D:\Apps\kibana-7.10.2-windows-x86_64\plugins\readonlyrestkbn\kibana\patchers/patches/http_server.js.patch
[ROR COMPAT] backup file found!
Restored D:\Apps\kibana-7.10.2-windows-x86_64\plugins\readonlyrestkbn\kibana\patchers/../../../../src/core/server/http/http_server.js
[ROR COMPAT] backup file not found, assuming patch was never applied...
[ROR COMPAT] found patch file D:\Apps\kibana-7.10.2-windows-x86_64\plugins\readonlyrestkbn\kibana\patchers/patches/kbn_server.js.patch
[ROR COMPAT] backup file found!
Restored D:\Apps\kibana-7.10.2-windows-x86_64\plugins\readonlyrestkbn\kibana\patchers/../../../../src/legacy/server/kbn_server.js
[ROR COMPAT] backup file not found, assuming patch was never applied...
[ROR COMPAT] found patch file D:\Apps\kibana-7.10.2-windows-x86_64\plugins\readonlyrestkbn\kibana\patchers/patches/read_config.js.patch
[ROR COMPAT] backup file found!
Restored D:\Apps\kibana-7.10.2-windows-x86_64\plugins\readonlyrestkbn\kibana\patchers/../../../../node_modules/@kbn/config/target/raw/read_config.js
[ROR COMPAT] backup file not found, assuming patch was never applied...
13

Quick Update. Even though, I still see the above error, when I run unpatch, i no longer get the originally reported error and able to login via the login page. Also, the space in folder name issue is also no longer an issue.

14

if you refer to:
[ROR COMPAT] backup file not found, assuming patch was never applied...

That is a poor log line which testifies that after the backup file was found and restored, the backup file was correctly deleted. So ignore it.

As far as I understand, you confirm this error is fixed:

TypeError: Cannot read property '0' of undefined
    at HtmlInjector.injectCssAndJsScriptsIntoHtml (D:\Apps\kibana-7.10.2-windows-x86_64\plugins\readonlyrestkbn\proxy\preKibanaProxy\injection\htmlInjector.js:45:118)

Is the space in path error still present?

15

Issue with space in folder name is also fixed. I moved the kibana folder back to original location which is under “Program Files”. Kibana loads up properly now with 1.30.0.

1 Like