[SUPPORT|kbn_ent]Kibana+SSL

ROR Version: Enterprise 1.56.0_es7.15.1 :unicorn:

Kibana Version: 7.15.1

Elasticsearch Version: 7.15.1

Steps to reproduce the issue
Use https kibana from ROR

server.ssl.enabled: true
server.ssl.keystore.path: "/etc/kibana-1/cert.p12"
server.ssl.keystore.password: ""
server.ssl.supportedProtocols: ["TLSv1.2", "TLSv1.3"]

Expected result:
Kibana works stably on HTTPS

Actual Result:
Kibana sometimes stops starting. Restarting Kibana or Elasticsearch does not solve the problem.
Using a different cluster elasticsearch also does not solve the problem.
This server has multiple kibana running on different ports.
Currently one is not running, but the other two kibanas are running.
They only have a different connection port to Elasticsearch.
The problem is also reproduced on servers with a single kibana process.
If you remove the SSL parameters. Then kibana starts but SSL does not work.
If you return the parameters after a successful launch of kibana, the problem will remain.

launch log

{"type":"log","@timestamp":"2024-09-24T17:23:46+00:00","tags":["info","plugins-service"],"pid":989830,"message":"Plugin \"telemetry\" is disabled."}
{"type":"log","@timestamp":"2024-09-24T17:23:46+00:00","tags":["info","plugins-service"],"pid":989830,"message":"Plugin \"telemetryManagementSection\" has been disabled since the following direct or transitive dependencies are missing, disabled, or have incompatible types: [telemetry]"}
{"type":"log","@timestamp":"2024-09-24T17:23:46+00:00","tags":["info","plugins-service"],"pid":989830,"message":"Plugin \"canvas\" is disabled."}
{"type":"log","@timestamp":"2024-09-24T17:23:46+00:00","tags":["info","plugins-service"],"pid":989830,"message":"Plugin \"graph\" is disabled."}
{"type":"log","@timestamp":"2024-09-24T17:23:46+00:00","tags":["info","plugins-service"],"pid":989830,"message":"Plugin \"metricsEntities\" is disabled."}
{"type":"log","@timestamp":"2024-09-24T17:23:46+00:00","tags":["info","plugins-service"],"pid":989830,"message":"Plugin \"ml\" is disabled."}
{"type":"log","@timestamp":"2024-09-24T17:23:46+00:00","tags":["info","plugins-service"],"pid":989830,"message":"Plugin \"monitoring\" is disabled."}
{"type":"log","@timestamp":"2024-09-24T17:23:46+00:00","tags":["info","plugins-service"],"pid":989830,"message":"Plugin \"osquery\" has been disabled since the following direct or transitive dependencies are missing, disabled, or have incompatible types: [security]"}
{"type":"log","@timestamp":"2024-09-24T17:23:46+00:00","tags":["info","plugins-service"],"pid":989830,"message":"Plugin \"security\" is disabled."}
{"type":"log","@timestamp":"2024-09-24T17:23:46+00:00","tags":["info","plugins-service"],"pid":989830,"message":"Plugin \"uptime\" is disabled."}
{"type":"log","@timestamp":"2024-09-24T17:23:52+00:00","tags":["info","http","server","Preboot"],"pid":989830,"message":"http server running at undefined"}
{"type":"log","@timestamp":"2024-09-24T17:23:52+00:00","tags":["warning","config","deprecation"],"pid":989830,"message":"\"logging.dest\" has been deprecated and will be removed in 8.0. To set the destination moving forward, you can use the \"console\" appender in your logging configuration or define a custom one. For more details, see https://github.com/elastic/kibana/blob/master/src/core/server/logging/README.mdx"}
{"type":"log","@timestamp":"2024-09-24T17:23:52+00:00","tags":["warning","config","deprecation"],"pid":989830,"message":"Setting [elasticsearch.username] to \"kibana\" is deprecated. You should use the \"kibana_system\" user instead."}
{"type":"log","@timestamp":"2024-09-24T17:23:52+00:00","tags":["warning","config","deprecation"],"pid":989830,"message":"plugins.scanDirs is deprecated and is no longer used"}
{"type":"log","@timestamp":"2024-09-24T17:23:52+00:00","tags":["warning","config","deprecation"],"pid":989830,"message":"\"migrations.enableV2\" is deprecated and will be removed in an upcoming release without any further notice."}
{"type":"log","@timestamp":"2024-09-24T17:23:52+00:00","tags":["warning","config","deprecation"],"pid":989830,"message":"Config key [monitoring.cluster_alerts.email_notifications.email_address] will be required for email notifications to work in 8.0.\""}
{"type":"log","@timestamp":"2024-09-24T17:23:52+00:00","tags":["warning","config","deprecation"],"pid":989830,"message":"Setting [monitoring.username] to \"kibana\" is deprecated. You should use the \"kibana_system\" user instead."}
{"type":"log","@timestamp":"2024-09-24T17:23:52+00:00","tags":["warning","config","deprecation"],"pid":989830,"message":"\"xpack.reporting.index\" is deprecated. Multitenancy by changing \"kibana.index\" will not be supported starting in 8.0. See https://ela.st/kbn-remove-legacy-multitenancy for more details"}
{"type":"log","@timestamp":"2024-09-24T17:23:52+00:00","tags":["warning","config","deprecation"],"pid":989830,"message":"\"xpack.reporting.roles\" is deprecated. Granting reporting privilege through a \"reporting_user\" role will not be supported starting in 8.0. Please set \"xpack.reporting.roles.enabled\" to \"false\" and grant reporting privileges to users using Kibana application privileges **Management > Security > Roles**."}
{"type":"log","@timestamp":"2024-09-24T17:23:52+00:00","tags":["warning","config","deprecation"],"pid":989830,"message":"Disabling the security plugin (`xpack.security.enabled`) will not be supported in the next major version (8.0). To turn off security features, disable them in Elasticsearch instead."}
{"type":"log","@timestamp":"2024-09-24T17:23:52+00:00","tags":["warning","config","deprecation"],"pid":989830,"message":"Session idle timeout (\"xpack.security.session.idleTimeout\") will be set to 1 hour by default in the next major version (8.0)."}
{"type":"log","@timestamp":"2024-09-24T17:23:52+00:00","tags":["warning","config","deprecation"],"pid":989830,"message":"Session lifespan (\"xpack.security.session.lifespan\") will be set to 30 days by default in the next major version (8.0)."}
{"type":"log","@timestamp":"2024-09-24T17:24:02+00:00","tags":["warning","environment"],"pid":989830,"message":"Detected an unhandled Promise rejection.\nError: LicenseService is not initialised yet: empty ActivationKey"}
{"type":"log","@timestamp":"2024-09-24T17:24:02+00:00","tags":["info","plugins-system","standard"],"pid":989830,"message":"Setting up [106] plugins: [translations,licensing,globalSearch,globalSearchProviders,banners,licenseApiGuard,encryptedSavedObjects,code,usageCollection,xpackLegacy,taskManager,telemetryCollectionManager,telemetryCollectionXpack,kibanaUsageCollection,securityOss,share,screenshotMode,newsfeed,mapsEms,mapsLegacy,legacyExport,kibanaLegacy,embeddable,uiActionsEnhanced,fieldFormats,expressions,charts,esUiShared,bfetch,data,fileUpload,savedObjects,visualizations,visTypeXy,visTypeVislib,visTypeTimelion,features,visTypeTagcloud,visTypeTable,visTypePie,visTypeMetric,visTypeMarkdown,tileMap,regionMap,presentationUtil,expressionShape,expressionRevealImage,expressionRepeatImage,expressionMetric,expressionImage,timelion,readonlyrestkbn,indexPatternFieldEditor,home,searchprofiler,painlessLab,grokdebugger,cloud,fleet,visTypeVega,management,watcher,transform,snapshotRestore,savedObjectsTagging,logstash,licenseManagement,ingestPipelines,indexManagement,remoteClusters,crossClusterReplication,indexLifecycleManagement,dataEnhanced,indexPatternManagement,advancedSettings,discover,discoverEnhanced,dashboard,maps,lens,dataVisualizer,dashboardMode,dashboardEnhanced,visualize,visTypeTimeseries,rollup,savedObjectsManagement,spaces,reporting,lists,eventLog,actions,alerting,triggersActionsUi,stackAlerts,ruleRegistry,cases,timelines,securitySo
lution,observability,infra,upgradeAssistant,enterpriseSearch,console,apmOss,apm]"}
{"type":"log","@timestamp":"2024-09-24T17:24:02+00:00","tags":["warning","plugins","encryptedSavedObjects"],"pid":989830,"message":"Saved objects encryption key is not set. This will severely limit Kibana functionality. Please set xpack.encryptedSavedObjects.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command."}
{"type":"log","@timestamp":"2024-09-24T17:24:02+00:00","tags":["info","plugins","taskManager"],"pid":989830,"message":"TaskManager is identified by the Kibana UUID: 746af7e8-4448-4195-a38d-262b332b9b19"}
{"type":"log","@timestamp":"2024-09-24T17:24:02+00:00","tags":["info","plugins","readonlyrestkbn"],"pid":989830,"message":"Setting up ReadonlyREST plugin - build info: {\"versionString\":\"1.56.0_es7.15.1\",\"kibanaVersion\":\"7.15.1\",\"rorVersion\":\"1.56.0\",\"isProduction\":true,\"isBuildExpired\":false}"}
{"type":"log","@timestamp":"2024-09-24T17:24:02+00:00","tags":["warning","plugins","reporting","config"],"pid":989830,"message":"Chromium sandbox provides an additional layer of protection, but is not supported for Linux Red Hat Linux 9.2 OS. Automatically setting 'xpack.reporting.capture.browser.chromium.disableSandbox: true'."}
{"type":"log","@timestamp":"2024-09-24T17:24:02+00:00","tags":["warning","plugins","actions"],"pid":989830,"message":"APIs are disabled because the Encrypted Saved Objects plugin is missing encryption key. Please set xpack.encryptedSavedObjects.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command."}
{"type":"log","@timestamp":"2024-09-24T17:24:02+00:00","tags":["warning","plugins","alerting"],"pid":989830,"message":"APIs are disabled because the Encrypted Saved Objects plugin is missing encryption key. Please set xpack.encryptedSavedObjects.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command."}
{"type":"log","@timestamp":"2024-09-24T17:24:02+00:00","tags":["info","plugins","ruleRegistry"],"pid":989830,"message":"Write is disabled; not installing common resources shared between all indices"}
{"type":"log","@timestamp":"2024-09-24T17:24:03+00:00","tags":["info","plugins","ruleRegistry"],"pid":989830,"message":"Write is disabled; not installing resources for index .alerts-observability.logs.alerts"}
{"type":"log","@timestamp":"2024-09-24T17:24:03+00:00","tags":["info","plugins","ruleRegistry"],"pid":989830,"message":"Write is disabled; not installing resources for index .alerts-observability.metrics.alerts"}
{"type":"log","@timestamp":"2024-09-24T17:24:03+00:00","tags":["info","plugins","ruleRegistry"],"pid":989830,"message":"Write is disabled; not installing resources for index .alerts-observability.apm.alerts"}
{"type":"log","@timestamp":"2024-09-24T17:24:03+00:00","tags":["info","savedobjects-service"],"pid":989830,"message":"Waiting until all Elasticsearch nodes are compatible with Kibana before starting saved objects migrations..."}
{"type":"log","@timestamp":"2024-09-24T17:24:03+00:00","tags":["error","savedobjects-service"],"pid":989830,"message":"Unable to retrieve version information from Elasticsearch nodes. connect ECONNREFUSED 127.0.0.1:38087"}

Minimum config to reproduce the problem:

server.port: 5601
server.host: "0.0.0.0"

elasticsearch.hosts: ["https://localhost:9201"]
elasticsearch.username: login
elasticsearch.password: pass
migrations.enableV2: false
monitoring.enabled: false
console.enabled: true

server.ssl.enabled: true
server.ssl.keystore.path: "/etc/kibana-1/cert.p12"
server.ssl.keystore.password: ""
server.ssl.supportedProtocols: ["TLSv1.2", "TLSv1.3"]
elasticsearch.ssl.alwaysPresentCertificate: true
elasticsearch.ssl.verificationMode: none
readonlyrest_kbn.cookiePass: "cookiepass"
readonlyrest_kbn.cookieName: "cookiename"

To be precise, the kibana process is hanging, but it doesn’t open in the browser and doesn’t create indexes in Elasticsearch.

{“customer_id”: “6c4a385b-2ae8-4f02-a9cd-ef24addfb5b3”, “subscription_id”: “32d4073f-dc2f-4056-a868-842727c637cd”}

Hi @driveirk

Thanks for the reproduction steps, I will check what’s going on based on your config and let you know. Could you provide also ROR Kibana plugin logs?

There are no logs in the access logs. That is, no connections to Elasticsearch occur.

By default Kibana logs are destined to the console, if you run Kibana and Kibana plugin, there must be any logs related to it at least about starting. Could you set in the kibana.yml

logging.dest: /var/log/kibana/kibana.log

Thanks to it, all Kibana and Kibana plugin logs, will be saved to the file

Hi @driveirk

Did you have a chance to set the logging destination to file and receive logs?

The problem has not been reproduced yet. As soon as it appears again I will do so.
The problem appears approximately once a month.