Hi
Every time I duplicate the current page or click on a short link, I get to the authorization page. Although I am authorized in the next tab.
Is it possible to solve this issue somehow?
version ROR: Enterprise-1.39.0_es7.15.1 
{“customer_id”: “6c4a385b-2ae8-4f02-a9cd-ef24addfb5b3”, “subscription_id”: “4c690a6a-9a7b-4fea-973b-690baee6d4d5”}
You mean in Canvas Kibana app? Do you have ES logs with some “FORBIDDEN” entry?
Yes, you understood correctly, the error appears when working through Kibana.
There are no messages in the access log when a tab is duplicated. It just opens authorization.
I had a try to duplicate the canvas page, it worked in my test using the duplicate button (tested both as admin, and rw user):
Can you provide more detailed instruction to reproduce the issue? It would be also useful if you can attach also your ACL to support the test case.
Sorry, I guess I misunderstood you.
I meant opening the page in the next browser tab.
Duplicate page in browser.
I logged in to kibana. But if I try to open this link in the next tab, then I will have to log in to kibana again.
Oh ok this is different. I had a look and cannot reproduce this in the latest Kibana with latest ROR Kibana plugin installed.
What app are you using when this happens? Can you reproduce it consistently?
If you press F12 to get the Chrome devtools, and go to “console”, you will see some logs from the browser. Can you see there any errors?
I only see one error in the console browser:
Failed to load resource: the server responded with a status of 401 (Unauthorized)
I also noticed that when I open the second tab, I log out in the first tab too.
my ROR conf
readonlyrest:
audit_collector: true
audit_serializer: tech.beshu.ror.requestcontext.QueryAuditLogSerializer
audit_index_template: "'.pro-readonlyrest-audit-'-yyyy-MM-dd"
prompt_for_basic_auth: false
response_if_req_forbidden: Forbidden by ReadonlyREST ES plugin
access_control_rules:
- name: "Allow unrestricted access "
type: allow
ldap_authentication:
name: "ldap"
ldap_authorization:
name: "ldap"
groups: ["group"]
kibana_access: unrestricted
my kibana konf
server.port: 5601
server.host: "127.0.0.1"
elasticsearch.hosts: ["http://127.0.0.1:9200"]
server.publicBaseUrl: http://example.org
elasticsearch.username: asd
elasticsearch.password: asd
kibana.index: ".kibana-1"
monitoring.enabled: false
console.enabled: true
xpack.security.enabled: false
xpack.infra.enabled: true
xpack.maps.enabled: false
xpack.canvas.enabled: false
xpack.apm.enabled: true
xpack.uptime.enabled: false
xpack.spaces.enabled: true
xpack.graph.enabled: false
xpack.grokdebugger.enabled: true
xpack.apm.ui.enabled: true
xpack.reporting.encryptionKey: "asdasd"
readonlyrest_kbn.kibana_custom_css_inject: "[data-test-subj*=spacesNavSelector] { display: none !important } [data-test-subj*=homeLink] { display: none !important } [data-test-subj*=discover-addRuntimeField-popove] { display: none !important } [data-test-subj*=discoverOptionsButton] { display: none !important } [data-test-subj*=lnsIndexPatternActions-popover] { display: none !important }"
readonlyrest_kbn.kibana_custom_js_inject: "$('[data-test-subj*=spacesNavSelector]').remove() $('[data-test-subj*=homeLink').remove() $('[data-test-subj=discover-addRuntimeField-popover').remove() $('[data-test-subj*=discoverOptionsButton]').remove() $('[data-test-subj=lnsIndexPatternActions-popover').remove()"
elasticsearch.requestHeadersWhitelist: ["authorization", "x-forwarded-for", "x-passed-nginx"]
readonlyrest_kbn.cookiePass: "asdasdasdasd"
readonlyrest_kbn.cookieName: "rorasdasdasdasdasdsad"
readonlyrest_kbn.store_sessions_in_index: true
readonlyrest_kbn.sessions_index_name: ".readonlyrest_kibana_tribe_sessions"
readonlyrest_kbn.sessions_refresh_after: 1000
readonlyrest_kbn.whitelistedPaths: [".*/api/status$"]
readonlyrest_kbn.sessions_probe_interval_seconds: 10
readonlyrest_kbn.login_html_head_inject: "<style> body { background: #eee !important;}.ror-logo { visibility: hidden;}.login > h1 { margin: 0 auto; background: url('https://my.company.org/img/logo.png') no-repeat; background-position: center center; background-size: auto;}.ror-logolink { display: none !important;}#form-username, #form-password { background: none white !important; outline: 0; padding: 10px; color: #777; text-shadow: none !important; border: 1px solid #aaa !important; border-radius: 2px !important;}#form-username:focus, #form-password:focus { background: none white !important; color: #777 !important;}.loginButton { color:#fff !important; text-decoration:none; font-size:16px; width:100%; padding:10px 15px; background: #1890ff !important; border:solid 1px #1890ff !important; border-radius: 2px !important; -webkit-border-radius: 2px; -moz-border-radius: 2px}.loginButton:hover{ color:#666; border-color:#40a9ff !important; background: #40a9ff !important; border-radius:2px; -webkit-border-radius:2px; -moz-border-radius:2px}.ror-footer { background: #eee!important; color: #444 !important;}.ror-footer > div > p { visibility: hidden !important; position: relative;}.ror-footer > div > p:after { content: '?2022 my company'; visibility: visible; display: block; position:absolute; top: 0; left: 0; idth: 100%; margin: 0 auto;} .message {color: #f7981d;}</style>"
apm_oss.indexPattern: "*:*-apm-*"
apm_oss.errorIndices: "*:*-apm-*"
apm_oss.onboardingIndices: "*:*-apm-*"
apm_oss.spanIndices: "*:*-apm-*"
apm_oss.transactionIndices: "*:*-apm-*"
apm_oss.metricsIndices: "*:*-apm-*"
apm_oss.sourcemapIndices: "*:*-apm-*"
migrations.enableV2: false
logging.verbose: false
readonlyrest_kbn.sessions_probe_interval_seconds: 10
this is really low, try putting it to 30 or more. No interesting logs in kibana server either?
Do you have multiple Kibana instances in high availability behind a load balancer?
Here is the log that constantly flashes in kibana
[10:48:55:560] [info][plugins][ReadonlyREST][preElasticsearchProxy] Could not forward all whitelisted headers, left out: x-forwarded-for,x-passed-nginx
{"type":"response","@timestamp":"2022-05-20T10:48:55+00:00","tags":["access:apm"],"pid":22490,"method":"post","statusCode":200,"req":{"url":"/api/apm/settings/agent-configuration/search","method":"post","headers":{"x-forwarded-for":"10.10.10.250","x-passed-nginx":"true","host":"kibana","user-agent":"Go-http-client/1.1","accept":"application/json","content-type":"application/json","kbn-xsrf":"1","accept-encoding":"gzip","connection":"close","x-ror-pkp-kibana-token":"uhwzarro7p7mr3iqblymyqyys5hfun","content-length":"90","accept-charset":"utf-8"},"remoteAddress":"127.0.0.1","userAgent":"Go-http-client/1.1"},"res":{"statusCode":200,"responseTime":16,"contentLength":252},"message":"POST /api/apm/settings/agent-configuration/search 200 16ms - 252.0B"}
[10:48:55:701] [info][plugins][ReadonlyREST][preElasticsearchProxy] Could not forward all whitelisted headers, left out: x-forwarded-for,x-passed-nginx
[10:48:56:035] [info][plugins][ReadonlyREST][authController] Refreshing session against ES
[10:48:56:083] [info][plugins][ReadonlyREST][tenantIndexBasedOnTemplateApplier] Template index not defined. Returning
The problem is reproduced even when you go to kibana directly.
I think the problem is errors.
But I still don’t know how to fix them.
There are 3 kibana in the cluster.
Each kibana is behind nginx.
I don’t see errors in these logs. But, because you have 3 instances of Kibana, I have more questions:
readonlyrest_kbn.sessions_probe_interval_seconds: 120 help to delay the issue?are you 100% sure the kibana.yml files are kept in sync in all 3?
Yes, I sure.
Can you check if setting readonlyrest_kbn.sessions_probe_interval_seconds: 120 help to delay the issue?
I think you don’t understand me. My problem is reproduced right after authorization without waiting 10 seconds. It has nothing to do with time.
As I thought. Increasing the time didn’t help.
Yes you are right, I didn’t understand. I re-read the whole thread and I understood this now, and I understand this is the issue:
Expected result in tab2: the same screen of tab1
Actual result in tab2: ROR login form
Correct?
Yes, that’s right. Were you able to reproduce the issue?
OK great. No, I cannot reproduce it in a single node environment. I will need to try with two nodes.
What kind of load balancer do you have in front of the 3 Kibana instances? Round robin? Sticky sessions?
For the purity of the tests, I do not use a balancer. And I go directly to kibana and the problem is reproduced.
I do this to be sure that neither haproxy nor nginx interfere. And that I go to the same server.
Generally:
haproxy(balancer, Sticky sessions) => nginx =>Kibana
OK so your test case is reproducible by you in isolation skipping all LB and reverse proxies, pointing the browser to a single Kibana instance.
The last thing I don’t understand is that in your config you show a single ES node (localhost) and you said you share this kibana.yml in all 3 Kibana nodes. How is this possible?
OK so your test case is reproducible by you in isolation skipping all LB and reverse proxies, pointing the browser to a single Kibana instance.
Yes, it is reproduced in a test environment, in a lab, in 2 stage clusters and in 4 production clusters.
The last thing I don’t understand is that in your config you show a single ES node (localhost) and you said you share this kibana.yml in all 3 Kibana nodes. How is this possible?
Kibana and ElasticSearch are on the same server.
Each Kibana has its own ElasticSearch.
All of these ElasticSearchs are in the same cluster.
kibana-1 => ElasticSearch-1 => \
kibana-2 => ElasticSearch-2 => cluster
kibana-3 => ElasticSearch-3 => /
OK So effectively, what I should test to reproduce the issue would be:
Hello, @driveirk I tried to reproduce this issue locally, and it looks like it can be connected with an overriding secure cookie. Are you able to set readonlyrest_kbn.cookieName unique for every kibana instance?