The “Edit security settings” is passing a different URL extension I think.
NGINX is unable to process , we need to add the nginx.conf “rules” to process the URL.
May I know what has changed? The error I get is “Unable to connect with a sad face icon”.
Hi @praveenmak, I guess you are upgrading from Kibana < 7.9.0?
In this case, yes: the URL of the settings is different because we moved the ROR settings web app from the trash UX of the Kibana plugin framework.
In the past, when you clicked on security settings, you were routed to an entirely different Kibana app. The the whole page would reload, and the navigation context would be lost.
With ROR for Kibana >= 7.9.0 we implemented our own ROR settings web app and we serve it inside an overlaid modal window that you can dismiss anytime and go back to where you left in Kibana navigation.
You can see the “X” for dismissing the modal window in the top right.
The new modal window is powered by an iframe loaded from the path /pkp/web, or if you are using a server.basePath in kibana.yml, it will load from /<basePath>/pkp/web.
So you might want to add rules to your NGINX accordingly.
I hope this helps.
Thanks @sscarduzio for the response.
I have not got past that error, I still get the “Unable to connect” with a Sad face when I try to Edit Security.
Once I get to that I have to “logout”, because nothing else works.
Here is the NGINX entry, or is this something that Kibana folks need to answer?
location ~ (/app|/translations|/pkp|/pkp/web|/node_modules|/built_assets/|/bundles|/es_admin|/plugins|/api|/ui|/elasticsearch|/spaces/enter|/s|/k)
Hi @praveenmak, there was an issue we resolved in the latest release 1.38.0
Fix (KBN) Editor rendering issue with kibana basePath enabledAnd this other one
…
So please have a try with the latest kibana plugin!
@sscarduzio - After upgrading to 7.17.0, it went a step further. But still no luck.
I cannot “Save” the edits now.
I get an error in browser. HTTP 404.
Below is the Kibana log message.
{
"type": "response",
"@timestamp": "2022-03-02T04:39:47-05:00",
"tags": [],
"pid": 27290,
"method": "post",
"statusCode": 404,
"req": {
"url": "/pkp/api/settings",
"method": "post",
"headers": {
"host": "kibanaprod.my",
"x-real-ip": "10.145.132.174",
"x-forwarded-for": "10.145.132.174",
"x-forwarded-proto": "https",
"x-forwarded-host": "kibanaprod.my",
"sec-ch-ua": "\" Not A;Brand\";v=\"99\", \"Chromium\";v=\"98\", \"Google Chrome\";v=\"98\"",
"accept": "application/json, text/plain, */*",
"dnt": "1",
"content-type": "application/json;charset=UTF-8",
"sec-ch-ua-mobile": "?0",
"user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.102 Safari/537.36",
"sec-ch-ua-platform": "\"Windows\"",
"origin": "https://kibanaprod.my",
"sec-fetch-site": "same-origin",
"sec-fetch-mode": "cors",
"sec-fetch-dest": "empty",
"referer": "https://kibanaprod.my/pkp/web/?basePath=/",
"accept-encoding": "gzip, deflate, br",
"accept-language": "en-US,en;q=0.9,kn;q=0.8,ja;q=0.7",
"connection": "close",
"x-ror-pkp-kibana-token": "ks2rsvqa2qlm5qdi8ua0ifxhugztwv",
"content-length": "33827",
"accept-charset": "utf-8"
},
"remoteAddress": "127.0.0.1",
"userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.102 Safari/537.36",
"referer": "https://kibanaprod.my/pkp/web/?basePath=/"
},
"res": {
"statusCode": 404,
"responseTime": 21,
"contentLength": 60
},
"message": "POST /pkp/api/settings 404 21ms - 60.0B"
}
Do you have any evidence that this POST request ever hits Kibana? Or is it stuck in Nginx routing?
The one I posted is from Kibana. It’s after NGINX has done its job,
Here is the error on the Kibana UI, after doing a “Security Save Edits”
{"message":"Request failed with status code 404","name":"Error","stack":"Error: Request failed with status code 404\n at e.exports
(https://kibanaprod.my/pkp/web/static/js/426.d66c65e8.chunk.js:2:143638)\n at e.exports
(https://kibanaprod.my/pkp/web/static/js/426.d66c65e8.chunk.js:2:334702)\n at XMLHttpRequest.f.onreadystatechange
(https://kibanaprod.my/pkp/web/static/js/426.d66c65e8.chunk.js:2:142489)","config":{"url":"settings","method":"post","data
@sscarduzio, sorry to bother you.
Ever since we upgraded to 7.16.X (upgrading to 7.17 also not helping) , we are stuck. We keep adding new users and modifying the users.
We need your help to fix this issue. I am 100% sure, this is not NGINX related.
Or please provide an alternative on how we can edit the Security settings of ROR without restarting the Nodes? Is there another way?
Thanks @praveenmak.
@Dzuming can you have a look at this? There is good information from the logged exception form Express, but I’m not seeing what possibly went wrong.
Hello @praveenmak
I checked Nginx with this location block configuration and it seems to work
location ~ / {
proxy_pass https://localhost:5601;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_cookie_path ~*^/.* /;
}
where https://localhost:5601 is a kibana URL
I added it into a kibana.yml
server.basePath: '/kibana'
server.rewriteBasePath: true
Could you verify that this Nginx and kibana.yml configuration works in your case ? I’m using The Enterprise-1.38.0_es7.17.0 version. If you don't need kibana basePath, you can skip the kibana.yml` config edit step.
@praveenmak can you please expand on what’s the final goal with using Nginx as a reverse proxy with this configuration? The information is not sufficient for us to reproduce the issue.
example.com and now you want ROR to show on a path i.e. example.com/kibana
example.com and kibana.example.com
How does your main nginx configuration look like? Are you following a particular guide/tutorial?
@sscarduzio - We don’t use Kibana BasePath.
The NGINX conf rewrite rule has been working since last 4 years with ROR enterprise plugin. After we upgraded to 7.16 it has been broken.
I am using this Location path in NGINX ever since I stood up the ELK cluster.
location ~ (/app|/translations|/pkp|/pkp/web|/node_modules|/built_assets/|/bundles|/es_admin|/plugins|/api|/ui|/elasticsearch|/spaces/enter|/s|/k)
Customers use this UR to login using ROR plugin without /kibana. Like I said , we never used BasePath. The below host is in our Private cloud Openstack DNS entry.
https://kibanaprod.my
Hope this helps, let me know what other information I can provide.
The Root Nginx.conf
location / {
# Main location Kibana default
root /usr/share/kibana/;
access_log /data/nginx/kibana-slash.log kibana buffer=1024 flush=5m;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Authorization "";
proxy_pass http://127.0.0.1:5600/;
}
The sub-url rewrite rule NGINX path.
location ~ (/app|/translations|/node_modules|/built_assets/|/bundles|/es_admin|/plugins|/api|/ui|/elasticsearch|/spaces/enter|/s|/k|/pkp/web|/pkp/api/ settings) {
root /usr/share/kibana/;
access_log /data/nginx/kibana-app.log kibana buffer=1024 flush=5m;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $http_host;
proxy_set_header Authorization "";
proxy_hide_header Authorization;
proxy_pass http://127.0.0.1:5600$request_uri;
}
Am I missing something? I am not so familiar with NGINX, I read some blogs to come up with this.
Thanks @Dzuming are you guys using ONLY “root” location?
Yes, our location looks like this:
location ~ / {
proxy_pass http://127.0.0.1:5600;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_cookie_path ~*^/.* /;
}
Please see mine , there is only one additional line “Authorization” in mine.
But anyways, I used yours and it didn’t help.
And I really doubt that this has anything to do with NGINX.
Are you guys using Kibana Namespace ? I am using.
I am not using BasePath.
Nothing in NGINX logs that stands out.
@praveenmak Thanks for checking everything. I analyzed all your previous messages (it was very helpful, thanks) to try to reproduce this issue, looks like I found probably the root of this issue. To be sure, could you send me your kibana.yml config? (you can send it via pm or email if you want). I’m especially interested whitelistedPath property (if it’s declared)