401: Unauthorized. [undefined] Forbidden by ReadonlyREST ES plugin, with { due_to={ 0="OPERATION_NOT_ALLOWED" } } on Kibana when accessing ILM

1

Hi,

When I try to open Kibana Index Lifecycle Policies page I get Error loading policies "401: Unauthorized. [undefined] Forbidden by ReadonlyREST ES plugin, with { due_to={ 0=“OPERATION_NOT_ALLOWED” } } .

I have this issue with plugin version 1.19.0.

Based on other topic on forum I update plugin to latest version.

readonlyrest

  • Plugin information:
    Name: readonlyrest
    Description: Safely expose Elasticsearch REST API
    Version: 1.23.0
    Elasticsearch Version: 7.5.2
    Java Version: 1.8
    Native Controller: false
    Extended Plugins: []
  • Classname: tech.beshu.ror.es.ReadonlyRestPlugin

but I have still the same issue.

Any ideas how to fix it?

@Simone,
Answer your question from my last post my company hasn’t an Enterprise contract.

Exanta

2

@Simone,

Any ideas?

BR
Exanta

3

@exanta could you please show us a ES log related to the forbidden request?

4

Mateusz,

ES log:

[2020-09-29T00:49:07,946][INFO ][t.b.r.a.l.AccessControlLoggingDecorator] [xxxx.xxxxxxxx.xxxx] FORBIDDEN by default req={ ID:1338697028-1#18104590, TYP:GetLifecycleAction$Request, CGR:N/A, USR:[no info about user], BRS:false, KDX:null, ACT:cluster:admin/ilm/get, OA:127.0.0.1/32, XFF:null, DA:127.0.0.1/32, IDX:<N/A>, MET:GET, PTH:/_ilm/policy, CNT:<N/A>, HDR:Connection=keep-alive, Content-Length=0, Host=127.0.0.1:9200, HIS:[AAAAAAAAAAAAAA-> RULES:[groups->false]], [BBBBBBBBBBBBBBBB-> RULES:[groups->false]], [CCCCCCCCCCCCCC-> RULES:[groups->false]], [DDDDDDDDDD-> RULES:[groups->false]], [EEEEEEEEEEEEEE-> RULES:[groups->false]], [FFFFFFFFFFF-> RULES:[groups->false]], [GGGGGGGGGGGGGGGGGG-> RULES:[groups->false]], [HHHHHHHHHHHHHHHHH-> RULES:[groups->false]], [IIIIIIIIIII-> RULES:[groups->false]], [JJJJJJJJJJ-> RULES:[groups->false]], [::ADMIN::-> RULES:[auth_key->false]], [::ADMIN ABCD::-> RULES:[groups->false]], [::LOGSTASH::-> RULES:[auth_key->false]], [::MONITORING::-> RULES:[hosts_local->true, actions->false]] }

BR
Exanta

5

To me, this seems to be case of missing auth headers from Kibana. I think using the ROR Kibana plugin should solve your issue.

6

@exanta do you use Kibana plugin?

7

No, but based on @askids advice I have installed (succesfully) ROR Kibana plugin (readonlyrest_kbn_free-1.23.0_es7.5.2.zip).

Installed succesfully but after installation I can’t log into kibana.
I get “{“statusCode”:500,“error”:“Internal Server Error”,“message”:“An internal server error occurred”}”

In my kibana.log I found:

> {"type":"log","@timestamp":"2020-09-29T21:06:29Z","tags":["info","readonlyrest_kbn:enrichFromEs"],"pid":6302,"message":"Using groupCurrent 'undefined' - received identity payload: {\"x-ror-username\":\"admin\"}"}
> {"type":"log","@timestamp":"2020-09-29T21:06:29Z","tags":["info","readonlyrest_kbn:loginHandler"],"pid":6302,"message":"identity error with identity: {\"x-ror-username\":\"admin\",\"username\":\"admin\",\"authHeaders\":{\"authorization\":\"Basic YWRtaW46ZGV2\"},\"hiddenApps\":[],\"kibanaTemplateIndex\":null,\"sid\":\"98475fd3-b7c3-4262-bd2c-3e03710903a1\",\"expiresAt\":1601672789938}"}
> {"type":"log","@timestamp":"2020-09-29T21:06:29Z","tags":["error","readonlyrest_kbn:loginHandler"],"pid":6302,"message":"error was: "}
> {"type":"log","@timestamp":"2020-09-29T21:06:30Z","tags":["error","readonlyrest_kbn:onPreResponse"],"pid":6302,"message":"got an error [500] Internal Server Error for path /login"}
> {"type":"log","@timestamp":"2020-09-29T21:06:30Z","tags":["error","readonlyrest_kbn:onPreResponse"],"pid":6302,"message":"ES just returned an error stack trace error, will return the useful error."}
> {"type":"error","@timestamp":"2020-09-29T21:06:28Z","tags":[],"pid":6302,"level":"error","error":{"message":"child \"kibana\" fails because [child \"index\" fails because [\"index\" must be a string]]","name":"ValidationError","stack":"ValidationError: child \"kibana\" fails because [child \"index\" fails because [\"index\" must be a string]]\n    at Object.exports.process (/usr/share/kibana/node_modules/joi/lib/errors.js:196:19)\n    at internals.Object._validateWithOptions (/usr/share/kibana/node_modules/joi/lib/types/any/index.js:675:31)\n    at module.exports.internals.Any.root.validate (/usr/share/kibana/node_modules/joi/lib/index.js:146:23)\n    at Config._commit (/usr/share/kibana/src/legacy/server/config/config.js:132:34)\n    at Config.set (/usr/share/kibana/src/legacy/server/config/config.js:102:10)\n    at obtainCurrentKibanaIndex (/usr/share/kibana/plugins/readonlyrest_kbn/server/routes/lib/identityManager.js:186:19)\n    at Object.setServerSideSession (/usr/share/kibana/plugins/readonlyrest_kbn/server/routes/lib/identityManager.js:276:18)\n    at writeIdentity (/usr/share/kibana/plugins/readonlyrest_kbn/server/routes/lib/auth.js:186:21)\n    at loginHandler (/usr/share/kibana/plugins/readonlyrest_kbn/server/routes/lib/auth.js:140:13)\n    at process._tickCallback (internal/process/next_tick.js:68:7)"},"url":{"protocol":null,"slashes":null,"auth":null,"host":null,"port":null,"hostname":null,"hash":null,"search":null,"query":{},"pathname":"/login","path":"/login","href":"/login"},"message":"child \"kibana\" fails because [child \"index\" fails because [\"index\" must be a string]]"}
> {"type":"response","@timestamp":"2020-09-29T21:06:28Z","tags":[],"pid":6302,"method":"post","statusCode":500,"req":{"url":"/login","method":"post","headers":{"host":"xx.xx.xx.xx:5601","connection":"keep-alive","content-length":"27","cache-control":"max-age=0","upgrade-insecure-requests":"1","origin":"http://xx.xx.xx.xx:5601","content-type":"application/x-www-form-urlencoded","user-agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.122 Safari/537.36","accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9","referer":"http://xx.xx.xx.xx:5601/login?nextUrl=/","accept-encoding":"gzip, deflate","accept-language":"en-US,en;q=0.9","kbn-xsrf":"7.5.2","kbn-version":"7.5.2"},"remoteAddress":"10.112.32.160","userAgent":"10.112.32.160","referer":"http://xx.xx.xx.xx:5601/login?nextUrl=/"},"res":{"statusCode":500,"responseTime":1272,"contentLength":9},"message":"POST /login 500 1272ms - 9.0B"}

Any ideas?

8

Yes. 1.23.0 ROR kibana plugin has a known issue which causes this error code 500. @sscarduzio is aware of this issue. I would suggest that you temporarily try it with 1.22.1.

For downloading previous version, use below link.

https://api.beshu.tech/download/kbn?edition=kbn_free&email=your_email_address&pluginVersion=1.22.1&esVersion=7.5.2

You can update the edition based on your subscription. More details on downloading different editions can be found in documentation site.

9

I will try version 1.22.1.

One more question. I need to downgrade Elasticsearch plugin do the same version (1.22) or it will work with different versions?

BR

10

You can temporarily try with 1.22.1. It will work. But its typically recommended to keep the version consistent across these 2 plugins.

11

We have 1.23.1 available that should address this.

12

Thanks @sscarduzio. Looks like download links are providing 1.23.1 during download, but modlog on download page is not yet updated and shows 1.23.0 as latest.

13

it’s fixed now. Thanks

1 Like
14

Both 1.22.1 and 1.23.1 fix my issue.

Thanks a lot !!

1 Like