JWT Forbidden 403

1

I know quite a few folks ran into this. Though i referred to all the topics, I couldnt get this to working. Below is my JWT configuration.

- name: Valid JWT token with a viewer role
  kibana_access: rw
  jwt_auth:
    name: "jwt_provider_1"
    roles: ["writer"]
  indices: [ ".kibana-cfxdls-customer1", "coke_gdt_*"]
  kibana_index: ".kibana-cfxdls-customer1"
  kibana_hide_apps: ["readonlyrest_kbn", "timelion", "kibana:dev_tools"]

jwt:
- name: jwt_provider_1
  signature_algo: HMAC
  signature_key: "wr8sFmw1jP9JpBjPGKJwvOJg4mIqAwMOaRZ7UrkhIgs_IqlORSEFVU3Go-Ozj9strEIb_NuLqdTCYz9IlZ-pa13S7lun7eyHwzHDKw8m-1LVq0hMUldT0JPLE7OQBY3hhnUk0gtfFW61x31frt7AMZuAPY9UbcaIbVfvwAb5xQgPlc1QkAA-uRfO2qUMPXRQcpIJSbWlqqZioFoAHUiJytRsg_QaWE1kmjHrkg1ueBZqaSXZyBQHnBepPvw8tX9dn-X1yTygiZZ9r1MNSNR64wXizXA8avzvDMMaJMiN1keLfntGjqV2yo6dPhvvc0YApQKpFXCmYsNSYBaHPQDQ5w"
  user_claim: customerid
  roles_claim: resource_access.client-app.roles # JSON-path style
  header_name: Authorization

Below is how i generated the JWT token.

import jwt >>> encoded_jwt = jwt.encode({‘customerid’: ‘customer1’, ‘roles’: ‘writer’}, ‘wr8sFmw1jP9JpBjPGKJwvOJg4mIqAwMOaRZ7UrkhIgs_IqlORSEFVU3Go-Ozj9strEIb_NuLqdTCYz9IlZ-pa13S7lun7eyHwzHDKw8m-1LVq0hMUldT0JPLE7OQBY3hhnUk0gtfFW61x31frt7AMZuAPY9UbcaIbVfvwAb5xQgPlc1QkAA-uRfO2qUMPXRQcpIJSbWlqqZioFoAHUiJytRsg_QaWE1kmjHrkg1ueBZqaSXZyBQHnBepPvw8tX9dn-X1yTygiZZ9r1MNSNR64wXizXA8avzvDMMaJMiN1keLfntGjqV2yo6dPhvvc0YApQKpFXCmYsNSYBaHPQDQ5w’, algorithm=‘HS256’)

encoded_jwt
‘eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJjdXN0b21lcmlkIjoiY3VzdG9tZXIxIiwicm9sZXMiOiJ3cml0ZXIifQ.20sGik490NEtgVphsnjfYadapuiWfZElHgm_pJTIin0’

jwt.decode(encoded_jwt, verify=False) {u’customerid’: u’customer1’, u’roles’: u’writer’}

How i launched the login is

https://10.95.101.140/***/****/login?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJjdXN0b21lcmlkIjoiY3VzdG9tZXIxIiwicm9sZXMiOiJ3cml0ZXIifQ.20sGik490NEtgVphsnjfYadapuiWfZElHgm_pJTIin0

I have the below config in kibana.

readonlyrest_kbn.jwt_query_param: “jwt”

[2019-01-27T16:19:16,492][DEBUG][t.b.r.a.b.Block ] [cfxdls-esclient-elk-04.qa.engr.cloudfabrix.com] [Valid JWT token with a viewer role] the request matches no rules in this block: { ID:1857277983-1391166973#162, TYP:RRAdminRequest, CGR:N/A, USR:customer1, BRS:false, KDX:null, ACT:cluster:admin/rradmin/refreshsettings, OA:10.95.101.140, DA:10.95.101.133, IDX:<N/A>, MET:GET, PTH:/_readonlyrest/metadata/current_user, CNT:<N/A>, HDR:{authorization=Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJjdXN0b21lcmlkIjoiY3VzdG9tZXIxIiwicm9sZXMiOiJ3cml0ZXIifQ.20sGik490NEtgVphsnjfYadapuiWfZElHgm_pJTIin0, Connection=close, content-length=0, Host=10.95.101.140:9201, X-Forwarded-For=10.95.101.133, X-Forwarded-Host=10.95.101.140:9201, X-Forwarded-Port=9201}, HIS:[::LOGSTASH::->[auth_key->false]], [::KIBANA-SRV::->[auth_key->false]], [CFXDLS Tenancy Template->[groups->false]], [Cloudfabrix->[groups->false]], [::RO::->[auth_key->false]], [::RW::->[auth_key->false]], [::ADMIN::->[auth_key->false]], [::RO1::->[auth_key->false]], [::RW1::->[auth_key->false]], [::ADMIN1::->[auth_key->false]], [Valid JWT token with a viewer role->[jwt_auth->false]] }
[2019-01-27T16:19:16,492][TRACE][o.e.t.TaskManager ] [cfxdls-esclient-elk-04.qa.engr.cloudfabrix.com] unregister task for id: 162
[2019-01-27T16:19:16,492][DEBUG][r.suppressed ] [cfxdls-esclient-elk-04.qa.engr.cloudfabrix.com] path: /_readonlyrest/metadata/current_user, params: {}
tech.beshu.ror.es.IndexLevelActionFilter$1$1: Access not allowed.
at tech.beshu.ror.es.IndexLevelActionFilter$1.onForbidden(IndexLevelActionFilter.java:163) ~[?:?]
at tech.beshu.ror.acl.ACL.lambda$check$4(ACL.java:208) ~[?:?]
at java.util.concurrent.CompletableFuture.uniApply(CompletableFuture.java:602) ~[?:1.8.0_172]
at java.util.concurrent.CompletableFuture.uniApplyStage(CompletableFuture.java:614) ~[?:1.8.0_172]
at java.util.concurrent.CompletableFuture.thenApply(CompletableFuture.java:1983) ~[?:1.8.0_172]
at tech.beshu.ror.acl.ACL.check(ACL.java:203) ~[?:?]
at tech.beshu.ror.es.IndexLevelActionFilter.handleRequest(IndexLevelActionFilter.java:158) ~[?:?]
at tech.beshu.ror.es.IndexLevelActionFilter.lambda$apply$1(IndexLevelActionFilter.java:134) ~[?:?]
at java.security.AccessController.doPrivileged(Native Method) ~[?:1.8.0_172]
at tech.beshu.ror.es.IndexLevelActionFilter.apply(IndexLevelActionFilter.java:130) ~[?:?]
at org.elasticsearch.action.support.TransportAction$RequestFilterChain.proceed(TransportAction.java:165) ~[elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:139) ~[elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.action.support.TransportAction.execute(TransportAction.java:81) ~[elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.client.node.NodeClient.executeLocally(NodeClient.java:87) ~[elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.client.node.NodeClient.doExecute(NodeClient.java:76) ~[elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.client.support.AbstractClient.execute(AbstractClient.java:395) ~[elasticsearch-6.5.4.jar:6.5.4]
at tech.beshu.ror.es.rradmin.rest.RestRRAdminAction.lambda$prepareRequest$1(RestRRAdminAction.java:56) ~[?:?]
at org.elasticsearch.rest.BaseRestHandler.handleRequest(BaseRestHandler.java:97) ~[elasticsearch-6.5.4.jar:6.5.4]
at tech.beshu.ror.es.ReadonlyRestPlugin.lambda$null$5(ReadonlyRestPlugin.java:197) ~[?:?]
at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:239) [elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.rest.RestController.tryAllHandlers(RestController.java:335) [elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.rest.RestController.dispatchRequest(RestController.java:173) [elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.http.netty4.Netty4HttpServerTransport.dispatchRequest(Netty4HttpServerTransport.java:545) [transport-netty4-client-6.5.4.jar:6.5.4]
at org.elasticsearch.http.netty4.Netty4HttpRequestHandler.channelRead0(Netty4HttpRequestHandler.java:137) [transport-netty4-client-6.5.4.jar:6.5.4]
at io.netty.channel.SimpleChannelInboundHandler.channelRead(SimpleChannelInboundHandler.java:105) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at org.elasticsearch.http.netty4.pipelining.HttpPipeliningHandler.channelRead(HttpPipeliningHandler.java:68) [transport-netty4-client-6.5.4.jar:6.5.4]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.30.Final.jar:4.1.30.Final]
at io.netty.handler.codec.MessageToMessageCodec.channelRead(MessageToMessageCodec.java:111) [netty-codec-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:102) [netty-codec-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:323) [netty-codec-4.1.30.Final.jar:4.1.30.Final]
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:297) [netty-codec-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.handler.timeout.IdleStateHandler.channelRead(IdleStateHandler.java:286) [netty-handler-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.ChannelInboundHandlerAdapter.channelRead(ChannelInboundHandlerAdapter.java:86) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:340) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1434) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:362) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:348) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:965) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:163) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:644) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:544) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:498) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:458) [netty-transport-4.1.30.Final.jar:4.1.30.Final]
at io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:897) [netty-common-4.1.30.Final.jar:4.1.30.Final]
at java.lang.Thread.run(Thread.java:748) [?:1.8.0_172]
[2019-01-27T16:19:16,498][TRACE][o.e.i.b.request ] [cfxdls-esclient-elk-04.qa.engr.cloudfabrix.com] [request] Adjusted breaker by [16440] bytes, now [16440]
[2019-01-27T16:19:16,498][TRACE][o.e.i.b.in_flight_requests] [cfxdls-esclient-elk-04.qa.engr.cloudfabrix.com] [in_flight_requests] Adjusted breaker by [0] bytes, now [0]
[2019-01-27T16:19:16,499][TRACE][o.e.i.b.request ] [cfxdls-esclient-elk-04.qa.engr.cloudfabrix.com] [request] Adjusted breaker by [-16440] bytes, now [0]
[2019-01-27T16:19:16,499][TRACE][o.e.h.n.Netty4HttpServerTransport] [cfxdls-esclient-elk-04.qa.engr.cloudfabrix.com] channel closed: [id: 0xb885452e, L:/10.95.101.133:9200 ! R:/10.95.101.140:55820]
[2019-01-27T16:19:16,499][INFO ][t.b.r.a.ACL ] [cfxdls-esclient-elk-04.qa.engr.cloudfabrix.com] FORBIDDEN by default req={ ID:1857277983-1391166973#162, TYP:RRAdminRequest, CGR:N/A, USR:customer1, BRS:false, KDX:null, ACT:cluster:admin/rradmin/refreshsettings, OA:10.95.101.140, DA:10.95.101.133, IDX:<N/A>, MET:GET, PTH:/_readonlyrest/metadata/current_user, CNT:<N/A>, HDR:{authorization=Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJjdXN0b21lcmlkIjoiY3VzdG9tZXIxIiwicm9sZXMiOiJ3cml0ZXIifQ.20sGik490NEtgVphsnjfYadapuiWfZElHgm_pJTIin0, Connection=close, content-length=0, Host=10.95.101.140:9201, X-Forwarded-For=10.95.101.133, X-Forwarded-Host=10.95.101.140:9201, X-Forwarded-Port=9201}, HIS:[::LOGSTASH::->[auth_key->false]], [::KIBANA-SRV::->[auth_key->false]], [CFXDLS Tenancy Template->[groups->false]], [Cloudfabrix->[groups->false]], [::RO::->[auth_key->false]], [::RW::->[auth_key->false]], [::ADMIN::->[auth_key->false]], [::RO1::->[auth_key->false]], [::RW1::->[auth_key->false]], [::ADMIN1::->[auth_key->false]], [Valid JWT token with a viewer role->[jwt_auth->false]] }
[2019-01-27T16:19:17,053][TRACE][o.e.i.IndexingMemoryController] [cfxdls-esclient-elk-04.qa.engr.cloudfabrix.com] total indexing heap bytes used [0b] vs indices.memory.index_buffer_size [2.9gb], currently writing bytes [0b]

2

I reviewed my config again and was able to solve with the below change.

  roles_claim: roles

Somehow I was thinkining roles is an internal attribute and hence wasnt focusing on changing this.

1 Like