Sorry Pavel that I was no so precise as you expect me to do.
In my previous post I focused on explainning the ACL and why the FORBIDDEN log look like I showed.
I assumed that it clarify the misunderstanding. I see it’s not, so, I’ll try to exlpain it deeper and refer specifically to what you are asking about.
Let’s start with my understanding of the root of the problem from this thread.
I would use examples. I think this is simplified version of what you have in your ROR settings:
readonlyrest:
# in this section of ROR, we primarily determine which local groups each user belongs to
users:
- username: user1
auth_key: user1:test # it could be any authentication/authorization rule here - it's not relevant in this example
groups: [EndUsers, BusinessUsers]
- username: user2
auth_key: user2:test # it could be any authentication/authorization rule here - it's not relevant in this example
groups: [EndUsers]
access_control_rules:
# this block is used by the Kibana technical requests only
- name: "KIBANA"
type: allow
auth_key: kibana:kibana
verbosity: error
# block that allows access to "frontend_logs" index for "EndUsers" tenant/group
- name: "End users"
groups: ["EndUsers"]
indices: ["frontend_logs"]
# block that allows access to "business_logs" index for "BusinessUsers" tenant/group
- name: "Business users"
groups: ["BusinessUsers"]
indices: ["business_logs"]
Sometimes your users asked you (admin) why they cannot do something.
Let’s consider adding a new document (the example you used):
Example 1:
The “user2” that has no permission to “business_logs” index, is trying to add a new document:
curl -k -u user2:test -XPOST https://localhost:19200/business_logs/_doc -H "Content-Type: application/json" -d '{"timestamp": "2024-01-15T10:30:00Z","user_id": "user123","action": "purchase","revenue": 299.99,"product": "Premium Subscription","region": "US-East","status": "completed"}'
{"error":{"root_cause":[{"type":"forbidden_response","reason":"Forbidden by ReadonlyREST","due_to":"OPERATION_NOT_ALLOWED"}],"type":"forbidden_response","reason":"Forbidden by ReadonlyREST","due_to":"OPERATION_NOT_ALLOWED"},"status":403}
In logs (and in the audit too) we can find:
FORBIDDEN by default req={ ID:ea249e8b-0790-457e-a78c-a613ee6cbc82-985068797#20985, TYP:IndexRequest, CGR:<N/A>, USR:user2 (attempted), BRS:true, KDX:null, ACT:indices:data/write/index, OA:172.67.160.34/32, XFF:null, DA:172.18.0.2/32, IDX:business_logs, MET:POST, PTH:/business_logs/_doc, CNT:<OMITTED, LENGTH=173.0 B> , HDR:Accept=*/*, Content-Type=application/json, User-Agent=curl/8.7.1, Host=localhost:19200, Content-Length=173, Authorization=<OMITTED>, HIS:[KIBANA-> RULES:[auth_key->false] RESOLVED:[indices=business_logs]], [End users-> RULES:[groups_any_of->true, indices->false] RESOLVED:[user=user2;group=EndUsers;av_groups=EndUsers;indices=business_logs]], [Business users-> RULES:[groups_any_of->false] RESOLVED:[indices=business_logs]], }
How we can interpret the HIS part?
- “KIBANA” block was not matched because the “auth_key” rule was not matched (auth_key->false)
- “End users” block was not matched because the “indices” rule was not matched (indices->false). However, we can see that the “groups_any_of” in this block was matched (groups_any_of->true), so user was authenticated and groups were authorized
- “Business users” block was not matched because the “groups_any_of” was not matched (groups_any_of->false)
Result:
The request was not matched by any block, so it was forbidden (“FORBIDDEN by default”). ES returned 403 - request was not authorized.
Example 2:
Let’s consider the same request but let’s assume that user entered wrong password:
curl -k -u user2:wrongpassowed -XPOST https://localhost:19200/business_logs/_doc -H "Content-Type: application/json" -d '{"timestamp": "2024-01-15T10:30:00Z","user_id": "user123","action": "purchase","revenue": 299.99,"product": "Premium Subscription","region": "US-East","status": "completed"}'
{"error":{"root_cause":[{"type":"forbidden_response","reason":"Forbidden by ReadonlyREST","due_to":"OPERATION_NOT_ALLOWED"}],"type":"forbidden_response","reason":"Forbidden by ReadonlyREST","due_to":"OPERATION_NOT_ALLOWED"},"status":403}
In logs (and in the audit too) we can find:
FORBIDDEN by default req={ ID:d506e69f-7955-482e-b7e0-784f7e6efd26-1865827387#25149, TYP:IndexRequest, CGR:<N/A>, USR:user2 (attempted), BRS:true, KDX:null, ACT:indices:data/write/index, OA:172.67.160.34/32, XFF:null, DA:172.18.0.2/32, IDX:business_logs, MET:POST, PTH:/business_logs/_doc, CNT:<OMITTED, LENGTH=173.0 B> , HDR:Accept=*/*, Content-Type=application/json, User-Agent=curl/8.7.1, Host=localhost:19200, Content-Length=173, Authorization=<OMITTED>, HIS:[KIBANA-> RULES:[auth_key->false] RESOLVED:[indices=business_logs]], [End users-> RULES:[groups_any_of->false] RESOLVED:[indices=business_logs]], [Business users-> RULES:[groups_any_of->false] RESOLVED:[indices=business_logs]], }
How we can interpret the HIS part?
- “KIBANA” block was not matched because the “auth_key” rule was not matched (auth_key->false)
- “End users” block was not matched because the “groups_any_of” was not matched (groups_any_of->false)
- “Business users” block was not matched because the “groups_any_of” was not matched (groups_any_of->false)
Result:
The request was not matched by any block, so it was forbidden (“FORBIDDEN by default”). ES returned 403 - request was not authorized.
Comparision with the “Example 1”:
In log of the “Example 1”, we saw that in the “End users” block the “groups_any_of” rule was matched (it means that authentication and groups authorization ended up with success). In this example we see that the “groups_any_of” of the block failed to matched, but it’s hard to reason why. In other blocks we have no clue too.
Example 3:
Let’s consider the same request but let’s assume that non existing user is trying to add new document:
curl -k -u nonexisting:pass -XPOST https://localhost:19200/business_logs/_doc -H "Content-Type: application/json" -d '{"timestamp": "2024-01-15T10:30:00Z","user_id": "user123","action": "purchase","revenue": 299.99,"product": "Premium Subscription","region": "US-East","status": "completed"}'
{"error":{"root_cause":[{"type":"forbidden_response","reason":"Forbidden by ReadonlyREST","due_to":"OPERATION_NOT_ALLOWED"}],"type":"forbidden_response","reason":"Forbidden by ReadonlyREST","due_to":"OPERATION_NOT_ALLOWED"},"status":403}⏎
In logs (and in the audit too) we can find:
FORBIDDEN by default req={ ID:6a55627e-b283-4f4d-91b1-e6a54e7f2e08-360200855#29062, TYP:IndexRequest, CGR:<N/A>, USR:nonexisting (attempted), BRS:true, KDX:null, ACT:indices:data/write/index, OA:172.67.160.34/32, XFF:null, DA:172.18.0.2/32, IDX:business_logs, MET:POST, PTH:/business_logs/_doc, CNT:<OMITTED, LENGTH=173.0 B> , HDR:Accept=*/*, Content-Type=application/json, User-Agent=curl/8.7.1, Host=localhost:19200, Content-Length=173, Authorization=<OMITTED>, HIS:[KIBANA-> RULES:[auth_key->false] RESOLVED:[indices=business_logs]], [End users-> RULES:[groups_any_of->false] RESOLVED:[indices=business_logs]], [Business users-> RULES:[groups_any_of->false] RESOLVED:[indices=business_logs]], }
How we can interpret the HIS part?
- “KIBANA” block was not matched because the “auth_key” rule was not matched (auth_key->false)
- “End users” block was not matched because the “groups_any_of” was not matched (groups_any_of->false)
- “Business users” block was not matched because the “groups_any_of” was not matched (groups_any_of->false)
Result:
The request was not matched by any block, so it was forbidden (“FORBIDDEN by default”). ES returned 403 - request was not authorized.
Comparision with the “Example 2”:
As we can see we got the same for as in the “Example 2” for non-existing user. So, looking at logs we don’t know if the user doesn’t exist or if the password was wrong. If you would use eg. ldap_auth in the “users” section, we could say the same about group membership. We couldn’t distinquish if user entered wrong credentials or if they simply is not a member of some group configured in the ldap_auth rule.
–
The 3 examples show the problem I tried to solve.
Before we move on, I will show how the problem is solved in the currently developer ROR ES version:
Example 1 (improved):
curl -k -u user2:test -XPOST https://localhost:19200/business_logs/_doc -H "Content-Type: application/json" -d '{"timestamp": "2024-01-15T10:30:00Z","user_id": "user123","action": "purchase","revenue": 299.99,"product": "Premium Subscription","region": "US-East","status": "completed"}'
{"error":{"root_cause":[{"type":"forbidden_response","reason":"Forbidden by ReadonlyREST","due_to":"OPERATION_NOT_ALLOWED"}],"type":"forbidden_response","reason":"Forbidden by ReadonlyREST","due_to":"OPERATION_NOT_ALLOWED"},"status":403}
In logs (and in the audit too) we can find:
FORBIDDEN by mismatch req={ ID:1b74ce59-8503-40c4-8899-8543b8372da2-1677372260#5849, TYP:IndexRequest, CGR:<N/A>, USR:user2 (attempted), BRS:true, KDX:null, ACT:indices:data/write/index, OA:172.67.160.34/32, XFF:null, DA:172.18.0.2/32, IDX:business_logs, MET:POST, PTH:/business_logs/_doc, CNT:<OMITTED, LENGTH=173.0 B> , HDR:Accept=*/*, Content-Type=application/json, User-Agent=curl/8.7.1, Host=localhost:19200, Content-Length=173, Authorization=<OMITTED>, HIS:[KIBANA: NOT_MATCHED (AUTH_FAIL(Username mismatch)) -> RULES:[auth_key->false]], [End users: NOT_MATCHED (AUTHZ_FAIL) -> RULES:[groups_any_of->true, indices->false]], [Business users: NOT_MATCHED (GROUPS_AUTH_FAIL(user2:GROUPS_AUTH_FAIL(No user's groups allowed); user1:AUTH_FAIL(Username mismatch))) -> RULES:[groups_any_of->false]] }
How we can interpret the HIS part this time?
- “KIBANA” block was not matched because the “auth_key” rule was not matched (auth_key->false). We also see detailed reason: AUTH_FAIL(Username mismatch)
- “End users” block was not matched because the “indices” rule was not matched (indices->false). However, we can see that the “groups_any_of” in this block was matched (groups_any_of->true), so user was authenticated and groups were authorized. We also see detailed reason: NOT_MATCHED (AUTHZ_FAIL) - it means, “other than groups authorization fail”.
- “Business users” block was not matched because the “groups_any_of” was not matched (groups_any_of->false). We also see the detailed reason: NOT_MATCHED (GROUPS_AUTH_FAIL(user2:GROUPS_AUTH_FAIL(No user’s groups allowed); user1:AUTH_FAIL(Username mismatch)) - here we see that in the “users” section the “user2” section was rejected because the groups authorization failed. The “user1” section was rejected because of username mismatch.
Result:
The request was not matched by any block, so it was forbidden (“FORBIDDEN by default”). ES returned 403 - request was not authorized. Details in the HIS block say us more about the rejection details.
Example 2 (improved):
Let’s consider the same request but let’s assume that user entered wrong password:
curl -k -u user2:wrongpassowed -XPOST https://localhost:19200/business_logs/_doc -H "Content-Type: application/json" -d '{"timestamp": "2024-01-15T10:30:00Z","user_id": "user123","action": "purchase","revenue": 299.99,"product": "Premium Subscription","region": "US-East","status": "completed"}'
{"error":{"root_cause":[{"type":"forbidden_response","reason":"Forbidden by ReadonlyREST","due_to":"OPERATION_NOT_ALLOWED"}],"type":"forbidden_response","reason":"Forbidden by ReadonlyREST","due_to":"OPERATION_NOT_ALLOWED"},"status":403}
In logs (and in the audit too) we can find:
FORBIDDEN by mismatch req={ ID:924d2b14-22f5-44dd-9e9a-9869838291d1-669128476#10456, TYP:IndexRequest, CGR:<N/A>, USR:user2 (attempted), BRS:true, KDX:null, ACT:indices:data/write/index, OA:172.67.160.34/32, XFF:null, DA:172.18.0.2/32, IDX:business_logs, MET:POST, PTH:/business_logs/_doc, CNT:<OMITTED, LENGTH=173.0 B> , HDR:Accept=*/*, Content-Type=application/json, User-Agent=curl/8.7.1, Host=localhost:19200, Content-Length=173, Authorization=<OMITTED>, HIS:[KIBANA: NOT_MATCHED (AUTH_FAIL(Username mismatch)) -> RULES:[auth_key->false]], [End users: NOT_MATCHED (GROUPS_AUTH_FAIL(user2:AUTH_FAIL(Invalid password); user1:AUTH_FAIL(Username mismatch))) -> RULES:[groups_any_of->false]], [Business users: NOT_MATCHED (GROUPS_AUTH_FAIL(user2:GROUPS_AUTH_FAIL(No user's groups allowed); user1:AUTH_FAIL(Username mismatch))) -> RULES:[groups_any_of->false]] }
How we can interpret the HIS part this time?
- “KIBANA” block was not matched because the “auth_key” rule was not matched (auth_key->false). We also see detailed reason: AUTH_FAIL(Username mismatch)
- “End users” block was not matched because the “groups_any_of” was not matched (groups_any_of->false). We also see detailed reason: NOT_MATCHED (GROUPS_AUTH_FAIL(user2:AUTH_FAIL(Invalid password); user1:AUTH_FAIL(Username mismatch)). This time we see that “user2” section in the “users” section was rejected because of “Invalid password”.
- “Business users” block was not matched because the “groups_any_of” was not matched (groups_any_of->false). We also see the detailed reason: NOT_MATCHED (GROUPS_AUTH_FAIL(user2:GROUPS_AUTH_FAIL(No user’s groups allowed); user1:AUTH_FAIL(Username mismatch)) - here we see that in the “users” section the “user2” section was rejected because the groups authorization failed. The “user1” section was rejected because of username mismatch.
Result:
The request was not matched by any block, so it was forbidden (“FORBIDDEN by default”). ES returned 403 - request was not authorized. Details in the HIS block say us more about the rejection details.
Example 3 (improved):
Let’s consider the same request but let’s assume that non existing user is trying to add new document:
curl -k -u nonexisting:pass -XPOST https://localhost:19200/business_logs/_doc -H "Content-Type: application/json" -d '{"timestamp": "2024-01-15T10:30:00Z","user_id": "user123","action": "purchase","revenue": 299.99,"product": "Premium Subscription","region": "US-East","status": "completed"}'
{"error":{"root_cause":[{"type":"forbidden_response","reason":"Forbidden by ReadonlyREST","due_to":"OPERATION_NOT_ALLOWED"}],"type":"forbidden_response","reason":"Forbidden by ReadonlyREST","due_to":"OPERATION_NOT_ALLOWED"},"status":403}
In logs (and in the audit too) we can find:
FORBIDDEN by mismatch req={ ID:92bda945-6449-4831-beb6-ece45fefb79d-1514476156#14060, TYP:IndexRequest, CGR:<N/A>, USR:nonexisting (attempted), BRS:true, KDX:null, ACT:indices:data/write/index, OA:172.67.160.34/32, XFF:null, DA:172.18.0.2/32, IDX:business_logs, MET:POST, PTH:/business_logs/_doc, CNT:<OMITTED, LENGTH=173.0 B> , HDR:Accept=*/*, Content-Type=application/json, User-Agent=curl/8.7.1, Host=localhost:19200, Content-Length=173, Authorization=<OMITTED>, HIS:[KIBANA: NOT_MATCHED (AUTH_FAIL(Username mismatch)) -> RULES:[auth_key->false]], [End users: NOT_MATCHED (GROUPS_AUTH_FAIL({user1,user2}:AUTH_FAIL(Username mismatch))) -> RULES:[groups_any_of->false]], [Business users: NOT_MATCHED (GROUPS_AUTH_FAIL(user2:GROUPS_AUTH_FAIL(No user's groups allowed); user1:AUTH_FAIL(Username mismatch))) -> RULES:[groups_any_of->false]] }
How we can interpret the HIS part this time?
- “KIBANA” block was not matched because the “auth_key” rule was not matched (auth_key->false). We also see detailed reason: AUTH_FAIL(Username mismatch)
- “End users” block was not matched because the “groups_any_of” was not matched (groups_any_of->false). We also see detailed reason: NOT_MATCHED (GROUPS_AUTH_FAIL({user1,user2}:AUTH_FAIL(Username mismatch))). Both sections in “users” part of ROR settings were not matched because of “Username mismatch”
- “Business users” block was not matched because the “groups_any_of” was not matched (groups_any_of->false). We also see the detailed reason.
This was a little bit too long preface. Now, I’m going to respond to your questions and comments.
I told you what would be let’s say a standard behaviour (in my opinion) and most user’s natural reaction after reading (AUTH_FAIL(Username mismatch)).
Yes, I cannot agree with it. When you think about typical, most common approach with is RBAC (Role based access control), you can just identified the following use cases:
- user is successfully authenticated and has access to resource
- user is successfully authenticated but has NOT access to resource
- user is NOT successfully authenenticated
But in our context it looks different. Simone explained it well above.
I’d add that ROR’s ACL can be compared to sth what you probably saw in Nginx configuration. AFAIR, in the Nginx configration we could define “locations”. When request in handled by Nginx, it goes though each location (starting from the upper one, going down in the location’s list) and tries to match it. It it matches location, the request is allowed (it’s obivously simplified use case). Then no location is matched, the request is forbidden/rejected.
The same we have it ROR. ROR goes though each block (in each block, it check each rule) and tries to match block. If all rules in the block are matched, the block is matched. When one rule in the block is mismatched, the whole block is considered being mismatched, and ROR tries the next block. When no block was matched, the request is forbidden and ROR returns 403 (always).
Each block could (but don’t have to) have an authentication or/and authorization rule. When the block is checked, the authn/authz/auth rules are checked in the first order. So, we can imagine that block could be rejected because of:
- authentication failure (user doesn’t exist, password mismatch)
- groups authorization failure (authz/auth rules) - the authenticated user doesn’t belong to the specified groups
- general/other authorization failure - e.g. user has no access to given index, or to given ES action, etc.
So, when the request is rejected we could end up with sth like that:
- “KIBANA” block - authentication failure
- “End users” block - general authorization failure (cannot access to index)
- “Business users” block - groups authorization failure
In the ACL approach, we cannot simply said that it was 401 or 403. We know that the request was not forbidden by the ACL defined rules.
that RoR evaluates the rules one by one until one of them matches (or none at all), so why do we need to see all the rules that did not match, it’s somehow redundand (if we open RoR configuration and find the rule that matched, we can also easily find all the preceding rules that did not match).
NOTE: I guess that saying a “rule” here, you had in mind a “block”, right?
I’d answer consider two cases:
- FORBIDDEN request
When request was forbidden, ROR went though all blocks and none was matched (TBH, there is the forbidden policy type block, but let’s not focus on this case now). To be able to analyse why your request was FORBIDDEN you have to know why each block was forbidden. That’s why we show all blocks history.
- ALLOWED request
When request is allowed, one block was matched. If this was not the first block, the HIS will contain also preceding blocks that was not matched. This information is needed for analysis too. You may ask “Why my request was allowed by this block, not by e.g. one of the preceding blocks”.
I think our natural habit is to read from left to right, not from right to left. With that said, some RoR audit setting (just to keep backwards compatibility) to write only the rule that matched would be nice-to-have as preceding unmatched rules are easy to find and following rules are not being evaluated anyway (I apologize if I’m missing something).
We could prepare such audit serializer that skips all unmatched blocks in history and adds info about the matched block only, but notice that it’s only refer to the 2nd case (see the above). And as you probably already noticed, some information could be lost or at least it would be harder to answer the question “Why my request was allowed by this block, not by e.g. one of the preceding blocks”.
Also you completely omitted the 401 vs 403 part… If there is a way to distinguish between wrong credentials in audit log and just not having permissions for some particular action, that also means that RoR “knows” at some point that user is not authenticated. Why not to ,return in such a case standard HTTP code 401 instead of misleading 403 (otherwise we should admit that there is no way how to reliably distinguish between failed authentication and authorization, i.e. insufficient permissions).
I think I already answet that in my first comment above in this post. Do you feel this one? I’m obviously understand that this is not the “natural” approach in the security layers we know from the most of apps we use on a daily basis. ROR uses the ACL for the historical reasons (maybe Simone reveal to us why this approach was picked ;)). Some things are easier to do in the ACL approach, some in the RBAC. It’s about the foundations. IMO, the problem raised in this tread rubs against the foundations.
Does this mean that we can only throw up our hands? Hell, NO!
We are planning to introduce the RBAC engine to the ROR. The goal is to let user pick the approach: ACL or RBAC. Looking at your ROR settings, I would guess that the RBAC approach will fit your settings well. But it’s in the design phase now.
What’s now? As I shown, we improved the causes a little bit. I understand now, that it doesn’t solve your problem in depth. But as I was trying to prove - IMO it’s about the approach.
Or maybe you have some brilliant idea how we could do it in the ACL approach? How we could make your support easier?
I’d be great if you could show this to us using the examples from this post. Then we will discuss here the potenantial problems of the proposed solution.
.
. My configuration is as follows (I use the config file from the RoR sandbox “as is” with one slight change - I explicitly gave admin all permissions for all indices):