Provide data enrichment capability using pipeline for ROR audit logs

:bulb: Provide data enrichment capability using pipeline feature for ROR audit logs

Currently ROR provides ability to capture audit logs either using standard audit log serializers that comes out of the box or by creating custom serializers in Scala or Java. Though, these are good features to have full control, this essentially needs some kind of additional coding and dependency on non-ES skills. Instead, ROR should provide an option to mention the pipeline to be included on the ROR audit log indexing action so that corresponding pipeline can be defined to either capture additional information as part of the audit logs or parse the incoming audit log using different pipeline processor. This will allow existing out of box serializers to be extended without needing to build separate custom serializers.

With version 7.5.0 enrich pipeline feature, this will even allow to add additional attributes from another ES index, there by allowing audit logs to be further enriched with additional information and without needing to setup separate logstash enrichment process for the audit logs.

:eyes: Example

New option should be made available at both overall level and ACL block level. Block level pipeline always wins, when this parameter is available at both global and block level.

audit_pipeline: add_userinfo

Other example could be, when query content is parsed and stored as JSON instead of string.

:rocket: Let’s do this?

  • 1
  • 2
  • 3
  • 4
  • 5

0 voters

Currently, we don’t have any other method of creating Serializers than the one described, but recently we have added a new type of output: logs (see documentation)

Introduce a new configuration option audit_pipeline at both the overall level and ACL block level in the ROR settings. This option allows users to specify the Elasticsearch pipeline to be applied during the index ing action of audit logs.
If defined at the global level, the specified pipeline is applied to all audit logs.
ror:
audit_pipeline: global_enrichment_pipeline

This newly added configuration option plays a crucial role in B2B API integration, allowing users to meticulously control the Elasticsearch pipeline applied during the indexing action of audit logs.

The audit_pipeline configuration option can be set at both the overall level and the ACL block level. This means users have the freedom to define specific Elasticsearch pipelines for audit logs either globally or for specific access control lists.

For a comprehensive global application, users can define the audit_pipeline at the overall level. In the provided example, the configuration at the global level is set to global_enrichment_pipeline. This implies that the specified pipeline, in this case, global_enrichment_pipeline, will be universally applied to all audit logs, ensuring a standardized and streamlined approach to the indexing process.

1 Like

@mitchell.washington thanks for the comprehensive description.
@coutoPL I’m curious, WDYT?

BTW @mitchell.washington are you an enterprise or pro user?

I’ve created a jira task for analysis. When it’s done I will write here what can be done in this topic.