entityanalytics_ad: map user group details to ecs fields #13550
Conversation
efd6
added
enhancement
efd6
force-pushed
the
13511-entityanalytics_ad
branch
from
63b02af to
bccafc2
Compare
|
Pinging @elastic/security-service-integrations (Team:Security-Service Integrations) |
|
@jaredburgettelastic I did not include the |
|
@efd6 Based on the data we have seen, copying further into the |
| @@ -358,21 +348,39 @@ processors: | |||
| '549': true | |||
| '551': true | |||
| source: |- | |||
| ctx.user = ctx.user ?: [:]; | |||
| ctx.user.group = ctx.user.group ?: [:]; | |||
There was a problem hiding this comment.
Please correct me if I'm wrong, I'm not a Painless expert, but looks like we're setting user.group.id and user.group.name to a collection (set) of values.
I think in ECS both id and name are keywords, so they only hold a single value.
For our use case, I think we would like these fields to contain the data from the group in the primary_group_id
There was a problem hiding this comment.
Interesting point from an ECS perspective.
Functionality-wise, we (and customers) definitely need the full list of groups for a given entity, not only the primary group.
I believe I've seen a precedent in our integrations for handling multiple values for a given field, which similarly populates multiple values onto a single field, @efd6 can you confirm?
There was a problem hiding this comment.
In Elasticsearch, any field that can take a single value can take multiple values. A keyword field can have a single string or multiple strings in it.
ECS says some field must be arrays, but otherwise it's open, and we often put multiple values, or even a single value in an array when we expect multiple.
Users of _source will see this difference and will sometimes need to handle both single and array values, but most documents will usually have one or the other in a given field.
There was a problem hiding this comment.
Got it, so if I were to give a contrived example document:
user:
group:
name:
- first_group
- second_group
id:
- id_1
- id_2 Is it correct to say we should expect the values in the document to correlate to one another through their position in the arrays?
In the example above, that means first_group has an id of id_1, and second_group has an id of id_2?
There was a problem hiding this comment.
Where it gets confusing is in export of documents that use our interpretation of ECS which as Chris says is relaxed; other implementations based on ECS do not allow vector values in scalar fields. In this situation, because we can, and from a security perspective (as you note) it is needed, I've made them vectors.
The ECS definition of these fields is a little underwhelming; there is no indication that the fields are scalar, but this is because the convention for indication of scalar v vector fields in the docs is by omission (vector fields are marked with the text "This field should contain an array of values" while scalar fields have nothing.) Because of this, in the ES ecosystem we see quite a few fields raised from scalar to vector on the basis of "that which is not forbidden is allowed". This is unfortunate in one sense, but happy in many others.
There was a problem hiding this comment.
@jaredburgettelastic For your question with the example, that depends on the type of the user.group field which in ECS is a group type. This means that there is no correlation between user.group.name and user.group.id in the example you gave. The order is retained so some correlation can be obtained, but it's not apparent when you only use a query.
There was a problem hiding this comment.
👍 Thank you for the explanation. From my perspective, I think this is appropriate. @tiansivive do you have any concerns?
There was a problem hiding this comment.
nope, and loads of TIL on this one! Thanks a lot!
| if (ctx.user.group.name == null) { | ||
| ctx.user.group.name = new HashSet(); | ||
| } |
There was a problem hiding this comment.
Can use the ctx.user.group.name = ctx.user.group.name ?: [:]; syntax as above.
There was a problem hiding this comment.
That's really interesting. This was a case of fast thinking blindness (Kahneman). Thanks.
🚀 Benchmarks reportTo see the full report comment with |
💚 Build Succeeded
History
cc @efd6 |
|
|
Package entityanalytics_ad - 0.13.0 containing this change is available at https://epr.elastic.co/package/entityanalytics_ad/0.13.0/ |
Proposed commit message
See title.
Checklist
changelog.ymlfile.Author's Checklist
How to test this PR locally
Related issues
user.groupfields #13511Screenshots