Adding a new field in logstash

Elastic Stack Logstash
1

Hi,

I have this input file sending data to logstash and want to add a new field name as seen below
I dont want to use
filter {
mutate {
add_field => { "new_field_name" => "new_field_value" }
}
}
such as
add_field => { "name " => "GigabitEthernet0/0" }
as this can change to GigabitEthernet0/1 or 2
e.g
Is there any way

``openconfig-interfaces:interfaces/interface/state/counters,host=ibcinmnrffd1v,name=GigabitEthernet0/0,path=openconfig-interfaces:interfaces/interface/state/counters,source=hclab043-gnmic,subscription=140 in_octets=3522806333i 1753175837837000000
openconfig-interfaces:interfaces/interface/state/counters,host=ibcinmnrffd1v,name=GigabitEthernet0/0,path=openconfig-interfaces:interfaces/interface/state/counters,source=hclab043-gnmic,subscription=140 in_octets=3522829569i 1753175867837000000

2

It's maybe obvious to you what you want to do here, but all you've really written is what you don't want to do?

I am presuming we are talking about only the name=GigabitEthernet0/0 bits?

Whats the range of values here, and what part of it do you want to distinguish? Is it always GigabitEthernetX/Y, X and Y always being positive integers, and both X and Y are always present? And you want to just drop the "/Y"? Or you want to drop "X/Y"?

e.g. what is the value for the name field that you desire for following inputs:

name=GigabitEthernet0/0
name=GigabitEthernet0/1
name=GigabitEthernet0/2
name=GigabitEthernet1/0
name=GigabitEthernet1/1
name=GigabitEthernet23/4
name=GigabitEthernet4/23
name=GigabitEthernet2
name=SomethingElse0/0
name=SomethingElse0/1
name=SomethingElse1/0
name=SomethingElse0
...
3

Maybe I should explain

I want where I can have any interface or maybe Gi0/0,0/1 having different in-octets and how this will be in kibana

I have this working with this config

image001.png988×514 11.6 KB

And output is like this

image002.png1564×739 82.1 KB

4

thanks for quick answer. Sadly I'm now even more confused than I was before, so I hope someone else is able to interpret your problem from the 2 descriptions, and can help you reach your goal.

5

Hello @DOkuwa

If i understand this below is your sample message/log lines :

message :

openconfig-interfaces:interfaces/interface/state/counters,host=ibcinmnrffd1v,name=GigabitEthernet0/0,path=openconfig-interfaces:interfaces/interface/state/counters,source=hclab043-gnmic,subscription=140 in_octets=3522806333i 1753175837837000000
openconfig-interfaces:interfaces/interface/state/counters,host=ibcinmnrffd1v,name=GigabitEthernet0/0,path=openconfig-interfaces:interfaces/interface/state/counters,source=hclab043-gnmic,subscription=140 in_octets=3522829569i 1753175867837000000

In current scenario you are adding a static field where name is "interface" & value is "GigabitEthernet0/0" for all records

add_field => { "interface" => "GigabitEthernet0/0" }

Currently in kibana for each record this value is added :

{
  "other_fields" : "values",
  "interface": "GigabitEthernet0/0",  // Static value added to every record
  "message": "openconfig-interfaces:interfaces/interface/state/counters,host=ibcinmnrffd1v,name=GigabitEthernet0/0,path=openconfig-interfaces:interfaces/interface/state/counters,source=hclab043-gnmic,subscription=140 in_octets=3522806333i 1753175837837000000"
}

But you do not want this to be static as the value will change & this should be extracted from the message/log file where field is "name"

name=GigabitEthernet0/0

I am not sure if you are looking for below where it will extract the data dynamically :

input {
  tcp {
    port => 5085
    codec => json {}
  }
}
filter {
   mutate {
    remove_field => ["_tags"]
    remove_field => ["tags"]
    remove_field => ["timestamp"]
    add_field => { "source" => "hclab043.zz.db.com" }
  }
  # Extract the `interface` value dynamically from the `message` field
  grok {
    match => { "message" => ".*name=%{DATA:interface},.*" }
  }
    mutate {
    rename => { "[fields][in_octets]" => "[in_octets]" }
  }
}

For sample below record in log file , i see this entry in Kibana

openconfig-interfaces:interfaces/interface/state/counters,host=ibcinmnrffd1v,name=GigabitEthernet2/1,path=openconfig-interfaces:interfaces/interface/state/counters,source=hclab043-gnmic,subscription=140 in_octets=3522806333i 1753175837837000000

image

Thanks!!