Problems sending .json w filebeat

Flozkel · July 29, 2025, 2:49pm

Hi guys,

For a while I have been trying to setup IDS on my RPI5. I'm rather new on this and experimenting for a homelab. The RPI does run a bit slow when everything is on, but again, its only for training.
I have been searching this and various forums, bit I haven't been able to get it to work.

I run Suricata on the RPI and Elastic, Kibana and Filebeat in docker containers.
Suricvata seems to run fine and I have managed to ship fast.log to Elastic. but after a re-install I can get nothing but 6000 empty fiels in Kibana.
Suricata runs fine and so does Elastic, Kibana and Filebeat seem to do. But I must be missing something.

The Elastic setup is from this guide

Here my filebeat.yml, docker-compose.yml (Elastic, Kibana and Filebeat) and suricata.yaml as well as the docker log from the Filebeat container.

gist.github.com

https://gist.github.com/Jakob2212/abdabcb05ebb01665e713f8664c8aa9e

Filebeat.log

{"log.level":"info","@timestamp":"2025-07-15T12:23:39.847Z","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cmd/instance.(*Beat).configure","file.name":"instance/beat.go","file.line":1062},"message":"Home path: [/usr/share/filebeat] Config path: [/usr/share/filebeat] Data path: [/usr/share/filebeat/data] Logs path: [/usr/share/filebeat/logs]","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2025-07-15T12:23:39.848Z","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cmd/instance.(*Beat).configure","file.name":"instance/beat.go","file.line":1070},"message":"Beat ID: f24b0a80-79b4-43e6-aff6-6e9e8a59a171","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2025-07-15T12:23:39.863Z","log.logger":"seccomp","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/common/seccomp.loadFilter","file.name":"seccomp/seccomp.go","file.line":125},"message":"Syscall filter successfully installed","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2025-07-15T12:23:39.863Z","log.logger":"beat","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cmd/instance.(*Beat).createBeater","file.name":"instance/beat.go","file.line":570},"message":"Setup Beat: filebeat; Version: 8.17.5","service.name":"filebeat","ecs.version":"1.6.0"}
{"log.level":"info","@timestamp":"2025-07-15T12:23:39.863Z","log.logger":"beat","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cmd/instance.(*Beat).logSystemInfo","file.name":"instance/beat.go","file.line":1623},"message":"Beat info","service.name":"filebeat","system_info":{"beat":{"path":{"config":"/usr/share/filebeat","data":"/usr/share/filebeat/data","home":"/usr/share/filebeat","logs":"/usr/share/filebeat/logs"},"type":"filebeat","uuid":"f24b0a80-79b4-43e6-aff6-6e9e8a59a171"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2025-07-15T12:23:39.864Z","log.logger":"beat","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cmd/instance.(*Beat).logSystemInfo","file.name":"instance/beat.go","file.line":1632},"message":"Build info","service.name":"filebeat","system_info":{"build":{"commit":"1247ce9b70a6497c3c30278e79b08f374c00dfe9","libbeat":"8.17.5","time":"2025-04-09T17:14:07.000Z","version":"8.17.5"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2025-07-15T12:23:39.864Z","log.logger":"beat","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cmd/instance.(*Beat).logSystemInfo","file.name":"instance/beat.go","file.line":1635},"message":"Go runtime info","service.name":"filebeat","system_info":{"go":{"os":"linux","arch":"arm64","max_procs":4,"version":"go1.23.8"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2025-07-15T12:23:39.866Z","log.logger":"beat","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cmd/instance.(*Beat).logSystemInfo","file.name":"instance/beat.go","file.line":1641},"message":"Host info","service.name":"filebeat","system_info":{"host":{"architecture":"aarch64","native_architecture":"aarch64","boot_time":"2025-07-15T07:18:42Z","containerized":false,"name":"52d1c4e9bb01","ip":["127.0.0.1","172.19.0.5","::1"],"kernel_version":"6.12.25+rpt-rpi-v8","mac":["f2:62:d0:f1:49:ba"],"os":{"type":"linux","family":"debian","platform":"ubuntu","name":"Ubuntu","version":"20.04.6 LTS (Focal Fossa)","major":20,"minor":4,"patch":6,"codename":"focal"},"timezone":"UTC","timezone_offset_sec":0},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2025-07-15T12:23:39.867Z","log.logger":"beat","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/cmd/instance.(*Beat).logSystemInfo","file.name":"instance/beat.go","file.line":1670},"message":"Process info","service.name":"filebeat","system_info":{"process":{"capabilities":{"inheritable":null,"permitted":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"effective":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"bounding":["chown","dac_override","fowner","fsetid","kill","setgid","setuid","setpcap","net_bind_service","net_raw","sys_chroot","mknod","audit_write","setfcap"],"ambient":null},"cwd":"/usr/share/filebeat","exe":"/usr/share/filebeat/filebeat","name":"filebeat","pid":7,"ppid":1,"seccomp":{"mode":"filter","no_new_privs":true},"start_time":"2025-07-15T12:23:39.050Z"},"ecs.version":"1.6.0"}}
{"log.level":"info","@timestamp":"2025-07-15T12:23:39.872Z","log.logger":"esclientleg","log.origin":{"function":"github.com/elastic/beats/v7/libbeat/esleg/eslegclient.NewConnection","file.name":"eslegclient/connection.go","file.line":132},"message":"elasticsearch url: https://es01:9200","service.name":"filebeat","ecs.version":"1.6.0"}

This file has been truncated. show original

docker-compose.yml



volumes:
 certs:
   driver: local
 esdata01:
   driver: local
 kibanadata:
   driver: local
 filebeatdata01:

This file has been truncated. show original

filebeat.yml

filebeat.inputs: #added 18.7.2025
        - type: log
          enabled: true
          paths:
            - /media/Flozkel/66FB-A7F8/suricata_logs/eve.json
          multiline.type: pattern
          multiline.pattern: '^{"timestamp":'
          multiline.negate: true
          multiline.match: after

This file has been truncated. show original

There are more than three files. show original

Topic		Replies	Views
Filebeat setup encounters bad json, but doesn't tell me what the problem is? Beats filebeat	3	444	December 26, 2019
Unable to send input log to Elastic search via filebeat docker installation Beats filebeat	3	392	July 28, 2020
Filbeat not showing json outputs Beats filebeat	0	98	May 30, 2024
Filebeat is not sending Json data to Elasticsearch Beats filebeat	5	1768	November 9, 2017
Filebeat not sending JSON to file Beats filebeat	8	2163	May 14, 2018

Problems sending .json w filebeat

Related topics