Graylog Extractors for pfSense 2.2 filter logs

Hi all,

I’m trying out Graylog for log collection, aggregation, and analysis. It’s free and pretty damn easy to deploy, available in OVA format.

The first thing I noticed is there seemed to be no extractors for pfSense 2.2’s new log format. Extractors allow you to parse a syslog message and place certain values into ‘fields’ for analysis or use in graphs.

Here’s one I prepared relatively quickly. You can import by:

  1. Click System -> Inputs in the Graylog UI
  2. Click ‘Manage extractors’ next to the relevant input
  3. Click ‘Import extractors’ in the ‘Actions’ menu at the top right of the page
  4. Paste the below script into the window and then click ‘Add extractors to input’

The extractors will parse the following fields out of the pfSense 2.2 filterlog messages:

  • Rule number into pfsense_filter_rulenum
  • Direction into pfsense_filter_direction
  • Ingress interface into pfsense_filter_ingress
  • Action into pfsense_filter_action
  • Protocol into pfsense_filter_proto
  • Source IP into pfsense_filter_sourceip
  • Source Port into pfsense_filter_sourceport
  • Destination IP into pfsense_filter_destip
  • Destination Port into pfsense_filter_destport

Right now they only interpret IPv4 logs, IPv6 log entries don’t get parsed (thanks to the condition regex) as they are formatted differently.

The script is available here, or click ‘Continue Reading’.

Hope this helps!

Nathan Manzi

I'm a geek, a husband, and a dad. I get excited when the word 'virtualization' is mentioned, and I spend most of my time working in that space. I also like to play computer games when I'm not playing with my kids, running, lifting heavy things, or studying.

  • Kyle

    I seem to be getting an error with the pfSense – Rule Number portion on import. Any suggestions? All of the other extractors seem to import fine. Currently on GrayLog v1.2.2 with pfSense v2.2.6.

    • Kyle

      Nevermind! Upgraded to Graylog v1.3 and it imported all the rules fine. Thanks for posting this!

  • Florian Jerusalem

    Great extractor. Just one thing to mention:
    I am using graylog for several pfsense instances.
    Somehow the extraction of the source-field seems to be not right:
    Keeps putting pfsense-logs into source “filterlog”, “charon”, “cron”, etc. – the matching does not seem to work right here.
    Any chance of rewriting the source-field here to match the gl2_remote_ip-field for example?

  • LiteBit

    I’ve the same issue as Florian
    I’m sending all logging from my pfsense host to graylog (so not just the filter log)
    it depends on which input you use :
    syslog UDP
    raw UDP

    using raw udp, I see as source the ip of the pfsense host but the extractors don’t work (because filterlog is not the first string in the message)

    using syslog udp: the extractors work, but we see “filterlog” as source, not the ipaddress/hostname of the pfsense host

    so I guess we need to decide which input type to use, then adapt the extractors to it


    • Andy Max

      LB, did you have any success in adapting the extractors?


  • Eugene Varnavsky

    I have same issue as LiteBit, in raw input my messages look like:

    Apr 5 15:07:17 filterlog: 7,16777216,,1000000101,em0,match,block,in

    so filterlog is not the first one and extractors don’t work.

    In syslog input source field is messed up.

  • Joseph Wagner

    Is there any hope of parsing IPv6 entries? Any ideas on what could be done to work around this?