Graylog Extractors for pfSense 2.2 filter logs

Hi all,

I’m trying out Graylog for log collection, aggregation, and analysis. It’s free and pretty damn easy to deploy, available in OVA format.

The first thing I noticed is there seemed to be no extractors for pfSense 2.2’s new log format. Extractors allow you to parse a syslog message and place certain values into ‘fields’ for analysis or use in graphs.

Here’s one I prepared relatively quickly. You can import by:

  1. Click System -> Inputs in the Graylog UI
  2. Click ‘Manage extractors’ next to the relevant input
  3. Click ‘Import extractors’ in the ‘Actions’ menu at the top right of the page
  4. Paste the below script into the window and then click ‘Add extractors to input’

The extractors will parse the following fields out of the pfSense 2.2 filterlog messages:

  • Rule number into pfsense_filter_rulenum
  • Direction into pfsense_filter_direction
  • Ingress interface into pfsense_filter_ingress
  • Action into pfsense_filter_action
  • Protocol into pfsense_filter_proto
  • Source IP into pfsense_filter_sourceip
  • Source Port into pfsense_filter_sourceport
  • Destination IP into pfsense_filter_destip
  • Destination Port into pfsense_filter_destport

Right now they only interpret IPv4 logs, IPv6 log entries don’t get parsed (thanks to the condition regex) as they are formatted differently.

The script is available here, or click ‘Continue Reading’.

Hope this helps!

(more…)

Ruby 1.9, Rails 2.3, and MySQL on Ubuntu 8.10 (Intrepid Ibex)

I haven’t the time to write out a big fleshy post atm, so I’ll post the short story.

I was trying to set my Ubuntu 8.10 VPS up with a fresh Ruby/Rails install to host some webapps I’m working on. Rather than apt-getting my way to glory, I decided to build Ruby 1.9.1 from source, the main reason being that the version available through apt is only 1.9.0 and I’m working with 1.9.1 on my dev machine.

After building/installing Ruby, I started installing rails, rake, rack, etc… via RubyGems. I knew that I couldn’t install the standard ‘mysql’ gem, as it hasn’t yet been updated for Ruby 1.9, so I added http://gems.github.com/ to my gem sources – if you don’t know how to do this, the command is:

gem sources -a http://gems.github.com

…and proceeded to install kwatch’s mysql-ruby gem.

To my surprise (as it had worked under OSX on my dev box), I got the following message:

plasma@syn-app01:~$ sudo gem install kwatch-mysql-ruby
Building native extensions. This could take a while...
ERROR: Error installing kwatch-mysql-ruby:
ERROR: Failed to build gem native extension.

/usr/local/bin/ruby extconf.rb
Trying to detect MySQL configuration with mysql_config command...
Succeeded to detect MySQL configuration with mysql_config command.
checking for mysql_ssl_set()... yes
checking for rb_str_set_len()... yes
checking for rb_thread_start_timer()... no
checking for mysql.h... yes
creating Makefile

make
gcc -I. -I/usr/local/include/ruby-1.9.1/i686-linux -I/usr/local/include/ruby-1.9.1/ruby/backward -I/usr/local/include/ruby-1.9.1 -I. -DHAVE_MYSQL_SSL_SET -DHAVE_RB_STR_SET_LEN -DHAVE_MYSQL_H -D_FILE_OFFSET_BITS=64 -I/usr/include/mysql -DBIG_JOINS=1 -fPIC -fPIC -O2 -g -Wall -Wno-parentheses -fPIC -o mysql.o -c mysql.c
In file included from /usr/include/stdlib.h:320,
from /usr/local/include/ruby-1.9.1/ruby/ruby.h:50,
from /usr/local/include/ruby-1.9.1/ruby.h:32,
from mysql.c:6:
/usr/include/sys/types.h:151: error: duplicate 'unsigned'
make: *** [mysql.o] Error 1

Gem files will remain installed in /usr/local/lib/ruby/gems/1.9.1/gems/kwatch-mysql-ruby-2.8.1 for inspection.
Results logged to /usr/local/lib/ruby/gems/1.9.1/gems/kwatch-mysql-ruby-2.8.1/ext/gem_make.out

Instructions on fixing the issue and installing the gem after the jump.

(more…)

Fink Troubles – Cannot perform symlink test

So, I’m trying to start developing synflare.com, using my MacBook and OSX as a development platform.

I need GD for PHP, and I can’t be screwed compiling it from source (that’s why I’ve also moved from Gentoo to Ubuntu. Binaries make life easier). Fink is a brilliant service that provides an almost ‘apt’ way of getting and installing software. With Fink, I can issue a single command in terminal and have GD install itself.

My problem, however, was that Fink wasn’t installing. At the install volume selection screen, I wasn’t able to select my root volume. It’s reasoning was: “You cannot install Fink on this volume. Cannot perform symlink test on this volume because of a permissions problem. Try performing the “Repair Disk Permissions” function in Disk Utility”.

After several Verify and Repair permissions commands, I still had no joy. I noticed that the Disk Utility log was spouting the line: “ACL present but not expected for…”. After some investigation, those lines are merely informational. As Fink still wasn’t installing, I decided to fix them by running ‘chmod -a# 0’ on the directories affected. This still didn’t help! I was at breaking point.

I decided to fix it my way (read: quick and simple).

Entering the Fink Installer Package (right-click, Show Package Contents – in Finder), I could see three scripts in the Resources folder, one of which was VolumeCheck, which basically tells the Installer if you have permissions to the Volume in question. Editing this script, I made sure it did nothing but return an exit-code of ‘0’ back to the installer.

Hey-presto, it works, and there’s no noticable issues with Fink. GD installs perfectly.

In a nutshell:

  1. Copy the Fink Installer Package out of the DMG and into your home folder.
  2. Go into the folder where you copied the package, right click on the package and click ‘Show Package Contents”
  3. Navigate to the Resources directory, which resides inside the Contents directory.
  4. Delete the existing VolumeCheck script.
  5. Download this file – volumecheck – and extract it into the Resources directory in the package.
  6. Run the installer!

If you have this problem, and this fix works for you, be sure to post a comment – I’m interested to see how many people this happens to.

– NM

CTCP request handling in Colloquy

Back in the day, when mIRC and NoNameScript were my friends, I used to have a CTCP trigger set up that gave people DCC leeching from me the ability to resume transfers if they disconnected for some reason, amongst other things.

Now that I’m all Mac, I’ve been using Colloquy, and haven’t started looking at scripting for it – that is, until today.

Austnet are now blocking all DCC by default, and the only way to allow someone to send you a file is to issue the user command /dccallow +User <timeout> – adding them to a temporary Allow list. This, of course, only works if your nick is registered and you have identified with /nickop.

In order to save time and allow for the receiving of DCC file sends while I’m AFK, I did some research and found that Colloquy Plugins are the method of choice for handling CTCP requests. Plugins can be created in a variety of programming languages, like Obj-C, Applescript, F-Script, Javascript, Python and Ruby. I decided to do mine in Applescript, mainly because there’s a lot more support for Applescript plugins over the others.

Place this script in your ~/Library/Application Support/Colloquy/Plugins directory, then issue a /reload plugins in Colloquy if it’s already open. People can then use /ctcp <Your Username> DCCALLOW to allow themselves DCC access to your username for 300 seconds. Additionally, if you want to auto-accept DCC requests from strangers, you’ll need to modify your Colloquy settings to allow it.

– NM