Build and Run Logstash Forwarder on a Rasperry Pi

The Logstash Forwarder project is designed to be run on small systems that don’t have alot of resources, and where the logs need to be sent somewhere else to be stored and analyzed anyways. This description matches Raspberry Pis pretty much perfectly: they’re tiny, and if you have alot of them you probably want a central place to store their logs anyways.

In this post I’ll show you how to install logstash forwarder on your Raspberry Pi. After that you’ll just need a little bit of configuration and you’ll be off to the races!

Logstash Forwarder has been replaced by Beats. See my guide for building and running beats on the Raspberry Pi here.

Install Go

First, we’ll need the latest version of go. If you’re Pi doesn’t have Go version 1.4.1 or higher, you’ll need to get a newer version. At the time of writing the version of go in the package repository was too old, so we’ll need to compile from source. Grab the latest sources from the Go Downloads. I’ll be using 1.4.2 below, but you should update the wget link with the link of the version you plan on using. The below script will remove any existing go install, so be careful if you want another go version for other things.

which go && sudo rm -rf /usr/bin/go* /usr/local/go
cd ~
wget https://storage.googleapis.com/golang/go1.4.2.src.tar.gz
sudo tar -C /usr/local -xvf go*.tar.gz
cd /usr/local/go/src
sudo ./make.bash
sudo ln -s /usr/local/go/bin/* /usr/bin

Note: This won’t run all of Go’s tests, which are probably a good idea to run. They also took a solid hour to run on mine, so I figured that for most people it was worth skipping. If you want to run the tests, or run into problems with your install substitute the sudo ./make.bash line with sudo ./all.bash to build go and run the tests.

Sit back and relax, compiling go will take quite a while. If you run into problems building it, check out the Go source installation instructions.

Now that we have a working go install, we need to build logstash forwarder. Below is how I did it, but they document the process on their GitHub Readme.

Build Logstash Forwarder

Grab the Logstash Forwarder sources, and build with go:

git clone git://github.com/elasticsearch/logstash-forwarder.git
cd logstash-forwarder
go build -o logstash-forwarder

Package logstash forwarder (optional, but especially useful if you plan on copying this to other Raspberry Pis). If you don’t already have ruby (ie which gem shows nothing), you’ll need to install it:

[[ ! `which gem` ]] && curl -L https://get.rvm.io | bash -s stable --ruby

Now bundle it:

sudo gem install bundler
bundle install
make deb

Now install the bundle:

sudo dpkg -i logstash-forwarder*.deb

And there you are! You should be able to go about configuring Logstash Forwarder to, well, forward your logs. Check out the instructions on the Logstash-Forwarder GitHub page to configure it, or this useful tutorial. Once you’ve applied the configuration, just restart Logstash Forwarder with the following command:

sudo service logstash-forwarder restart

The nice thing with having built the .deb package is that you can now just copy the .deb package to another machine, run the dpkg install command, and not have to go through all the trouble with Go and Ruby! Nice!

, ,

Trackbacks/Pingbacks

  1. Everything generates data: Capturing WIFI anonymous traffic using Raspberry Pi and WSO2 BAM (Part II) | Holistic Security and Technology - February 4, 2016

    […] Logstash is a set of tools to collect heterogeneous type of data and It’s to used with Elasticsearch, It requires Java and for this reason It is too heavy to run in a Raspberry Pi. The best choice is to use only Logstash Forwarder. Logstash Forwarder (a.k.a. lumberjack) is the protocol used to ship, parse and collect streams or log-events when using ELK. Logstash Forwarder can be downloaded and compiled using the Go compiler on your Raspberry Pi, for further information you can use this link. […]

Leave a Reply

Proudly made in Canada