Skip to main content

Open Source System Logging Solutions

For large datacenters collecting of server logs is headache. There are some opensource solutions for this. Graylog2, logix, logstash, Facebook Scribe are popular ones.


Graylog2's first release is June 2010. Standart syslog pakhages are sent to the central log server. There is web tool for analyzing and searching for logs. Logs are stored in MongoDb database.

logix can be considered as a log sending tool for graylog. In standard ways, some logs may be lost while sending over network. Logix takes logs in a queue then send them to log server.

Logstash is a popular solution like graylog. Also there is a web interface for searching logs. Logstash can store logs in elasticsearch.

Scribe is written by facebook using C++ server. Facebook collects billions of logs from thousands of its servers.



Etiketler:

Comments

  1. AnonymousMay 5, 2013 at 10:34 AM

    I just discovered logstash myself and i'm very excited about it! Would you happen to know whether or a not a pluggin exists for parsing ConfigMgr logs? Else I can always RTM and make one :)

    ReplyDelete
  2. Kadir SertMay 5, 2013 at 4:36 PM

    I think you have to write a filter to achieve custom parsing.

    From the logstash point of view, a log consists of timestamp plus data. So if any log data including ConfigMgr logs which has this format may be collected using logstash process. The only thing you have to do is denote the log file path as input. But if ConfigMgr file format is more than this simple approach you have to write your own filter. Simple example for syslog is http://cookbook.logstash.net/recipes/syslog-pri/

    ReplyDelete

Post a Comment

Popular posts from this blog

Creating Multiple VLANs over Bonding Interfaces with Proper Routing on a Centos Linux Host

In this post, I am going to explain configuring multiple VLANs on a bond interface. First and foremost, I would like to describe the environment and give details of the infrastructure. The server has 4 Ethernet links to a layer 3 switch with names: enp3s0f0, enp3s0f1, enp4s0f0, enp4s0f1 There are two bond interfaces both configured as active-backup bond0, bond1 enp4s0f0 and enp4s0f1 interfaces are bonded as bond0. Bond0 is for making ssh connections and management only so corresponding switch ports are not configured in trunk mode. enp3s0f0 and enp3s0f1 interfaces are bonded as bond1. Bond1 is for data and corresponding switch ports are configured in trunk mode. Bond0 is the default gateway for the server and has IP address 10.1.10.11 Bond1 has three subinterfaces with VLAN 4, 36, 41. IP addresses are 10.1.3.11, 10.1.35.11, 10.1.40.11 respectively. Proper communication with other servers on the network we should use routing tables. There are three
Post a Comment
Read more

3 Node (Master Slave Slave) Redis Cluster with Sentinel

It is possible to make your Redis cluster Fault Tolerant and Highly Available by building a replica set and then monitor these nodes using sentinel for automatic failover. I am going to give an example setup to explain it. The structure is built with three nodes running one as a master and two as slaves. Master Node: (Centos 7.2) 192.168.1.11 Slave1 Node: (Centos 7.2) 192.168.1.12 Slave2 Node: (Centos 7.2) 192.168.1.13 Edit System settings on each node: /etc/sysctl.conf Disable transparent hugepage (transparent_hugepage=never) on each node: /etc/default/grub Apply grub config and reboot each node: Master Node: /etc/redis/6379.conf Slave1 Node: /etc/redis/6379.conf Slave2 Node: /etc/redis/6379.conf Master Node: /etc/redis/sentinel.conf Slave1 Node: /etc/redis/sentinel.conf Slave2 Node: /etc/redis/sentinel.conf Each Node: /etc/systemd/system/multi-user.target.wants/redis-server.service Each Node: /etc/
1 comment
Read more