Open Source Linux Log Management Tools

log4cplus is a simple to use C++ logging API providing thread-safe, flexible, and arbitrarily granular control over log management and configuration. It is modelled after the Java log4j API.

OSSIM, AlienVault’s Open Source Security Information and Event Management (SIEM) product, provides event collection, normalization and correlation. For more advanced functionality, AlienVault Unified Security Management (USM) builds on OSSIM with these additional capabilities: * Log management * Advanced threat detection with a continuously updated library of pre-built correlation rules * Actionable threat intelligence updates from AlienVault Labs Security Research Team * Rich analytics dashboards and data visualization

PM2 (Process Manager 2) is a production process manager with its own built-in load-balancer for Node.js applications. If you want battle-hardened Node.js applications that can be monitored and kept alive forever, PM2 is the way to go. PM2 lets you manage, maintain and increase Node.js performance. It is constantly assailed by over 1800 tests so you can deliver high quality applications, and deploy confidently and more often. PM2 is cross-platform and arguably the most widely-used process manager for Node.js with over 100 million downloads. Setting it up is easy, it just takes a couple of minutes and one simple command.

HyperDX helps engineers figure out why production is broken faster by centralizing and correlating logs, metrics, traces, exceptions and session replays in one place. An open-source and developer-friendly alternative to Datadog and New Relic. The HyperDX stack ingests, stores, and searches/graphs your telemetry data. After standing up the Docker Compose stack, you'll want to instrument your app to send data over to HyperDX.

syslog-ng is the log management solution that improves the performance of your SIEM solution by reducing the amount and improving the quality of data feeding your SIEM. With syslog-ng Store Box, you can find the answer. Search billions of logs in seconds using full text queries with Boolean operators to pinpoint critical logs. syslog-ng Store Box provides secure, tamper-proof storage and custom reporting to demonstrate compliance. syslog-ng can deliver data from a wide variety of sources to Hadoop, Elasticsearch, MongoDB, and Kafka as well as many others. syslog-ng flexibly routes log data from X sources to Y destinations. Instead of deploying multiple agents on hosts, organizations can unify their log data collection and management. syslog-ng Store Box provides automated archiving, tamper-proof encrypted storage, granular access controls to protect log data. The largest appliance can store up to 10TB of raw logs.

OpenObserve is a cloud-native observability platform built specifically for logs, metrics, traces, and analytics designed to work at a petabyte scale. It is very simple and easy to operate as opposed to Elasticsearch which requires a couple of dozen knobs to understand and tune which you can get up and running in under 2 minutes. It is a drop-in replacement for Elasticsearch if you are just ingesting data using APIs and searching using Kibana (Kibana is not supported nor required with OpenObserve. OpenObserve provides its own UI which does not require separate installation unlike Kibana). You can reduce your log storage costs by ~140x compared to Elasticsearch by using OpenObserve. Below are the results when we pushed logs from our production Kubernetes cluster to Elasticsearch and OpenObserve using fluent bit. OpenObserve stored data in Amazon s3 and Elasticsearch stored data on Amazon EBS volumes.

NXLog is a modular, multi-threaded, high-performance log management solution with multi-platform support. In concept it is similar to syslog-ng or rsyslog but is not limited to unix/syslog only. It can collect logs from files in various formats, receive logs from the network remotely over UDP, TCP or TLS/SSL . It supports platform specific sources such as the Windows Eventlog, Linux kernel logs, Android logs, local syslog etc. Writing and reading logs to/from databases is also supported. The collected logs can be stored into files, databases or forwarded to a remote log server using various protocols. The old BSD Syslog and the newer IETF syslog standard is fully supported by NXLog in addition to Snare, XML, JSON, GELF, KVP, CSV and custom formats. A key concept in NXLog is to be able to handle and preserve structured logs. No need to convert everything to syslog and parse these logs again at the other side. It has powerful message filtering, log rewrite and conversion capabilities.

New version 1.03 is released! Check it out. LogEngine is a C++ library for easy and flexible logging to files. Multithreaded, easy to use and fast. Automatic logfile backup (several modes). A small amount of code. Platform Independent.

LogEngine2 is a C++ header only library for easy and flexible logging. Contains small amount of code in compare with other logging libraries. Just copy the files from include folder to your project folder and use a C++20 compiler. LogEngine2 can load loggers from .lfg file. Examples of .lfg files are included into bundle. LogEngine2 provides various log targets: regular files, rotating log files, console logging, string stream, user custom function. Log line format is easy configurable.

Log Searcher provided full indexed search engine for log files on each servers. It supported many powerful query types such as phrase queries, wildcard queries, proximity queries, range queries and more with Lucene and Solr technology. It comes with Web interface to search and generate chart accordingly. This project is Open Source project with Apache 2.0 license for allowing you to extra development on top of Log Searcher project. The initial configuration was written on readme.txt before implementation on production line. If you have idea for improvement, you can drop an email to me. Contact: log.searcher[at]yahoo.com

Logs Analyzer, Alerter & Reporter with a Web Interface

IDS via ACL log management, including basic reporting via PHP