Log Management and Intelligence

Log Management And Intelligence

Log management (LM) comprises an approach to dealing with large volumes of computer-generated log messages (also known as audit records, audit trails, event-logs, etc.). LM covers log collection, centralized aggregation, long-term retention, log analysis (in real-time and in bulk after storage) as well as log search and reporting.

Log management is driven by reasons of security, system and network operations (such as system or network administration) and regulatory compliance.

Effectively analyzing large volumes of diverse logs can pose many challenges — such as huge log-volumes (reaching hundreds of gigabytes of data per day for a large organization), log-format diversity, undocumented proprietary log-formats (that resist analysis) as well as the presence of false log records in some types of logs (such as intrusion-detection logs).

Users and potential users of LM can build their own log management and intelligence tools, assemble the functionality from various open-source components, or acquire (sub-)systems from commercial vendors. Log management is a complicated process and organizations often make mistakes while approaching it.