– Bruno Kurtic, Vice President of Product Management at SenSage (www.sensage.com), says:
Why is getting a handle on log data and actually turning it into useful information a challenge?
Sophisticated, long term analysis of log data is the key to addressing emerging security threats, compliance mandates and a host of risk management initiatives. Why? Compliance mandates require firms to retain and analyze event data for up to seven years. As a result, nearly every organization is required to create secure, centralized log and event data repositories.
The first place most organizations turn to is legacy data management suppliers. However, traditional data management systems were built for transactional data – not event data. The requirements to manage event data are different:
• Data – Log and event data can never be updated or changed
• Collection – Difficult due to hundreds of data formats and dispersed endpoints
• Analysis – Data must be analyzed in real-time and over extremely long time frames
• Users – Typically few users but they need access to years of data
• Queries – Often ad hoc, time-sensitive and dispersed across huge data sets
• Volume – Enormous volumes of data creation and collection
What are the biggest issues with log management tools?
Security vendors espouse the benefits of legacy log management and SIEM tools to manage event data. Unfortunately, these point solutions don’t scale, are difficult to customize and often can’t address many of the emerging use cases of event data management.
On the other hand, traditional data management systems and data warehouses are designed for transactional data, not event data, which leads to dramatically higher costs and complexity. Some vendors still try to convince customers that the “one database, one data warehouse” approach is the correct one, forcing them to over spend and endure extremely long implementation cycles.
The bottom line – IT and data center managers need a flexible enterprise log and event data management approach that provides true business intelligence, offers rapid delivery on queries and can scale on-demand.
How can data center/IT managers overcome these issues?
All of these issues point to the need for a data warehouse that is specifically designed for the unique requirements presented by event data, such as log files. SenSage log management software provides automated collection, storage, correlation and reporting to allow organizations to effectively monitor, report and investigate activity and events from thousands of different log sources throughout the enterprise.
SenSage log management functionality enables organizations to monitor end-users as well as administrators for the purpose of detecting suspicious behavior and intrusion attempts, establish audit trails for change control, enforce accountability over administrators and conduct better investigations and forensic analysis.
SenSage log management technology is based on a patented columnar database architecture approach for event data. Unlike traditional relational database management systems that use a row format, data is organized by column in a single, centralized data repository specifically designed for event data. While the difference may sound trivial, the performance gains are dramatic. Indexes are unnecessary as each column is actually an index, reducing storage and maintenance requirements.
Data is compressed at a 40:1 advantage vs. relational databases and stored in a hierarchical series of folders and flat files on each node’s local disk. The SenSage Event Data Warehouse easily scales by adding new nodes and takes advantage of new hardware features, such as multi-core processors and faster local drives. To maintain constant availability, backup copies of each node’s data are stored on another node for data redundancy and automatic failover. With SenSage, organizations can easily query years of data from multiple sources at any detail level to support their business requirements.
SenSage provides an intuitive graphical user interface that makes it easy for business analysts and executive users to create new business-specific rules, generate reports and analyze event data efficiently. In addition, SenSage provides powerful real-time engine as well, making the process of gleaning actionable intelligence from the reams of event data even faster and more cost-effective.
