Understanding SIEM logs

What is a SIEM?

SIEM stands for Security Information and Event Management. A SIEM brings together logs from devices, servers, firewalls, applications and identity systems so analysts can search and investigate activity in one place. Microsoft Sentinel is the SIEM used in this lab.

EDR focuses on endpoint activity. A SIEM can bring in information from other systems too, helping analysts connect sign-ins, processes and network activity. It can only show the data that has been collected.

Windows Event Logs and Sysmon

Windows Event Log records system and application events, such as sign-ins, service activity and errors.

Sysmon is a monitoring tool that adds more detailed endpoint events to Windows Event Log. Depending on its configuration, it can record process creation, command lines, file hashes and network connections.

Windows event log files commonly use the .evtx format. An agent or log forwarder can send the collected events to the SIEM, allowing analysts to investigate them centrally.

Tables, schemas and queries

Microsoft Sentinel stores collected logs in Log Analytics tables. These terms help explain how analysts search them:

Table

A collection of records organised into rows and columns. Different tables hold different types of log data.

Schema

The structure of a table: its field names and the types of values they hold.

Query

A request for information, such as finding events for a particular account.

KQL

Kusto Query Language, used to search, filter and analyse the logs.

SecurityEvent
| where Account == "admin"
| count

This example illustrates the query structure. Use the table, fields and account value that match your collected data.

Why parsing matters

Raw log data can contain several details inside one XML field. XML is a text format that uses tags to organise information.

Parsing means reading that data and extracting useful information into separate searchable fields, such as the user, process and command line. This makes the data easier to search and analyse.

<Data Name="User">SYSTEM</Data>
<Data Name="Image">powershell.exe</Data>

Normalisation and ASIM

Different log sources may use different field names for the same information. For example, one might use Image for an executable’s name, while another uses ProcessName. Normalisation puts equivalent information into a consistent format so it is easier to search and compare.

ASIM stands for Advanced Security Information Model. In Microsoft Sentinel, ASIM parsers extract useful fields and organise them into standard schemas. In this lab, they helped make Sysmon logs easier to investigate.

Some parsers work when a query runs, leaving the raw logs unchanged. Sentinel also supports normalisation as data is ingested. The approach depends on the data source and how the monitoring is set up.

Investigation, hunting and detections

A SIEM supports several related activities:

Investigation

Examining an alert or concern to understand what happened and whether action is needed.

Threat hunting

Proactively searching for suspicious activity instead of relying only on alerts. It can take place before, during or after an incident.

Detection rules

Automatic checks that generate alerts when defined suspicious patterns are found.

Correlation

Connecting related events, such as a sign-in, process launch and network connection.

Detection engineering

Creating, testing, tuning and improving detection rules.

A threat hunt may uncover a pattern that becomes a detection rule. The hunt explores the activity; the rule then checks for the defined pattern automatically.