Introduction

Where are we working? Reminder about the purpose of a SOC, a CERT, and a Security Information and Event Manager (SIEM)

The main purpose of a Security Operation Center (SOC) is to provide an in-depth defense… because it doesn’t matter how much money or time you’ll spend in your firewall-IDS/IPS-antivirus-anti-APT-super-sandbox architecture, some attacks will break through. Hence the need of:

1. Setting up mechanisms of :

• – Correlating events, in order to detect abnormalities, regarding known-attacks patterns as well as context-related rules.
• – Supervising and triggering alerts (followed by human analysis).
• – Detecting and responding to incidents “the industrial way”, that is to say an as standardized and rigorous way as possible.

2. A Security Operations Center (SOC) and a Computer Emergency Response Team (CERT), the two entities being likely to work together, (when not blended).

Security investigations with Splunk

Splunk and Security Operations

Splunk and supervision get along well, because Splunk has just been designed to collect, parse, index, and search logs, which many other SIEMs perform laboriously. In this presentation, we will take the example of this SIEM, but you can apply what follows to (some) other SIEMs, like QRadar.

Splunk functionning – overview

Brief explanations about Splunk architecture:

• – The core of Splunk is a C daemon. It’s fast, efficient, and stable.
• – The web interface uses the django framework. It’s nice and easy to browse.
• – Developers will be interested in SplunkJS, a Javascript stack following a full Model-View-Controller pattern, and above all in the REST API Splunk provides.
• – Splunk has a LOT of configurations files. It takes some time if you want to understand them, but it’s worth struggling, as you’ll be able to search data more efficiently by understanding how Splunk is configured.
• – The “data pipeline”: Splunk gets data, parse them, “licence meter” them, and index them.
  • – Some basic field extraction takes place at index time.
  • – A far more complex and configurable extraction takes place at search time.

The Splunk data pipeline

Using Splunk demands a fair amount of time understanding the tool, although the benefits of being able to write your own regex, requests, develop your own dashboards, apps… are pretty important.

Apps and Technology Add-ons (TA)

Splunk is extensible, because it comes with a lot of modules, called apps and technology add-ons: from parsers and dashboards to SIEM capacities, there are a lot of (free or not) functions in these modules.

For instance, you can download free Unix, Windows, or Fireeye apps, but you will have to pay to get a full-fledged Security Enterprise app.

Before treating it : detecting spam

Let’s enter the cybersecurity analyst’s job, now.

Detecting spam is hard, because nefarious malwares are constantly evolving, varying, and using incredibly well-developed botnets. When facing a spam campaign, one may see one different server, mail address, or IP address per mail, automatically generated attachment names and titles, unknown-by-virus-total hashes…

Let’s consider some humble detection techniques, then.

• – By key-words in (decoded) mail subjects and attachment types or names.
  • – May sound too simple… and it is. But is in our humble opinion one efficient way of detecting massive spam campaigns.
  • – Furthermore, some campaigns use macros to generate the titles and attachment names of the mails. Lexsi wrote a good article on that.
  • – This kind of detection is easily automated by sending alerts to analysts
• – Detected by the final user.
  • – Well, this is the less deterministic way… unfortunately not very useful in real life, unless you watching the security team (and event then, you could have suprises…).
• – Detect mail attachments signatures with suricata:

Suricata alerting

• • – Not very efficient, as are all signature-based detection products… even with Emerging Threat Pro Rules.
  • – Can nonetheless be improved by writing custom rules and modules. But it requires some skills in understanding how Suricata works.
  • – May easily be automated by sending alerts to analysts
• – Detect payload with network or endpoint antivirus
  

  Mail platform protection alerting

  • – Signature-based, not very efficient, then.
  • – May be easily automated as well.
• – Extract attachments and URLs contained in the message body automatically, and analyse them:
  • – Save a lot of time, but pretty complicated to be made industrial (we’ll come back on that later).
  • – The most interesting track to follow !

Analysing spam

Once you’ve detected potentially malicious emails, you have to understand which threat it is bound to (Cridex/Dridex, some evolved nation-state threat, a generic spear phishing link…?)

Let’s analyse a practical incident response scenario, and discuss practical means of responding to spam.

At first: get an alert

By any of the the means above, you should be able to get an alert, when undesired mails enter your infrastructure.

If you logged your security devices alerts, and correlate them, you could get an alert like the following:

Spam alert mail example

Analysts can implement basic analysis in the alert correlation, ttypically: security devices chaining, whois information, or correlation with threat intelligence feeds.

Once you’ve received this alert, you want to doscover more about the spam campaign: which are the originating IPs and mail addresses? Is it massive or targeted? Have we missed previous mails?

In Splunk: get the mails !

Once in Splunk, you will be able to look for more context: particular requests or dashboards may be helpful:

A mail analysis dashboard in Splunk

In the dashboard above, you may look for the spam sender or recipients, the subject or attachment name patterns.

It is worth mentioning that aggregating logs from smtp gateways isn’t an easy task. Let’s take the example of Cisco’s Ironports: you need to understand how the MID, ICID, DCID are related, and more globally the security devices relations.

Splunk will help you consolidate data from several indexes in one final index, which will save you a huge amount of time.

On the screenshot below, you can see how we are able to find all the campaign-related mails from the dashboard:

Recovering all campaign’s mails

Get the indicators of compromise!

This is what you have to do as fast as possible, because you have to:

1. Block malicious sites distributing the dropper / final payload / infecting the poor users browser with a 0-day flash exploit.
2. Send malwares to you security editors, in order to make your infrastructure immune to the payload within a few days.
3. Optionally, follow which users have clicked / answered to phishing mail (not only to slash them, but to educate them J).

Malicious URLs

In the following example, we gathered directly in the logs the nasty URL, which allows us to have it blocked within a few minutes (and then save a lot of malware spreading / desktop re-imaging / user bashing).

Seeing malicious URL directly in the logs

Malicious attachments

Getting nefarious attachments, for instance the last office document with macro you received, and analyzing them automatically, is pretty difficult, unless you buy a dedicated (and expansive) equipment.

Let’s see both possibilities (with and without a dedicated equipement).

Scenario 1 – you’re poor:

• – Either ask the user directly to send you the malicious attachment.
  • Slow, might give you headaches if the users don’t understand what you want from them, sometimes impossible if you’re not able / allowed to contact the user directly.
  • Impossible to make industrial !
• – … or get the pcap containing the spam.
  • Investigate it with something like :

tcpdump -vv -lnA -r <mysuperpcap.cap> | grep <mysuperstringORregex> –color=auto

Or:

tcpdump -qAn -r <mysuperpcap.cap> host a.b.c.d -w ~/mail.pcap

… followed by a wireshark analysis.

Once you’ve the pcap, you can extract automatically the TCP session that encapsulates the SMTP stream you want, for instance with this trivial script:

Session extraction script

You may then:

• – Extract the base64-encoded payload.
• – Decode it.
• – Analyse it! Let’s take the (frequent, too frequent) example of a .doc document with a malicious embedded macro: you may retrieve and deobfuscate the macro using the gorgeous tool oletools, developed by Didier Stevens:

Embedded macro – part 1

[…]

Embedded macro – part 2

As we wrote above, and as you can see on Figure 10, analyzing the macro lets you:

• – Submit the malwares to Virus Total, and to your antivirus editors
• – Block / investigate the connections to or from the malicious IPs / domains
• – Check the presence of the malware’s Indicators Of Compromise in your infrastructure.

Scenario 2 – you are rich:

• – Buy a product dedicated to detect Advanced Persistent Threats, which comprises analyzing mail attachments and the URLs embedded in the message’s body as well.
  • – We tested a lot of so-called Advanced Malware Protection Appliances… and I can say that these products are not equals! Some are very mature, some are just a marketing try…!

This is what you get for instance with FireEye EX:

Viewing detailed malicious attachment analysis

Note that it’s far faster to get the attachment and its IoC with such an equipment. And of course, you’ll have an evolved analysis, thanks to the sandbox, which you can’t get with free sandboxes (but perhaps you don’t need it, or you don’t have the human resources to use such an analysis).

• – Correlate mail and web threats: what about a flash exploit following the visit of a waterhole? This is one other reason why anti-APT appliances may be useful, and you shoul take advantage of interfacing it with other equipments like proxies, firewalls, your SIEM naturally…

And what about Threat Intelligence ?

Correlation is key to detect and fight back threats.

It is crucial to understand the context of a threat, and it is fundamental for more evolved (for instance behavioural) analysis.

This topic is a bit beyond the scope of this article, but it’s worth mentioning that you should build (although it’s absolutely not an easy task):

• – A “knowledge database”, aggregating data about the IPs you’ve seen attack the organization, which were the targets, the security devices and splunk alerts triggered…
  • – For instance, the few results below show an IP that triggered some suricata alerting:

An idea of a knowledge base

• – Data visualization tools, for:
  • – Stats, both for understanding global trends and showing off beyond your boss (yes).
  • – Saving (a lot) of time understanding the context of an incident

Splunk dashboard example for stats

Splunk dashboard example for context data

Conclusion: recommendations for preventing spam

Considering that a lot of users keep enjoying clicking everywhere, we recommend to:

• – Forbid the executables – and more generally all unneeded – extensions, including those in zip, gz, bz2, 7z, rar archives:
  • – If you need to exchange such files (for instance for development purposes), use an internal authenticated sharing space!
• – NOT make the users administrators of their machine. And strictly monitor those who are.
• – Disable automatic execution of macros in Office documents.
• – Blacklist certain categories of sites known to distribute corrupted exe, spywares, riddled-with-trojans ads… yes, this is difficult, and that’s why you need intelligence.
• – Set up a “trash” mailbox where users can forward undesired mails, and where analyst can have a look on a regular basis.
• – Have an incident response team able to quickly:
  • – Provide a list of URLs and IPs related to the spam to the network security team
  • – Submit the malwares to the security products editors your infrastructure use, in order to get your signatures database updated as soon as possible.
  • – Perform any additional analysis that may give clues to mitigate the spam.