Security Analytics Part 3: Selecting the Right Analytics Platform

This is the final part of our three-part series on Security Analytics. You can view Part 1 and Part 2 here.

In Part 1 and 2 of this series, we looked at the technology behind Security Analytics (SA) and where these technologies are going.  In this final part, we will address how to select a SA platform for your business.

Questions for Evaluating Security Analytics

Like any complex technology, SA can create more questions than it answers. Before you invite the vendors in for the hard-sell, you should answer some important questions first.

1. Do You Need Security Analytics?

It is a simple question, but the most important question you can ask yourself about any new technology. Be careful not to fall into some of these faulty justifications for an SA platform.

• “We need it for regulatory compliance.”
  There are no regulatory requirements for SA technologies.  PCI, HIPAA, and such do not require anything as powerful as SA.  The only thing PCI requires is some kind of logging and alerting, which most SIEM products can do just fine.  You cannot, legitimately, use regulations as a primary driver for SA.
• “We don’t know what we don’t know.”
  A fine answer, but if you do not know what is going on in your environment, security analytics is not necessarily going to tell you.  SA platforms are “high-maturity” technologies.  You must have a very mature information security team and program before you can make SA work.  SA cannot compensate for the lack of fundamental security controls like patch management, change control, IDS/IPS, or secure authentication.  These are sexy products with tremendous possibilities, but they are not for immature companies or inexperienced security practitioners.
• “We are concerned about APT and/or state-sponsored attacks.”
  SA platforms can help spot these types of attacks if you can actually leverage to capabilities of the solution(s).  However, there are probably a lot of other security controls you need before investing in a complex SA solution.  So far, the data shows that state-sponsored attackers typically exploit well-known vulnerabilities or trusted third-parties.  Therefore, a strong vendor and vulnerability management program may have more impact on stopping state-sponsored attacks than an expansive SA implementation. Lastly, you need to fess up to the real risk of state-sponsored attacks in your organization. Unfortunately, some security practitioners exaggerate the risk of state-sponsored attacks to create drama and inflate their value to the organization.
• “We need to know exactly what happened after an attack.”
  This is the best of all the answers. Post-incident forensics is probably the strongest case for SA products.  Since most of these products are “after-the-fact” analyzers, they can be very useful to track down the origin and scope of an attack.  However, this means you need to have strong forensics capabilities either inside your company or available from a trusted third party. Also, you should seriously consider adding sandboxing to your UTM/NGFW appliance if you are considering products like FireEye or Damballa.  If you have a Palo Alto or Fortinet appliance at your perimeter, adding sandboxing is a snap, and rather inexpensive compared to the other players. In the case of Fortinet, they have one of the highest-scoring sandbox appliances out there based on the latest NSS report.
• “Our SIEM/ IDS / DLP / AV / is not protecting us.”
  SA is not a replacement for traditional security controls like SIEM, IDS/IPS, UTM/NGFW, DLP or AV.  You will still need all those, and yes even AV.  What SA can do is make those controls work better.  With the improved intelligence from an SA platform, you can reconfigure security controls to better detect and block attackers.  Any evaluation of SA technologies should consider how to both leverage and integrate with existing security controls. However, you should also ask why your SIEM, IDS, DLP, etc. is not protecting you. It may be the people running those systems who are the real problem. If your people lack the skills, motivation, and/or focus to manage these tools correctly, then the underlying technology is irrelevant.  Remember, how tools are managed and used has profoundly more impact on their ability to protect you than the underlying technology itself.
• “We must stop APT style attacks or suffer another breach.“
  SA platforms are not necessarily going to stop anything.  Most organizations do not deploy these products in any kind of active defense configuration.  Moreover, there is already malware in the wild that is specifically designed to bypass some SA platforms.  As SA products become more prevalent, attackers will develop new technologies to bypass their detection.  This is why some of the next questions are important to ask your shortlist of SA vendors.

 

2. What kind of SA platform is it?

Data Analyzer (DA) or Behavior Analyzer (BA)? So far nobody does both. Bluecoat/Solera is close. This should be obvious before you even consider a product.

3. Where Does the Threat Intelligence Come From?

How does the vendor get their intelligence data that actually gives the technology value? Any decent vendor should be able to explain their threat intelligence at exhaustive length. After all, it is a defining trait of SA products. If they cannot give insight into their threat intelligence, they probably do not have any (or its no good.)

Also, be mindful of the “black box” argument.  The idea that the vendor’s threat intelligence is so special, they cannot tell you about it. That is a cop out. If it is so ultra-secret and special they cannot tell you how it works, then its a good chance there is nothing there.

4. How Many Appliances / Consoles Do I Need?

Data analyzers are very storage and resource-intensive. You may quickly discover you need millions of dollars worth of appliances to truly cover your network.  For example, a complete RSA Security Analytics deployment can consume a full rack of data center real-estate.  Moreover, many of these products are not fully integrated with the corporate management platforms.  This can lead to a proliferation of consoles none of which talk to each other making management and usage tedious.

5. How is it Managed?

Get a thorough, in-depth walkthrough of the management consoles. Watch out for the endpoint BA type products, as they, like all endpoint products, will have heavy management overhead.

Which gets us to the next, and possibly the most important question.

6. Who will Manage the Platform?

SA platforms are just as resource intensive as SIEM and NGFW/UTM products (perhaps more).  Any vendor or reseller who tells you this is a plug-n-play deployment is lying (and a lot of them will tell you this.)  SIEM products failed in the market because they were complex.  SA products have expanded capabilities and threat intelligence, but they are not necessarily less complex.  In some ways, they are more complex because they intake different kinds of data and attempt to normalize it.

Regardless of the platform type, you will need trained, skilled, and competent security professionals to make these platforms deliver their value (note the emphasis that professionals is plural).  Moreover, these are not junior-grade people. They are senior-level professionals that can (and should) command six-figure salaries.  If you are not comfortable with committing a senior-level person to managing this investment, do not make the investment.

If you are planning to use a managed security vendor for your SA platform, few (if any) of them have the capability or skills to manage an SA product.  Target had outsourced the management of their FireEye product and it was largely ignored leading to one of the largest breaches in history.  A better solution is to contract skilled people to work locally or remotely to manage the platform.

If you are going to cheap out on management of your SA investment, seriously, do not make the investment.  These are not silver bullets that will instantly detect hackers. They are complex platforms that demand time, effort, and management (like pretty much every other security control.)

7. How Does it Work? How Does it Get its Data? How does it Alert? How do you Trace Down an Attack? What Reports does it Generate?

These are the “meat and potatoes” questions for an SA platform.  How does the platform analyze whatever it analyzes?  Does it detonate malware? Use data sifting? Some advanced mathematics (like Cylance).  If it takes in data, how does that data get to it?  And how much can it store? Where is it stored? How is data handled?

These big, open-ended questions should be an easy opening for the vendor to give you their pitch and explain their value.  If they cannot answer these big questions, then they are really missing the boat.

8. How Does the Platform Integrate with our Existing SIEM? Does it Integrate?

How will the platform interact (or not interact) with your existing SIEM?  Again, this is a big, open-ended question that should yield lots of interesting discussions from the vendor.

Incidentally, SIEM and SA will merge at some point into a common platform.  That has not happened yet, but it will.  RSA did it. Others will follow.

9. Does the Product have a Facility for Campaigns?

Ask vendors about the ability to handle Campaigns. This is a very useful aspect of SA, but many of the SA platforms do not have the constructs for campaigns, yet. However, it should be on their roadmap. If they do not recognize what a campaign is or what it means, then you might want to rethink the vendor.

Lastly, make sure you are placing your SA investment into the hands of a skilled security analyst.  It does not matter how much money you spend, these tools are still only as good as the people using them.  If you do not have in-house people, outsource the oversight and management to a skilled managed security provider.  However, most managed security providers do not, yet, support SA platforms.

10. What is on Your Roadmap?

This is a standard question for any IT vendor, but it has unique relevance for SA platforms.  This is a rapidly evolving market and innovative vendors have some very cool stuff on the horizon.  It is important to remember, whenever you invest in a new technology, you invest in the future, not the present.  Make sure the vendor has a good vision for the future and they can explain that vision to you.

Analyst’s Commentary

In this series, we have looked at the SA market, the technologies and the way it is being sold. Based on Anitian’s analysis, we are very positive, at this time, on the following vendors: BlueCoat / Solera, Cylance, CounterTack, and ThreatStream. These companies have some very innovative technologies.  We are particularly impressed with what BlueCoat is doing.  Their product line is still a bit clumsy at this point, but there is real promise there.

CounterTack, CrowdStrike, and Cylance all have fascinating endpoint products.  If they can overcome the hurdle of endpoint management, they could be serious contenders. Palo Alto Networks just entered this market with its acquisition of Morta Security.

ThreatStream has a very attractive TI-only product.  Its a good add-on for an existing SIEM if you want to roll your own SA platform.

For gateway BA products, the NSS report tells a clear story there: Trend Micro, Fortinet, Sourcefire, and Fidelis are the ones to review.  We are still amazed at Fortinet being on that list.  We have always been positive of Fortinet (except for their technical support, it’s awful), but this was a pleasant surprise.  Trend Micro was another big surprise. We have never liked Trend Micro since they went on a lawsuit binge a few years ago.  However, they can still turn out gems, occasionally.  Sourcefire, of course, they were a leader. They have a very strong track record of producing great technology. We just hope Cisco does not ruin them.

We are negative on RSA/EMC and FireEye but for different reasons.

FireEye just got a massive injection of reality with the NSS report.  Their response to the NSS report was an epic fail.  We have been generally positive on FireEye, until this NSS escapade. Whoever crafted that response should be terminated, immediately. FireEye came off as a crybaby whining about NSS Labs testing methods. That kind of response only works when you have an army of loyal users (like Apple or Palo Alto Networks) who can amplify your message into the Internet echo chamber. FireEye does not have that level of loyalty in the market, as such the response caused a massive drop in credibility (as well as stock value.)  FireEye still could pull it together if they learn some humility.

RSA/EMC has a solid platform, but RSA as a company feels desperate.  RSA Security Analytics is complex and very expensive. Overall, is a great forensics tool, but a mediocre SA product. Netwitness was a very innovative technology and so was enVision (once upon a time).  Together they do not  add up to a fully baked SA platform.  To make RSA really work, you need to invest in their entire eco-system of Archer and DLP, which is an extremely costly proposition.  To top it all off, RSA’s marketing for their SA platform is absolutely cringe-worthy. This video, for example, is awful. The acting, the premise, the laughable drama, is one, long, embarrassing FUD-filled facepalm. RSA needs to take a step back, dial the FUD back a few notches, and make Security Analytics (as well as the rest of their products) something that is more fully formed.

Conclusion

“It is a capital mistake to theorize before one has data. Insensibly one begins to twist facts to suit theories, instead of theories to suit facts.”  Such was the mentoring of Sherlock Homes to Watson in a Scandal in Bohemia.  It is advice that is equally valuable for today’s security analysts.

The Security Analytics market is all about finding the data that tracks down an attacker.  Holmes would be just as comfortable in front of an analytics console as he would lounging around 221b Baker Street obsessing over the characteristics of tobacco ash.

If anything is going to hold back the security analytics market it is the lack of Sherlocks.  There is a dearth of qualified security people who can manage these complex tools.  As such, any organization considering security analytics must also consider the people who will manage this environment.  Whether they are employees or a third party, security analytics is still only as good as the people who use it.

However, the technology will allow a skilled analyst to much more quickly identify, track, and prevent attacks.  As attackers become more sophisticated, and state-sponsored attackers become bolder, the value of sophisticated analytics become clearer.