There are a ton of tools out there, some work well, some not so well. Most of them have some sort of secret sauce, you know the stuff we are not supposed to see because it might be a trade secret or the company is just not good at documenting them. I recently have been dealing with the Carbon Black Defense EDR(Endpoint Detection and Response) tool, and let me tell you, the secret sauce there is really a drain on what is a nice, if not quite mature, tool for Endpoint Response.

Before I start getting more into it, I have worked with Carbon Black products for the last 3 years. The majority of it was with Response and Protect (formerly Bit9), and I found that neither of those tools have this same problem that I am running into. Defense(formerly Confer), which I started dealing with a few months ago, I figured would not be too difficult to pick up since I had a background in the other software. Truth is, it was not difficult to make the switch in a generalized sense. It has been painful in the, I know what needs to be done and what I need to see but they are not allowing either to show or easily be accessed. And while simplicity is not a bad thing, lack of controls is not simplicity but rather creates a more complex and less usable product.

The idea behind EDR solutions such as Carbon Black Defense is to allow more insight into a machine, and to be able to block malicious software (especially the unknown unknowns) before any damage is done. Defense does a good job at this (although not perfect as you can lookup the whole “We Stop Mimikatz” debacle from last year’s Blackhat), but at a major price, which is time. The amount of false positives that get generated is amazing, which is why any solution like this needs to be tuned for efficiency. Yes you can use a SIEM to cut through a lot of the clatter, but in the end you still need to tune the solution. With CB Response and Protect, you can get very granular on rules and alerts to cut through the noise. Defense, though a “Response” tool in its own right, does not allow for granularity in the allow and deny rules. In fact I see it as more of an all or nothing based on their “Secret Sauce”, of which you get but a small idea in their documentation, of TTPs lining up with Threat Indicators. Here are the screens for allowing and blocking:

Allowing software by rule

Blocking by Rule

You can see both the Allow and Block rule construction is simplified. You can put in an application by path (with wildcards to cover directory structure up and down). Allowing allows you the options of completely bypassing CB Defense (Not a great solution) or in most Threat Indicator categories Allow without logging or Allow with logging. Why one would allow without logging makes no sense, but it is an option. Blocking you get a choice of Terminate Process or Deny Operation. The little target symbol will bring you to an investigation page searching for that operation. Notice you do not know how it determines what operation is what (there are a couple examples for most items in the documentation but not enough). Also, it is a generic allow or deny for the path or the application itself, and I mean you cannot say if X application invokes Y then it is ok or not ok. This is surprising considering all the hashing that goes on to make some of the determinations.

Lets look at an example. I am going to focus on Outlook starting up and using a plugin for encryption. I get an alert every time someone goes through this startup. Yes I have told Defense not to alert on this behavior, but the smallest change (IP address, machine name, new version any change) makes the alert as seen by CB Defense a completely different creature.

Outlook Alert

The alert I am talking about are the bottom two (each one is a separate machine but alerting on the exact same thing). You see the alert level is 3 and just so you know these are not blocked by policy, but they are still alerted. Not a lot of information on the alert, in fact you do not even see what it tried to execute from a memory buffer. When I go into the Alert Triage section on the bottom alert I see the following:

Alert Triage Outlook

Here I see the process outlook is invoking. No path of the process, not a whole lot of detailed information to make a decision on in the environment, but a lot of  trust us we know what we are doing type information. There is a similar type screen in Response but it gives much more info and gets way more detailed (including thread lists). The same information is in the background somewhere but is not being shown. There is the investigation button, so let us see if that gives us more info so we can make an informed decision:

Investigation Screen

Well, we do get more information, but what does it mean? We can see the paths being invoked and loaded (and functions called). We do not see full command lines (at one point I did with Powershell invoking commands but that has disappeared recently). I see the TTPs and Hashes. While not on the screen shot, there is a button if I click on the Application that allows me to look up the hashes on Virus Total (one bad hit there and it skewers the rating system toward alerting). So even without a ton of detailed information, I can make a guess that this is good if I know the environment uses said plugin.I do not know how it determined why it alerted, outside of Outlook invoking the plugin. This happens when you open a document, spreadsheet or pdf from outlook, or even from the web. I get alerts from people opening up spreadsheets they have created or that hook into software specific to the vertical to pull data from it. There are many other cases where you do not even get this much information.

So I should be able to not alert on this right? Wrong. I can whitelist everything and still be alerted. I cannot say do not alert if Outlook invokes this application. I can’t even say do not alert if whitelisted invokes whitelisted. I am using the word alert specifically, because that is what the SOC will see. The logging needs to happen for both good and bad. I can search through the investigation area for specific files, hashes, machine names, but in the end the info I get is as above. Now imagine getting 5 alerts every 5 minutes and having to go through each one. That can be a ton of alerts in a large company. What I get told is to look at the Priority Score. That helps very little as there is no documentation as to what causes a score to be. How many points is each TTP? The information is there in the background somewhere. Why can it not be viewed in an advanced mode?

I have not looked at too many other EDR (briefly Rapid 7’s at a prior place of employment) but it seems each one has their faults. I have heard about the false positive issue with some of them by asking around. Also, I am not saying that CB Defense doesn’t do what it is supposed to do, which is block and alert. What it does not do though is make life more autonomous in the Endpoint Detection and Response area, if anything it makes it more human resource intense just to get through the false positives. The idea of alerting on non-blocked events is great, it is how one can find unknown/unknown threats and actors potentially living off the land. The technology behind these alerts allows for more granular tuning, and the priority score is a great idea. Without knowing the secret sauce behind the alerts, if for nothing else than tuning purposes, software like this becomes a time sink.