Using Advanced Heuristic Algorithms to Risk Score Your Web Apps

When I first witnessed active IPS mitigation driven by signature-based attack detection, I joked that this was like issuing machine guns to a blindfolded army. That is, I felt that even the best intrusion detection solutions of the time – roughly the mid-1990’s – would typically misinterpret attack indicators, and then make things worse by shunning innocent, duped source IP addresses. I held this view for two decades.

More recently, however, I’ve had the opportunity to delve into the details of how front-end, cyber intrusion detection systems might more accurately identify risk indicators. I’ve also taken the time to look more closely at how back-end, intrusion prevention systems might initiate safer response actions, devoid of those nasty side-effect conditions. In both cases, I can report that things are looking much better.

I had the privilege to broach this topic with two of our industry’s smarter minds: Stuart Laidlow and St. John Harold, both with UK-based Cyberlytic. During a lengthy videoconference held across the big pond, we spent some time digging into the technical details of the front-end passive profiler and the corresponding back-end web application firewall (WAF) that comprise their promising new cyber security solution. Let me try to summarize what I learned:

When asked about improving the quality of threat determination, both men drew on their extensive experience in British intelligence and law enforcement to highlight the importance of high quality machine learning algorithms, combined with a specially designed risk classification model. That is, they’ve designed their front-end profiler to collect data from your SIEM or firewall, and to then apply advanced heuristics to estimate the risk intensity of observed web-based indicators. This seemed sensible to me.

Their smart use of advanced heuristic algorithms allows for a more intelligent response triage of observed activity, using the tool’s risk classification model to prioritize real-time response. This risk-based approach lies in stark contrast to early IPS devices, which would see a simple regular expression match of some indicator, and would then simply knock off the offending source IP address. You will recall that this led to enough unintended consequences that most security teams would just shut off the active mode software. The Cyberlytic triage approach seems to improve on this weakness considerably.

One class of real world applications that we discussed in our videoconference involved using front-end intelligence to correlate observed web indicators with out-of-band errant activity. For example, if a commercial power plant was experiencing unusual outages, remarkably useful anomaly indicators (e.g. metadata from users checking outage maps on the power company’s web site) might be derived by security teams based on front-end web-based intelligent analysis, integrated with local knowledge of the user’s domain. This is reminiscent of pharmacies using metadata from users buying over-the-counter remedies to predict imminent flu outbreaks. If the predictive risk model in any of these cases also considers all-source intelligence about tangible on-going threats, then a world-class security operations environment is achieved. This is a cool and profound concept.

So, while I’m not ready yet to proclaim that any WAF, even one as powerful as you’ll find with Cyberlytic, is going to detect every advanced persistent threat aimed your way by a nation state actor, I am perfectly willing to recommend that a tool such as from Cyberlytic, in front of your web-based workloads would seem like a more-than-responsible means for tightening up your cyber defense.

You’ll therefore be doing yourself a favor to take some time to inventory your web applications – including any workloads behind REST APIs – and make sure to embed some good front-end, risk-based, machine-learning intelligence in the web traffic path. And yes, that is a big mouthful to say, but if you do it right, you’ll be presenting an even bigger mouthful for your adversary.

Let me know what you think.