Crowdsourced Security Testing

Identifying exploitable vulnerabilities in enterprise environments is a difficult pursuit – one that CISOs and their security teams spend considerable time and effort trying to accomplish. An important resource that can be unleashed to drive progress in this area is the collective power of vetted and skilled security experts – sometimes referred to as ethical hackers or white hats – to identify problems before a malicious adversary can do so.

Synack has been innovating for years now in crowdsourced security testing. Their solution involves use of a vetted community of researchers who can provide risk reduction for the enterprise through controlled crowdsourced penetration testing and vulnerability discovery. We recently caught up with Jay Kaplan, CEO of Synack, to ask about his team’s newest updates and how the enterprise can adopt this important area of crowdsourced security services.

EA: What are the differences between bug bounty programs and Synack’s crowdsourced security testing?

JK: Bug bounty programs create marketplaces for researchers to report vulnerabilities. This approach has improved security testing, but extends an invitation to outsiders to test your systems, which could add risk to your organization. Synack utilizes the bug bounty concept as part of our offerings, but we focus on a platform, rather than a marketplace. Synack’s crowdsourced testing platform supports efficiency, effectiveness, and control levels unattainable through a bug bounty marketplace. With our platform, you can augment and scale your team’s efforts without extra operational and resource burdens. We triage vulnerabilities submitted through the platform so that only valid ones are passed to customers; we also handle bounty payouts and researcher community management. Often it takes 24 hours to start an engagement, 24 hours to find the first severe vulnerability, and less than 72 hours to verify a patch. We deliver real-time security intelligence that bug bounty marketplaces cannot achieve. Customers can see their testing coverage data, researcher engagement data, and security scores based on real performance data. This helps managers make prioritized decisions to minimize security risk. And lastly, the Synack platform gives the customer a lot of control over the crowd. Security managers can activate and pause the crowd’s activity with the push of a button, and they can have visibility into all test activity, as well as full ownership of all findings and IP.

EA: How does the Synack solution work?

JK: All client asset testing is conducted through Synack’s secure VPN gateway. Directing all traffic through a VPN gateway helps us capture data behind the testing, and gives customers control to start and stop testing at any time. The testing data powers intelligence like testing coverage maps, attack type analysis, and security scores in our portal. The Synack Red Team (SRT) is our private network of highly-curated, skilled and vetted security researchers that power the testing. We have proprietary scanning technology that provides automated analysis to human researchers. During an engagement, we continuously scan all assets in scope, and researchers are alerted to detected changes, suspected vulnerabilities, and defensive technology sensing. The Synack Mission Ops team is our internal team of vulnerability experts that work closely with customers during an engagement. Mission Ops helps with asset definition and scoping, manage researcher communication and payouts, triage submitted vulnerabilities, and hold customer support meetings regularly. We offer our customers crowdsourced vulnerability discovery, crowdsourced penetration testing, continuous testing, and a managed responsible disclosure program.

EA: What is your process for selecting, vetting, and deploying your security researchers?

JK: Synack Red Team (SRT) researchers go through a rigorous 5-step vetting process to prove their technical qualifications and trustworthiness. Only 10% advance to become a Synack Red Team member. The first step involves evaluating and cross-referencing the candidate’s claims surrounding work experience, certifications and education with Open Source Intelligence (OSINT) sources. The second step involves a live behavioral interview, where determine the candidate’s character, motivations, and goals, and we get a sense of the candidate’s primary technical competencies. We also gather secondary information relevant to the vetting process to help us uncover any potential red flags. The third step involves a written skills exam, designed to evaluate the candidate’s fundamental understanding of a specific technical domain. The fourth step involves a background and ID check with identity verification and criminal background check, completed by designated and qualified third-party assessors. The first step involves acceptance and monitoring, where, once a researcher is accepted, we closely monitor them on the platform for a 45-day qualifying period. The researcher is required to submit a valid vulnerability report before fully being on-boarded. Even after researchers make it through this process, we still have controls in place to minimize risk. We uphold a zero-tolerance policy, actively monitoring all researcher traffic. Any inappropriate behavior results in termination of their SRT membership. We also give the customer control to activate or pause testing, based on their visibility into testing traffic through the customer portal.

EA: What are some of the more interesting vulnerabilities that your team is finding?

JK: Due to confidentiality requirements that protect our customers and researchers, we don’t disclose specific details about vulnerabilities we find during our engagements. But I can give you a breakdown of the vulnerability types that we see most frequently. Cross-site scripting is by far the most common vulnerability reported and validated through the Synack platform, followed by authorization/permissions and information disclosure vulnerabilities. The average payout for these vulnerabilities ranged from $200 to $900 last year. SQL injection and remote code execution vulnerabilities were the highest-paying vulnerabilities reported and triaged through the Synack platform last year. The payouts for these vulnerabilities averaged over $2,500 and made up close to 10% of total vulnerabilities reported and triaged last year.

EA: Have you seen any shifts in the selection and use of crowdsourced security testing solutions by enterprise?

JK: Gartner published a paper last year about the crowdsourced testing market called Emerging Technology Analysis: Bug Bounties and Crowdsourced Security Testing. In it, they estimated that more than 50% of enterprises will utilize automated and crowdsourced security testing platform products and services by 2022. In the next few years, crowdsourcing will be standard among enterprise. It won’t be a matter of if companies will utilize a crowd for their security, it’s a matter of how they’re going to do it. The conversation around crowdsourced security and the success metrics measured against these solutions are shifting. I think CISOs are becoming more concerned about their resistance to cyber threats and less preoccupied with just complying to regulatory standards. And rather than security teams focusing on the number of vulnerabilities being found, they are starting to care more about security scores, resilience or resistance to attack, and measurable risk reduction. As a result, CISOs and their security teams are more concerned about getting data and intelligence from their crowdsourced security, because that will help them better understand their security posture, prioritize their resources, and minimize their risk over time.