English
Fair Testing in an Unfair World.
Today’s content-aware security and inspection devices are being put under more demands and pressure than ever before. Network equipment vendors work every possible position and spin to showcase how well their solutions will perform over their competitors in production networks. The lines are blurred when users have to make a decision of whose solutions to deploy or what inspection policies and rule sets should be used for specific applications. Many enterprises have relied on assessment bake-off reports of various network solutions, usually done by private test labs. The issue with these private bake-offs is that the tests themselves are typically defined by the test lab and are not necessarily based on real-world use cases. The results might be published, but that actual test’s methodologies are usually kept secret – otherwise, you have to pay to see what was actually done. In fact, there have been recent lawsuits by some vendors against specific test labs citing unfair practices in results reporting, negatively impacting a vendor’s standing in the market.
Enter NetSecOPEN, an industry consortium of network equipment vendors, test labs, test equipment vendors and enterprises – all looking for a better way to test and assess next generation security devices in an open and fair way. Standardizing any aspect of technology can be a daunting task. Getting multiple entities to agree upon methods and best practices with differing opinions and approaches is a process unto itself. The outcome of this effort, however, presents more meaningful and transparent assessment solutions. Let’s consider IETF RFC2544, RFC3511 and a host of other ratified test methodologies that have been used in practice for many years. NetSecOPEN is bringing this concept to advanced security and performance testing for today’s world of advanced HTTP, HTTPS, TLS, attack and malware inspection services, with test methodologies that are in process for IETF ratification.
As a founding member of NetSecOPEN, Spirent has been intimately involved in the definition, creation and use of NetSecOPEN methodologies. We have built these tests into our CyberFlood and Avalanche solutions for physical and virtual use cases. Never before has advanced performance and security assessment been made so easy and available to vendors and end users alike. The Enterprise Perimeter Mixed Traffic Security test can showcase a Next Generation Firewall’s ability to handle massive amounts of mixed application clear text and TLS traffic to validate application identification polices. The new NetSecOPEN security assessment methodology defines over 400 real attacks to be used for base line security efficacy testing. When vendors go for certification, the test methodology definition also indicates 20 attacks will be “hidden” from the vendor. This makes both the test as fair as possible and the device being assessed truly tested with an unknown set of attacks. This in turn validates security accuracy without the test bed being choreographed to purposefully “catch” all attacks used in the assessment. NetSecOPEN also defines over 50 primary performance tests for HTTP/HTTPS open connections, session setup rates, and bandwidth - all with the ability to auto goal-seek to quickly find a device’s true upper end performance capabilities.
Spirent provides NetSecOPEN methodologies built into our CyberFlood assessment solutions for use on a variety of our appliance platforms. Methodology configuration set-up is quick and easy and test results are easily obtained, providing a complete environment for ongoing NetSecOPEN assessments and performance validation.
With NetSecOPEN, network vendors and their customers can finally see comparative results of how advanced security solutions will work under real-world conditions with fair and open test methodologies, all under the oversight of a neutral standards organization.
