English
There is a lot at stake in the performance and security of network devices, and that is why testing has been always been carried out by independent, third-party organizations.
But as we’ve all seen, independent testing does not necessarily ensure clear and fair results.
I have discussed the problems with relying on private companies and their somewhat secretive testing assessments in previous blogs. The test constructs are often not published, the methodologies are not transparent and there is limited documentation available to verify that the results are fair, accurate and repeatable.
A recent example of how proprietary testing can go wrong is the NSS/CrowdStrike lawsuit that occurred during a test of CrowdStrike’s Falcon breach protection solution. In this case, as Dark Reading stated: “The fact that NSS Labs retracted its rating of CrowdStrike's Falcon platform highlights one of the primary issues with closed or proprietary network security testing standards. Without visibility into the testing protocols and standards used, there is no way for organizations to objectively determine whether an NSS Labs assessment is right or wrong.”
Growing support for a transparent approach
NetSecOPEN was founded in 2017 to provide a transparent, standards-based approach to security performance testing. Today, the non-profit, membership-driven organization is supported by many of the world’s leading security product vendors, test equipment vendors and testing laboratories. These organizations all agree upon a growing set of publicly available test methodologies with the first set of methodologies being fully ratified by the IETF.
The difference between the NetSecOPEN model and the traditional model embodied by NSS (now defunct; see the recent Dark Reading article for details) is that NetSecOPEN uses a community-based approach and openly builds on the work of others. For example, the IETF Benchmark Working Group defines how to measure bandwidth of a firewall and NetSecOPEN applies this to modern security solutions as they operate in realistic scenarios.
To be clear, NetSecOPEN was not specifically formed as an alternative to private organizations such as NSS. In fact, private testing companies such as EANTC and UNH-IOL are founding members of NetSecOPEN and are part of any company looking to have NetSecOPEN certification. NetSecOPEN was founded to deliver apples-to-apples performance tests that realistically portray the security capabilities and performance of equipment with tests that are open and collaboratively created.
And in the past few months, I have seen mounting evidence that there is momentum behind embracing the NetSecOPEN model. For example:
Since its founding in 2017, NetSecOPEN has become the leader in performance security testing.
NetSecOPEN has grown to 14 members, including Cisco, EANTC, Fortinet, InterOperability Laboratory, IXIA, Juniper, Palo Alto Networks, SE Labs, SonicWall, Sophos, Spirent, Trend Micro, VIAVI, and WatchGuard.
NetSecOPEN continues to expand and broaden its scope in network security testing and assessment, with specific direction set in collaboration with its members
The former CEO of NSS recently formed a new organization, CyberRatings.org, that “aims to provide a more open and inclusive source of security product assessments…”
Spirent is a founding member of NetSecOPEN and is deeply involved in ongoing efforts with the consortium on defining and developing this standards-based approach to assessing modern security solutions. This will become even more important as digital transformation moves more organizations to cloud based services and security. NetSecOPEN test methodologies are included in Spirent’s Security and Performance assessment solutions for vendors and users alike to do the same test plans from currently over 50 defined NetSecOPEN methodologies.
Spirent solutions were heavily used in this initial set of NetSecOPEN certifications and all the test plans are available in the Spirent CyberFlood assessment platform, allowing users to model NetSecOPEN tests in their own labs. Learn more about CyberFlood.
