English
Future-Proofing Begins Now. In this M2M blog installment, I go into the steps that organizations need to take with their testing strategy and labs to reduce their testing lifecycle from months to minutes. This process begins with an organization clearly understanding where they are in their test and measurement (T&M) footprint, and where they need to go from there.
The traditional test lab. Currently, the majority of labs are configured in static operational silos established in physical hardware-based infrastructures, with highly manual procedures for setups and tear downs. Development and QA teams need lab infrastructure to support the development and testing of applications running on the systems in these labs. The challenge with these silos and manual lab operations is it can take hours or days to support one request. In an agile environment, where test “sprints” must improve to perform in minutes and seconds, this approach begins to break down.
Inherent inefficiencies of the traditional lab. These types of environments also introduce bottlenecks when the quality of a pending release is poor and requires a project to go back to development. The people building the environment manually are skilled in network engineering and the people automating are skilled in software engineering. Communication and collaboration breaks down between these groups as they throw unvetted assessment artifacts over the fence, then point fingers at whose problem it is. This adds time-consuming iterations to resolve and isolate issues found in test. Other concerning characteristics of this model include:
Overprovisioned infrastructure – systems under test (SUTs) and test tools
Stranded resources with low utilization, including devices and applications
Systems always powered on vs. a managed power operation
High cost of having highly skilled networking labor repeating setup and tear down tasks
The bottom line. As everyone operates in silos, development and operations teams don’t have access to realistic test environments and cannot test to ensure production-ready performance. Given that low-quality application releases can damage the brand – with the scale and speed of testing required in an agile world to support the emerging challenges of digital disruption – this model cannot scale in the long run.
Phase 1 – Lab consolidation. Implementing this phase involves addressing the physical networks that make up the majority of infrastructure for traditional test labs. The objective is to migrate standard physical lab elements, or labs, into an environment that operates like a virtual one. This virtual transformation of the physical infrastructure, through pooling of lab resources and automation to deliver these resources as automated services, is similar to how IT operations have managed similar requirements. Traditionally responsible for connecting servers and networks, IT groups embraced virtualization technologies to consolidate and optimize use of compute, storage and networking technologies for the enterprise today.
Components of consolidation. This consolidation process requires a cloud platform such as lab as a service (LaaS) that provides unified access, resource management, software defined connectivity, workflow automation and data analytic capabilities These integrated platform services, along with easy-to-use natural workflows native to LaaS, enable higher scale and performance throughput of lab functions on an enterprise scale. Through improved management and visibility, business leaders and managers also begin to realize well-known benefits from employing the lean manufacturing methodology. Those benefits include:
Eliminating waste in all lab processes through improved data management and automation
Building quality into the solution by integrating labs with development operations (DevOps)
Creating knowledge which is easily distributed through sharing of reproducible environments
Delivering accelerated productivity through speed of process with integrated dev-QA workflows
Optimizing the entire test lab environment and processes through automated services
Phase 2 – Lab and test automation incorporating DevOps and Agile principles. The overall evolutionary roadmap of change involves three elements: people, process and technology. The first phase mainly focused on the last two of these elements. Now, optimization of the test lab must be addressed incorporating all three. Once the physical infrastructure is centralized and virtualized for greater efficiencies, the next tier of advancement involves lab and test automation leveraging DevOps and agile principles to deliver comprehensive continuous testing (CT) and to support continuous integration (CI) and continuous delivery (CD).
Where the rubber meets the road. In contrast with the monolithic waterfall development methodology of the traditional lab approach—which often represents months-long development and delivery cycles, or longer—the agile approach embraces continuous short cycles of process, traditionally with two-week “sprints.” Leveraging the guiding principles of lean manufacturing and agile methodology used in Phase I, and utilizing DevOps by tool chaining development tools and on-demand lab services, eliminates well-known process friction and waste. Too often these process impairments have been accepted as normal between development-QA and lab groups. Examples of these inefficiencies include: finding equipment, setting up environments, configuring environments for development or test, troubleshooting, with dozens more in most labs. Examples of integrating the pipelines with LaaS are:
Test management system – Mapping LaaS resources with test requirements and/or linking bugs and logs with LaaS configurations of systems or devices under test (SUTs or DUTs)
Test executive and build systems – Integrating build environments with real-world network test environments, including automatically finding equipment, connecting it, configuring it, running tests on it and tearing it down when finished
Inventory systems – Map your lab or data center (DC) inventory system to LaaS resource management to monitor and maximize usage while minimizing conflicts
Cloud or network orchestrators – Automatically connect VNF and SDN systems seamlessly with any legacy physical systems without the complex user mapping of L1 and L2 connections
Once implemented, the value grows. Many more examples exist, but as you can see by abstracting lab infrastructure as a service (IaaS) with APIs, test operations are streamlined at all stages of the development pipeline. As a result, continuous test coverage increases. This in turn creates and captures more valuable data for future testing and predictive analytics. These benefits all support the development team’s need for agility and continuous integration, along with operations’ need for stability and reliable delivery.
Phase 3 – Democratized and liberated testing. This is a state that many organizations aspire to reach, and which few have yet achieved. It is one, however, which must be realized sooner than later to meet the challenges of 2025. Since the promise of NFV and 5G infrastructure is to virtualize the networks created in these emerging environments, everything will be ubiquitous to the cloud. Organizations won’t see traditional IT infrastructure anymore, nor should development and QA see lab infrastructure. With networking and applications integrated into the cloud or multiple connected clouds, and delivered as on-demand automated services, next generation LaaS infrastructure empowers the creation of a mobile enterprise workforce, performing testing continuously throughout a product or service delivery pipeline.
Realizing an archetype paradigm. By reaching this stage you’ve achieved a maturity cycle where a culture has embraced a new normal, where labs operate as on-demand services. This solution enables rich and reliable automated services and functions as a critical enterprise-wide business system. Democratizing means accessible to everyone. So why not offer this to customers, vendors or partners? This capability extends the collaboration and resources of operations to improve innovation, or create new revenue streams by offering the lab as a service for consumption of tools, tests, devices, and more. For example, with this advanced phase of operational maturity, and by building new user portals on top of your LaaS operation, you can enable a new self-service TAC center. This would allow customers to help you reproduce their environments in your environments, or demonstrate new release capabilities and easily compare them against prior versions, or enable test services to perform customer tests on their configurations. This modernization of the lab provides the next big step in this transformation of the lab. In doing so, it shifts the business model of labs from a cost center to a new revenue stream. The benefits of this solution approach are transformational:
Global resource “cloud” offering universal access to consume costly CAPEX-intensive resources
New operating models to enable seamless scale of testing physical and virtual systems
Enablement of new business models by opening LaaS operations to customers, vendors and partners, from cost center to revenue center
A 30,000 foot view of change. Understanding the progression of test lab maturity at a high level helps all members of the organization understand the context and importance of this transformation process. It highlights where you need to aim to justify this transformational leap: democratization of all lab resources, both physical and virtual. I’ve experienced this transformation at every stage, and seen the return on investment (ROI) in the process. As a result, I’ve also seen CEO’s and CFO’s acknowledge the value and become more deeply involved and invest more into realizing this vision. Eventually the time will come to move a framework designed in the 20th century to one designed for the 21st century. By taking this leap, you also leap from Months to Minutes.
The Months to Minutes blog series highlights how continuous testing, driven by test and lab automation, optimizes the validation of an organization’s networks and business offerings amidst the growing challenges of digital disruption. The result: taking testing processes which too often take months to perform, and deliver improvements where it can all be achieved in minutes – with increased efficiencies and capabilities.
