An Era of Test Environment Management.
To keep pace with next generation of digital offerings, testing organizations across industries have moved with speed to define new test processes, use cutting edge tools and train teams in newer testing methodologies.In the midst of their pursuit to achieve higher standards of testing, what arguably has been missed is the importance given to the most basic foundation block of testing maturity: Test Environment Management (TEM).
The ever growing horizon of testing has produced more unforeseen challenges than ever before. Increased number of complex integrations between applications built on disparate technologies have only made the testing job more demanding and consequently agility associated with testing is at its peak. The response time available with the testers is getting increasingly shorter as continuous feedback based on customer experience is quickly being fed back into the development cycle. To deal with such unprecedented situations, it is crucial that testing teams are supported by well-maintained test environments to ensure that testing doesn’t become the bottleneck in the overall scheme of project deliverable.
The mishandling of ‘not-so-critical’ test environment management has often been associated with critical problems thereby incurring huge costs to the organizations. As per the survey done by Caper Jones & Associates in 2011, poor software quality costs organizations in excess of $150 billion annually in US alone. Poorly built and managed test environments impacts not only the schedule of projects but also leads to high cost incurred on fixing those defects at a later stage. Lack of proper planning around test environment scheduling and allotment leads to substandard environment configuration and downtime. Unplanned outages during testing results in reduced quality / throughput of testing and increased risk. Lack of readily available and accurate metrics around environment management leads to crucial steps being missed and ultimately results in production issues.
At the outset of any testing effort, it is of paramount importance for testing teams to have well defined, established and professionally managed test environments in place. Without corresponding changes in test environment management maturity, it is unlikely that testing will deliver better, faster and more efficient results.
The below diagram depicts the overview of recommended Test Environment Management focus areas.
1) Organization & Operating Model
Formalize Non Production Environment (NPE) Management Operating Model:
- Centralized NPE Governance (Process & Measurements)–The proposed governance structure is as shown in the diagram below. This centralized environment management model connects the people, processes and tools aspect of TEM.
- Formalize roles and responsibilities — Define and implement roles and responsibilities for environment management function focused around Test Environment Manager and Test Environment Analyst.
- Business Outcome — Streamlined roles and responsibilities would result in support for complex inter-dependencies, different release velocities and a unified view of non-production environment issues.
- Define Service Catalog for Non Production Environment (NPE) — Establish a service catalog for NPE covering Incident & Problem Management, Change Management, Request Management, Access Management, Deployment management, Environment Definition and Provisioning, Booking & Scheduling services.
- Business Outcome– Increased accountability & governance, formalized structure of environment management services, increased awareness around environment management operations.
- Establish training and development — Develop training programs and forums to facilitate knowledge transfer to resources and improve resource flexibility and understanding of environment management.
- Business Outcome — Formalized training on Environment Management, consistent and improved understanding of environment management process.
2) Non-Production Environment Processes
- Establish Non-Production Environment booking and scheduling — Establish non-prod environment booking and scheduling process and automate using NPE management tool.
- Business Outcome — Enable faster decision making on environments available and alternatives, enables release planning and scheduling with environment related dependencies, increased transparency on environment utilization enhances environment capacity planning capability, reduced incidents & failed releases through early detection of conflicts, enables environment availability information to facilitate reuse/recycling of existing environment(s) or provisioning of new environment(s).
- Establish ITSM Best Practices in NPE Management — Define Problem Management (PM) process and formalize Incident (IM), Service Request (SRM) and Access Management (AM) processes for NPE.
- Business Outcome — Streamlined process and central tracking leads to faster turnaround time, secured environment with controlled access, consolidated and identified dependencies thus enabling valid requests are fulfilled and conflicting requests are avoided.
- Establish Monitoring for NPE — Establish monitoring within NPE & implement a single view dashboard with notifications.
- Business Outcome — Usage of defined process and a central monitoring dashboard leads to higher environment availability.
- Establish Change & Configuration Control for NPE — Establish inventory maintenance by utilizing NPE management tool as the CMDB (Configuration Management Database). Extend and formalize the Change & Configuration Management process to non-production environments.
- Business Outcome — Reduction in unauthorized changes, centralized inventory for maximum asset utilization, unified and updated repository with all NPE related information, reduced incident and failed changes.
- Develop and update SOPs, architecture blueprints, runbooks, templates — Establish a robust framework for knowledge acquisition, storage & dissemination of processes and application hand-offs.
- Business Outcome– Up to date application documentation, formal artifact repository/centralized knowledge management.
- Establish Prod sync process for lower environments — Maintain non-production environments similar to production environment in terms of application software version and infrastructure.
- Business Outcome– Optimal quality of testing, reduced defect leakage.
- Implement communication and escalation model — Structure and standardize management reporting at all levels (corporate projects, base projects, maintenance). Define communication model for internal and external stakeholders to increase transparency and trust.
- Business Outcome — Standard & periodic communication to all key stakeholders, improved transparency & visibility on NPE performance.
3) Tool, Infrastructure & Support function
- Establish inventory for NPE in the Configuration Management Database (CMDB)– Maintain a repository of CIs (Configuration Items) as well as their relationships for all non-production environments.
- Business Outcome — Increased control and better utilization of infrastructure, enhanced reconciliation of systems and infrastructures.
- Establish a Booking & Scheduling tool–Automate the process of booking and scheduling by choosing the fit to use environment management tool.
- Business Outcome — Enhanced visibility and environment conflict resolution, increased release and environment collaboration.
- Extend monitoring tool to NPE–Use the production monitoring tools to monitor non-production environments.
- Business Outcome — Early detection of environment issues in test regions.
- Extend tools for NPE Known Error Database (KEDB), incident & change management — Extend the production tool usage to test environments for KEDB.
- Business Outcome — Avoid efforts to detect root cause of known errors in NPE.
- Establish TDM & Deployment automation–Establish effective test data management through appropriate usage of processes and tools.
- Business Outcome–Better test coverage, optimized testing across regions.
Originally published at medium.com