A major key to high-quality software development and maintenance is having the best possible configuration management (CM) solution. This article highlights the process for choosing the best possible CM tool, shows the best deployment process for that solution, and outlines the actual CM processes that are part of the CM solution.
Companies that seek to improve the quality of their software development and maintenance realize that a major part of quality improvement is to improve their CM solution. Modern CM tools, especially those with automated CM process support, cover all aspects of software development and maintenance, including version control, team engineering, testing, quality assurance (QA), release management, and change tracking [1, 2, 3]. The automation of all these aspects provides repeatabilityhence, the ability to more accurately forecast product release cycles. More important, automation of CM activities ensures all necessary steps and procedures have been followed, which provides a higher sense of confidence about the quality of resultant products. There is assurance that the unit testing has been done along with integration and QA testing, sign-offs by managers, and notifications to interested parties such as the change control board.
A modern CM solution can provide companies with many benefits such as
Hence, many companies are eager to ensure they are using the best possible CM solution today. Apart from the need to improve the quality of their products and the processes that create those products, there are other reasons that companies require a better CM solution: an unacceptable pain level with an existing solution, a need to meet various internal or external quality standards, the goal to maintain a competitive edge, and the desire to use better technology.
Many companies have used a simple version control tool, but the capabilities offered by modern CM systems are much more sophisticated and address many more problem areas [1, 2, 3].
Attaining a modern, effective CM solution involves more than just buying the latest tool, however. It starts with an evaluation process that determines the best possible CM tool for the company's needs and the way it will fit in with the overall CM solution. It then involves deploying that solution throughout the company or throughout specific projects and ensuring that the solution is actually used. Much assurance comes from having the right CM processes in place. This article addresses these key areas of evaluation, deployment, and CM processes.
The Evaluation Process
A quality CM solution starts with doing the best possible evaluation of the tools in the marketplace. The evaluation is ideally intended to find a CM tool that can automate most of the CM solution. The keys to success for evaluations are having a properly skilled evaluation team, designing useful evaluation criteria, having a realistic and comprehensive set of requirements, and running meaningful evaluation tests based on the criteria. In a good evaluation process,
The Evaluation Team
This team is made up of various representatives of the user community. It can include developers, testers, QA people, technical leaders, build managers, and project managers. All provide perspective and ensure their needs are addressed, while providing their own experiences, skill set, and processes to address three important areas apart from functionality requirements: usability, performance, and scalability requirements.
Usability requirements are those that make the CM solution easy to use, including an intuitive interface, simple setup and installation, straightforward migration of legacy code and CM information, minimal change in usage model, clear understanding of methodologies, minimal learning time, and easy customization.
Performance requirements are those that make usage of the solution feasible, including acceptable speeds for processing large volumes of data, command response, checkout time, builds, work area updates and synchronizations, and customizations.
Scalability requirements address the ability of the solution to scale up to meet the large number of users, the large volumes of data, and the large number of platforms (clients) distributed around the world. It also includes the potential for growth. A CM solution needs to evolve with the organization's needs.
Many companies choose the wrong tool because they only perform a feature-to-feature comparison and ignore usability, performance, and evolvability requirements, or they only want the cheapest tool in the marketplace.
Evaluation Methodology
The evaluation methodology entails finding the available products in the commercial marketplace and learning how much each costs and what requirements they meet. The final candidate tools are evaluated by detailed, hands-on use. Each candidate is rated against the selection criteria, which are based on the priority of the requirements. The tool that scores the best in all the tests and meets the greatest number of high-priority requirements is chosen. Once chosen, the tool and its related services are purchased. Then, preparation for deployment begins.
Deployment Process
An excellent deployment team is required to lead and manage the deployment process. A high-quality deployment process consists of a risk-based, methodical approach to the changes associated with technology transition. A description of the team and the adoption methodology follows.
Deployment Team
The deployment team leads, manages, and implements the change. The team consists of representatives from each target user group who champion the tools, needs, and concerns of their respective groups. The team has a leader equivalent to a project manager, since the deployment is treated as a project in itself. Plans are developed, risks are mitigated, schedules and milestones are set up, etc., based on an adoption strategy.
As a whole, the team needs to have a broad set of skills, including excellent communication and group facilitation, conflict resolution, analytical skills, strong technical experience (especially in CM and software development and maintenance), negotiation skills, public relations skills, leadership, great amounts of courage, and "thick skins." The team needs at least one member, ideally the leader, with "boundary spanning" skillsthe ability to communicate effectively up and down the corporate hierarchy, from developers to management. To function effectively, the team needs to go through team-building activities that develop trust and confidence among its members. To deploy the solution as quickly as possible, the team needs to work full time on the deployment activities.
A Risk-Based Adoption Methodology
The essence of successful deployment is to follow a risk-based adoption strategy. Risk management enables all the correct strategic deployment decisions to be made. A high-quality adoption strategy consists of a five-phased approach. A short description of each follows.
Phase 1: Planning and Analysis. In this phase, the team performs all the necessary preparation, strategizing, analysis, and scheduling required for the deployment. This result is a corporate adoption plan that consists of all the steps, goals, milestones, success criteria, decision-making issues, strategies, schedules, and other information needed to enable the deployment. From an analysis of this plan, a risk management plan is created. This plan describes all risks, mitigation strategies, unanswered questions, contingency plans, etc., that need to be addressed as part of risk management.
Phase 2: Process Definition. In this phase, the team understands the existing CM processes so that it can refine them or define new processes. These processes are then tested in the next phase. A process definition document captures the processes, and these processes are coordinated with the activities, procedures, and policies captured in the CM plan.
Phase 3: Pilot Projects. In this phase, the team designs a number of pilot projects that mitigate specific risks identified during the planning phase. The results of each pilot are used to complete the risk management activities, thereby eliminating risks where possible. The pilot results are also used to fine-tune all preparations required for roll-out, such as determining the right kind of training and training materials. These enable the next phase (roll-out) to begin as quickly and as painlessly as possible. For each pilot project, the team writes a pilot plan that covers all tasks required to carry out the pilot, capturing the results of the pilots along with a subsequent refinement to the roll-out plans.
Phase 4: Roll-Out. In this phase, the team systematically rolls out the new CM solution to each group. Few problems and surprises are encountered because of all the risk mitigation activities from the previous phase. This phase essentially involves finding the right opportunity to prepare, train, and roll out each group. For each project group, a group roll-out plan is written that covers all tasks required to ensure successful deployment.
Phase 5: Process Improvement. This phase is geared toward capturing lessons learned once the roll-out is complete. From those lessons, plans for continuous improvement are made. The risk-based adoption methodology is explained in more detail in [4], and ways to automate risk management are described in [5].
The CM Processes
A CM solution typically integrates all of the following processes across the call tracking, problem tracking, and development systems. A high-quality CM solution has all these processes integrated and automates them as much as possible, thereby providing a complete workflow- or groupware-like solution:
The Call Tracking (CT) system handles calls from customers who are having difficulties making their release work. It is responsible for bug identification and analysis, ensuring suitable patches or work-arounds are made available. Each released patch needs to be recorded to help the company manage its subsequent customer upgrades. For large companies with multiple product families in the marketplace, call-handling can be a monumental task, with up to 20,000 calls a day. The CT system feeds the new bugs into the Problem Tracking and Change Management (PT) system.
The PT system manages the problem reports, the lifecycle of bugs, and enhancements. It receives bugs from the CT system and receives enhancement requests from the marketing and research and development departments. Approvals for changes are given by the Change Control Board. The PT system feeds the necessary tasks to the Development and Maintenance (DM) system.
The Development and Maintenance (DM) system develops and maintains code. It ensures the code goes through a quality lifecycle with all the appropriate build management, levels of testing, QA, release management, and distribution back to customers.
Typical processes for CT, PT, and DM systems are described below, along with a typical corporate process that influences all of them.
The Corporate Process
The corporate process is the standard procedure used to develop new products at a company. The typical process described below indicates the major steps that must occur for a new product to be developed.
Because the corporate process effectively sets the lifecycle for a product and new releases, it must be aligned with all other processes that make up the CM solution, for it does influence the design of CM processes. Different groups will be responsible for different pieces of this process, and together with different tools, will likely provide all the functionality necessary to support the CM process.
The CT Process
The CT system allows all information regarding customer interactions to be recorded. It is an important part of the CM solution because it feeds the bugs into the PT system and uses information (such as problem number, patch number, and release number) to ensure the customer always has a version of a released baseline that will work for them.
When a customer calls, all details about the calls, e.g., the discovery of a possible bug, are captured, along with what solution was offered (a patch for that bug or a promised date by which it will be fixed). Whatever the case, the CT system sends a message to the PT system. The PT system responds by acknowledging receipt of the "bug" or other information (via a PT number) and begins the problem-tracking and change management processes.
A good call-tracking system has capabilities such as
The CT system helps companies respond to customer problems and helps the hot line support manager function by providing visibility into the support staff's work, plus it analyzes customer satisfaction.
The PT Processes
PT systems reflect the lifecycle of change. At any point in time, the status of any bug or enhancement is visible to the user of the system. A simplified PT process for a bug report or a change request follows.
A bug report or change request is entered into the PT system. Depending on the kind of bug, its severity, or the site responsible for analyzing it, it will be forwarded to the appropriate PT database. Once in a database, it goes through a review process such as the Change Control Board (CCB). The CCB may decide to reject it or to defer until a later time. Otherwise, developers are assigned to fix it, which results in tasks being fed into the DM system for those developers. Once those tasks have been completed, the DM process feeds back into the PT process so that the bug report or change request can be resolved and concluded. Some companies have simpler processes, some more complex; it depends entirely on the company's needs and the scale of complexity (the number of parallel releases, the volume of change, etc.)
From the PT system, companies generate reports and metrics to examine the quality of the products they develop and maintain. In particular, the kinds of metrics include the
In addition to managing the change, the PT system helps the PT manager to control staff workload by providing visibility into the status of all bugs.
Development, Maintenance, Testing, and QA Processes
Most companies automate their development, maintenance, testing, and QA processes as much as possible in the DM system. Tasks fed into the DM system are assigned to developers (typically through E-mail). Each developer works on a piece of code on a platform that is isolated and protected from other developers, but allows visibility into each other's work and the ability to merge and integrate changes. Sometimes, each works on the same piece of code in parallel, each fixing different bugs. Following is a description of a typical lifecycle for a piece of code.
A code unit begins when a developer creates or changes it in the working state. From there, the developer can checkpoint it (to baseline a personal copy) or can integrate that code with the other developers' code once unit testing is done. This enables integration testing by the testers. Once the testers have completed their work, the code can be sent to the QA department for final quality testing. If at any point during testing the code is found to have errors, that version of the code can be rejected, and a new working version (with the appropriate change) is created, and the cycle begins again. Once the code has passed QA testing, it can be released to its customers. Associated with the lifecycle are roles, transition conditions, events, and semantics of code visibility and isolation that provide the full details of what actually can and cannot happen.
Organizations use a type of process designed to optimize the edit-build-test-debug procedures. The process also enables developers to work in parallel with each other and in parallel with testers who are testing particular releases; parallel work is also possible with QA people who are doing their tests on particular releases and with release managers who are releasing their alpha, beta, or general baselines to their customers. Baselines are used for concurrent bug fixes and enhancements, stable baselines for testing, and QA. This optimizes the amount of concurrent development that can take place at any point in time.
Conclusion
A better CM solution can significantly help companies improve the quality of their entire software release lifecycle and products, giving them the competitive edge. But such a solution does not magically come overnight by simply installing a new CM tool. Such a solution requires effort, time, resources, commitment, and sponsorship to attain. The main steps involve sensible evaluation, careful deployment, and proper implementation of CM processes.
References
1. Rigg, W., C. Burrows, and P. Ingram, Ovum
Evaluates: Configuration Management Tools, Ovum Limited, 1995.
2. Cagan,
M. and A. Wright, "Requirements for a Modern Software Configuration Management
System," Continuus White Paper, January 1992, pp. 17.
3. Dart, S., "Concepts
in Configuration Management Systems," Proceedings of Third International
Conference on SCM, Trondheim, Norway, June 12-14, 1991, pp. 18.
4. Dart,
S., "Adopting an Automated Configuration Management Solution," Proceedings of
the Sixth Annual Software Technology Conference, Salt Lake City, 1994, pp.
15.
5. Dart, S. and J. Krasnov, "Experiences in Risk Mitigation with
Configuration Management," Proceedings of 4th SEI Risk Conference,
Monterey, Calif., Nov. 6-8, 1995, pp. 11.
About the Author
Susan A. Dart is president of Dart Technology Strategies, Inc., specializing in technology-based process improvement. She is the former vice president of process technology at Continuus, specializing in configuration management adoption strategies. During a seven-year tenure at the Software Engineering Institute (SEI), she established herself as a world expert in configuration management and software development environments. Prior to the SEI, she worked at Tartan Inc., Telecom Australia, and RMIT. Dart has a bachelor's degree in computer science from RMIT, Australia and a master's degree in software engineering from Carnegie Mellon University.
Dart Technology Strategies, Inc.
1280 Bison, PMB 510
Newport Beach, CA 92660
Voice: 949-515-4441
Fax: 949-515-4442
E-mail: sdart@earthlink.net
This article is based on a speech delivered at the Eighth Annual Software Technology Conference held in Salt Lake City April 1996. The author was then employed at Continuus Software Corporation.