Changing methodology in the midst of the high-risk project is a risky proposition. At the University of Michigan we had several options: spoon feed and implement as you go, make immediate and drastic changes, plan carefully to implement confidently and thoroughly.

In 2005, Microsoft’s DevDiv (with 2000 participants and 40 million lines of code) overhauled its engineering practices to improve agility, quality, and customer satisfaction. Four years into the journey, customer satisfaction has increased dramatically. Product quality improved 10x. Velocity improved 2x, with schedule time for major releases was cut by eighteen months and quarterly releases of “power tools” allowed incremental delivery to external customers. Practices that change include planning, org, quality gates, branching, testing, tooling, reporting, backlogs, transparency.

There is much information available about how to begin automated UI testing projects. There is little information available about how to maintain successful, effective, long-term, large-scale UI testing projects.

Over the course of more than two years, my company Socialtext was able to grow a test automation project from a proof of concept of 400 test steps, run on demand, to nearly 10,000 test steps run automatically 24 hours a day, 7 days a week. This talk will cover test design, test architecture, test creation, test maintenance, and the project’s future steps.