I am very pleased to introduce guest blogger, Toby Morris, for this latest post.  Toby, known around these parts as “The Amazing Toby”, is a software developer at Valpak.  Toby has been with Valpak for many years and has had the opportunity to experience our Agile transformation from the trenches, as the title of this post suggests.  Toby gives a (sometimes brutally) honest and extremely insightful perspective on Agile.  Enjoy!

Over the past year, this blog has chronicled the Agile transformation of a company from the viewpoint of a self-professed “Agile enthusiast”. So, you have to take those posts with a grain of salt. Of course she’s going to have a relatively rosy version of things, seeing the big picture, seeing how upper management reacts to the change, seeing how the new practices is affecting the quality and quantity of product being produced. But one of Agile’s tenants is people over processes. So to truly judge the success and impact of Agile on a company, you have to get in the trenches and see how those changes are impacting the grunts. From the viewpoint of a grunt, I’m pleased with the change.

To show how things have changed, I have to tell you how they were. Just prior to our Agile transformation, we had several problems as a department. Morale was pretty low, and people were fairly unhappy with their jobs. Our department held a series of meetings to try and find the root of this dissatisfaction. After a few talks, some common themes emerged.

Most technical workers in the department had a director, a manager, and sometimes a lead. Most worked on multiple project teams, and each project team had at least one Project Manager. Each Project Manager considered their projects and tasks to be the highest priority for their teams. People were being pulled apart by competing PMs.

Another common issue was unrealistic expectations for project performance. We frequently worked on projects that were breaking new ground, but were required to make estimates that were little more than wild guesses, and we were held to them. If we “guessed” it would take six months to learn and implement a new technology, but it actually took eight or nine months, we would fail, leaving a shattered trail of disappointed business people and angry project managers.

Along with the missed deadlines, the business was becoming concerned about our ability to deliver quality products in reasonable amounts of time. We constantly ran over, and with every looming deadline, overtime pushes, sometimes all night and all weekend long, were the norm.

When the big leap was announced, there was a lot of skepticism. We were in a very sore spot, and now someone wanted to move our cheese. They actually wanted to replace our cheese with some type of healthy, new age vegetable, and we were scared. We were confused. There was a great deal of apprehension. How could we do everything we’ve been doing in a tried and true Waterfall world in short, little two-week sprints? And even if we could do it, why bother? Would it really make a difference?

Yes, it did.

We went from having multiple overseers to overseeing ourselves. We had one person determining what we’d work on, and we would decide how we did it. We broke big, multi-month projects into small, two-week stories. Some teams even broke them down smaller, opting to work in simulated one-week sprints.

It wasn’t easy to change. We all had to move, physically, to be closer to our teams. And we lost our walls. We moved into more collaborative cube setups. This provided a much more significant benefit than imagined. Sure, we could communicate readily now. No more emails or phone calls. No more standing up and walking across the room to ask a simple question. We could now turn, get the person’s attention and get our answer in seconds. Sure we could more easily collaborate. Having the developer sitting next to the QA, or the front tier, middle tier and back-end guy sitting next to each other, we could make changes and fixes in minutes rather than hours. But the real benefit is camaraderie. These people that we self-organize with become more like friends than just co-workers. A true sense of team starts up, and before you know it, you’re working together more efficiently.

We also had to learn new words and principles and practices. We learned how to effectively scrum each morning. We learned how to write user stories, and break them into actionable tasks. We even got yellow penalty flags to throw at others when they slipped up and let some waterfall back in.

We had a lot of changes.

But we got more benefit from those changes than imagined. Working in small, self-organizing teams, with a single product owner determining what we were working on for the next two weeks made all the difference. No longer did we have ambiguously ranked priorities. We knew exactly what needed to be done first, second and third. No more were we getting constant interruptions and additional tasks from various project managers. What we have to work on at the start of the sprint is what we’re working on for the whole sprint, and heaven help the person that tries to add a user story mid-sprint.

By our second sprint the business had already noticed and welcomed the new process. Instead of waiting months to see anything tangible, and to find out a project would be delayed even further for obscure, incomprehensible technical reasons, they were now seeing incremental upgrades to existing apps and new features being added to new apps on a bi-weekly basis. And even better, their feedback was incorporated into the rapidly evolving applications they depend on. They were helping to shape the tools they use.

Having a business person intimately connected to each technical team also opened the top secret technical world up and exposed our black magic. We gave our product owner an understanding of the whys and hows of software development. There were no longer the “we’re unable to do that for technical reasons”. Now, we would tell him that the protocol we’re using doesn’t support that level of encryption, and that we’d need to re-engineer the system to use a different API to achieve his needs. And that it would take another sprint to discover what that would require. And he would understand what we’re saying, why it’s causing a delay, how long the delay will be and ways to work around it. The smaller tasks and user stories mean smaller and easily recoverable failures.

Collaborative and adaptive. That’s how we work now. Morale from a technical worker standpoint has significantly improved. Our products are visibly more stable, bug-free, and valuable. Our business teams are now part of the team, and believe in our ability to make their jobs easier. We are no longer adversaries. We are agile, not hostile.