Agile Engineering

Testivus for the Rest of Us: A Whole Team Testing Experience

in Agile/Agile Engineering/Humor/Leadership/Teams by Matt Badgley

The Story

I want to share a true story of a relatively newly formed agile team. We were made up of ten engineers writing and testing code in C++, Java, and fully embracing the emerging javascript frameworks. The application we were building was both complex and complicated. We were learning how to launch a product, trying to embrace user-centric design, and honestly, I often screwed up as a lean agile leader. All that said, we were a pretty great team and we were constantly learning.

We were in the midst of a new product release that was careening towards a GA date. The testers on our team — who were pretty great at what they did — were getting pressed against that GA date with the build-up of feature inventory. As a team, we were doing an okay job at establishing automation, but we could do more — a lot more. So with the testers in the needs of some aid and the need to shift left more testing, we arrived at Testivus for the Rest of Us.

How It Started

As a team, we often themed our sprints. Sometimes these sprint themes were merely broad brushed sprint goals. We already had the “Unleash the fury, Mitch” sprint in which we literally sprinted to the finish line several key product features. In need of raising our game around test automation and in general, switch the team from a “test right” to “test left” where quality was truly the starting point of all work — we decided to theme a sprint “Testivus.” What better theme than Testivus — time to air our grievances and demonstrate feats of strength.

NOTE: In case you’re not a Seinfeld fan, Testivus is a play on Festivus. See Festivus – Wikipedia for more information.

In preparation, we built the sprint backlog around those items that needed the most attention (we looked at a heat-map of components based on bugs previously found, code complexity, and skeletal impact to the product). This backlog wasn’t just a bunch of technical stories, we focused on the functional areas and created functional validation stories that restated the user outcomes, and we refined the acceptance criteria based on usage validation (as well as early learning from the field – e.g., performance needs, essential data needs, etc.). Some may call this just a regression backlog, and that’s fair; however, our mindset was different — it was all about outcome validation and, to some extent, validation that the product we were making was innovative and would kick-ass in the minds of our prospective users. Also, to note, this wasn’t about a “Hardening” sprint. You could argue this; however, we weren’t at the end of the release, and our core goal was to bring testing left in the cycle.

Once we built the stories and the resulting tasks included fleshing out a test harness, more in-depth unit testing practices and practicing TDD, functional UI test automation, and refining our CI process to extend into CD including bootstrapping test and demo data creation.

The Results of the First Testivus

The tangible outcomes were:

Stronger installation and deployment automation including packaging.
A robust test data generator and demo data bootstrap.
A new & improved test harness that handled both functional aspects as well as continuous performance benchmark validation. The harness covered over 1,800 scenarios nightly.
Increased code coverage from single digits to a substantial double-digit percentage.

The most impactful outcome was that team’s mindset which was already pretty good, became hyper-focused on how quality should be built-in. From this point forward, during refinement, the first thing we would talk about is attributes of testing.

The Longer Term

Testivus became a regular occurrence in the context that we called for a “Testivus” day or two every 5-6 sprints. Sometimes the need surfaced when we noticed a trend of bugs in a particular area, field feedback, and a couple of times we “Testivus” as part of a quality hack day.

Reflecting on Testivus, there was a need and the theme was awesome. It changed our mindsets, we had some focused energy on quality, and we had some fun — even though it was intense at times.

The great thing is — this concept wasn’t unique to us. Sometimes teams need a cause and a theme to try something new, and this helped us gel around some learning of good practices.

A couple of years later I joined VersionOne where the development team adopted “meatfest” which was a bug bashing event that had the team grilling out meat and vegetables, and drinking some beer throughout the day — all the while just fixing a bunch of stuff (I would point you to the awesome blog on this topic, but someone took it down).

Please let me know if you have experienced the same or would like to discuss further. Thanks for reading!

Update: We definitely weren’t the only one’s going down the Testivus path — just Google “Testivus.”

An Agile Team Task by Any Other Name – A To Do

in Agile/Agile Engineering/Leadership/Teams by Matt Badgley

Let me first say, I know the idea of task decomposition is fairly well covered and some might call it a basic practice of any team; however, as a coach helping teams to adopt agile software development, I don’t always see teams finding this practice of defining “how to get to done” as a natural team planning activity. Instead, I see teams getting frustrated with a clunky approach or a high-level of disengaged team members or a member of the process police controlling the meeting that is distinctly a team meeting or I still see the “lead” define the tasks and simply dole them out.

Defining “How” to get stuff done as a team should be the straightforward activity around software development. It should be an engaging activity of brainstorming, learning, sometimes arguing, and ultimately a good feeling of “let’s git’r done.” All-too-often it seems like a torture technique riddled with challenges. These challenges are often rooted simply in poor practices, some of the which include:

Creating faux work breakdown structures (yes, I said it — WBS) of standard tasks (e.g. each story has “Design”, “Code”, “Review”, “Test”, “Deploy” tasks). Some would argue this is not a poor practice, but tasks should have context and be relative. Not to mention, this is kinda mundane, requires no thought, and is flat out lazy and contrived.
Focusing on the accuracy of hour level estimation. If we are consistent with relative estimation and looking at cycle time, we can come up with the “low risk” and lean approach to determine which stories to work on. Estimation at this level is more of a measure of certainty.
Creating tasks and tests that are ginormous — like a 40 hour task for a two week iteration. If someone tells me that it’s going to take 40 hours, then I view that as high risk and we need to keep talking about it to better understand it.
On the other hand, creating micro tasks — yes, so many that the Task board looks like like a sticky note factory exploded on the wall or invoke pagination within the tool.
Having the developer and tester creating tasks and tests in isolation without any conversation with each other or the rest of the team.
Oh yeah, let’s not forget when I get asked about reports that measure how underutilized people might be. Often the focus of understanding “How” becomes simply an exercise to maximize utilization.

That last point about utilization, it seems like a frequent driver of the planning exercise. It’s odd in that sometimes people think that if we don’t have a plan with everything defined that we are going to do before we go do it; well — when the stuff we do get done is complete, we’ll simply stop working and sit there and stare at each other. I’m pretty sure that this is never the case and if it is, either get new people or look in the mirror.

Does any of this sound familiar? Does any of this frustrate you?

Well, I have a few ideas for you to try — most of these are common sense and talked about by others, but it’s always good to mix it up and try something different to get out of the rut or remove the pain associated with planning.

First, abandon planning using ideal hours — I know, I’m crazy — but hear me out. The common rule for any to do or task is that it should take no more than a day to get completed, ideally less. Tobias Mayer’s book called The People’s Scrum contains a curated set of his past blog posts, one in particular called, “When is Scrum not Scrum?” explains his reasoning for non-estimated, day or less tasks — (1) eases tracking – just burndown tasks — the burndown is number of remaining open tasks and (2) it helps to unearth unforeseen impediments that are often just part of what we normally do. While these are great, I would add that focusing on estimates and accuracy of the estimates actually just results in taking the eye off quality and flow of value.

Next, decompose just enough stories to commit to the goal and get started. The Scrum Guide is pretty clear about this:

… enough work is planned during Sprint Planning for the Development Team to forecast what it believes it can do in the upcoming Sprint. Work planned for the first days of the Sprint by the Development Team is decomposed by the end of this meeting.

Now, new teams may need to decompose all stories during a sprint planning, but teams that have been together for a while may be able to quickly determine what stories to do in the next two weeks, how they fit a incremental goal, and then just go. Don’t make your sprint planning meetings more painful — if you are planning for 2 hours and you have enough work to start the sprint, then go start.

Try different techniques for this meeting that encourage engagement. For example:

Taskstorming. A simple approach where the team selects which stories they are going to work on based on past velocity and then, interviews the Product Owner on each. During the interview, the teams is noting on sticky notes the tests and tasks. Once a goal emerges, then it’s time to task [and test] storm. On a whiteboard, write the title and draw a large box for each story. Now have the team place their stickies up on each story. Duplicates can be discussed as well as someone can put up something that doesn’t make sense to anyone else — well discuss it. Through this process, let the conversations flow – observe a story with a ton of task, then maybe it should be broken apart? Taskstorming not only makes the meeting more efficient, but it’ll make it more effective because everyone is involved and engaged in the process. At the end, have folks pair up and help get the data captured in your favorite tool (which means VersionOne).

Other techniques you can try include Task-n-Tell and Test First Tasking.

Finally, as I’ve mentioned in a couple past blogs, put this on the team – this is a team activity. If you are a team member, take ownership of this meeting and you lead it. The Scrum Master or Lead or Facilitator of the team would typically do this, but if you find them directing or assigning tasks out without engagement. Well you have a problem and the best way to change it is for a team member to take over. If you are a Scrum Master or Lead and you are struggling with getting the team engaged, try something new — don’t be insane. Try introducing a facilitation game or asking someone else to facilitate (not the product owner or anyone else in a position of authority).

No one should suffer through a sprint planning meeting. Yes they can be hard, but they shouldn’t be torture — so, inspect and adapt. Time to make them fun and effective.

Remember that at the end of the day, tasking is simply defining the proverbial To Do list to get a story done.

10 Causes of Chaos in Agile Software Development

in Agile/Agile Engineering/Product Management/Project Management/Teams by Matt Badgley

“Everything is so chaotic”
“Seems like we are constantly in a state of chaos.”
“Is agile always crazy and chaotic?”

Chaos is a word I hear a lot lately while working with software development teams that are either initially adopting agile software development or possibly undergoing a Lean/Agile reshaping to improve their existing agile approaches. I am often asked if the chaos will subside, and — the good news, it will — to a certain extent. But to best understand when things will slow down (or even if they’ll slow down), it’s good to explore some of the causes that make things chaotic with software development.

And in my world, there’s no better way to assess than to make a top 10 list:

1. New Teams Forming.

This one is obvious — as people come together to work with one another there’s a feeling out period. Usually during this period, teams are learning how to collaborate and work through conflicts. In addition to the people learning to work with one another, there are plenty of established organizational structure and culture barriers that slow the progress of a team forming.

2. Process Learning.

Another obvious one. Although most agile process frameworks (e.g. Scrum or Extreme Programming) are fairly easy and well documented, it takes time and practice to learn the mechanics. Couple this with #1, and well — there can be some real challenges to getting things done.

3. HEAVY Learning Mode.

This may seem redundant, but it really cannot go under emphacized. In addition to learning about each other and learning the process frameworks, as engineers — we are constantly learning about technologies, learning about our products, learning about our customers, well — just getting smarter. Needless to say, this all adds up.

4. Greenhorns.

If you ever have watched the Deadliest Catch on Discovery Channel, you get to see the struggles and pains of the first time deckhands – called Greenhorns. They are often way over their heads, running into challenges around every corner, and are just flat out exhausted. In addition to physically and mentally killing themselves, they are trying to prove themselves. Well, this is true with just about every new team member. Not only are they dealing with items 1-3 above, the intensity of the learning is magnified and until they have some wins and time under their belt, chaos exists.

5. When Quality is NOT Built-In.

It is my opinion that in software development, over the years we’ve invented this approach to quality that is “risky and invites failure.” [1] Yes, I stole that — in most software development shops, quality is something that is done by different people to ensure “separation of duties” and because of the mentality that developers make bad testers. Couple that with the attitude that QA Engineers can’t or don’t necessarily need to know how to code, we have what we do today — a ton of end-of-stream testing, out-of-band automation efforts, and honestly, even the staunches of agile shops struggling with testing and ensuring quality. Without quality weaved into the Design>Build>Test cycle, we tend to see a lot more of these noisy things called Defects. Defects take a ton more time and energy than building it right in the first place.

6. Quality Automation Doesn’t Exist.

Without automation your going to find it almost impossible or at least extremely constraining to effectively and efficiently deliver with quality in a predictable manner. Similar to the “build quality in” challenges, teams often struggle because of their estimation processes calls quality out separately (which makes it a target for evil doers) and often it doesn’t account for the work around automation. Many organizations adopting agile software development don’t have an automation strategy for their legacy code. Therefore, they tend to have bouts of analysis paralysis around the problem space or they simply say, “our product or software is too hard to automate the testing” — so they won’t try. The other challenge around automation is some see it solely as an end-of-line UI automation thing where a couple engineers work on it — test automation is a holistic challenge and needs to be treated as such.

7. Lack of Cadence.

When teams are starting out, they don’t realize that the first thing to establish is their cadence — get their schedule in place and time box everything. The cadence surrounds the process touch points that require formal communication which help us to build habits; thus, making the process aspects of agile software development more muscle memory. If you feel like you are always in meetings or your iteration meetings are not occurring at the same Bat-Time and same Bat-Place, it might be time to reset — your cadence is lost.

8. Unpredictable Release Cycles.

This one is an enigma, because there are teams I run into that say, “our product is to large and complex to release in short cycles” and then there are those that say, “we just release when we have to, it could be twice today or two weeks from now.” Looking at these two statements, they appear to be opposite in cause — however, it all comes down to #7 above and understanding what is the ideal batch size that reduces thrashing, allows for tighter alignment amongst teams, reduces “Integration Hell”, and prevents the amoeba style releases that never seem to end.

9. Deadline Rich Environment.

Projects are the problem — or at least the traditional sense and meaning of a project drives the idea of fixed scope and fixed schedule. Let’s look at the PMI definition of what is a project?

It’s a temporary group activity designed to produce a unique product, service or result.

A project is temporary in that it has a defined beginning and end in time, and therefore defined scope and resources.

At the end of the day, we drive our business off the idea of push for a date — we get emails from colleagues asking “when?”, we get emails from tools telling us “now!”, and we have other teams telling us “yesterday!” Ultimately, projects drive expectations which are generally dates, we can’t seem to get away from them until everyone realizes, we should design and define scope to fit the schedule, not the schedule to fix the scope — because the schedule usually flexes in these situations not the scope.

10. Estimation (and for that matter, Capacity) is Not Understood.

We often see teams measuring productivity units of time versus being measured as units of value. This is the case even in mature agile shops. Everyone is so focused on trying to come up with a voodoo formula to determine capacity of a team or organization and another voodoo formula to normalize story points across teams in order to build a long-term plan based on the cooked up numbers. The funny thing is that in many cases the approach used for estimation doesn’t really changed once an organization starts using agile — everyone continues to predictively plan what we really don’t know. Agile Software Development is an adaptive approach to estimation and capacity. We work with what we know, we measure value, we assess complexity, and we often simply size efforts based on relative uncertainty. If people would just keep it simple, try to get all stories to the same level or near the same level of certainty, and do the same with the to-do’s (a.k.a. tasks and tests) — then in a couple iterations, teams should be able to just count the stories and count the to-dos accomplished within a time box and use that for planning. Oh, if only it could be that simple … it is.

Okay, this was just my brainstorming or brain dump (literally) of ten things that cause chaos in software development, in particular in the situations where an agile adoption or reshaping is underway. Just keep in mind, change is constant in business — now, more so than ever. Software development is complex, so are people. The great thing about tomorrow is, we don’t know what is going to happen. So, just practice and keep in mind, if today is bad — then there’s tomorrow.

What Does Every Agile Development Team Want?

in Agile/Agile Engineering by Matt Badgley

Win! Duh, Winning!

Okay, I know that is so 2011. But it is only human nature to want to not be put in a losing position, in fact we talk a lot about winning. We are naturally competitive, some folks more so than others — you know who you are. This may be an evolutionary thing because of our need to be adaptive. I’m not saying we are totally dog-eat-dog. We like to be able to look back and see that we did our best or that we delivered the best as a team. In my opinion, we win when we meet or deliver beyond our own expectations as well as the expectations that we helped others to set.

As an agile development team, when we set commitments at the beginning of the iteration, we are setting expectations to win. If we miss our commitments — clearly we are not winning. This sometimes can be compounded when teams don’t even try to innovate and adapt in an effort to overcome our missed commitments. In these cases they are complacent, and they quit trying to get better, which means they clearly are not going to win. My guess is that most people on these teams are not very happy, and especially not happy to be on a losing team.

Let’s face it, winning is hard. Outsiders (a.k.a. stakeholders including customers, executives, management, and other teams) will levy expectations upon us and the team, many times these expectations seem unrealistic and, at times, seem maliciously imposed. Also, in order to win, you sometimes have to lose — winning takes practice, which means you and your team will likely fail before you win. Often winning is hard because we are trying to do so as a team. Becoming a team is a whole blog on it’s own; however, remember that in Agile we measure outcomes based on the team. So we win and lose together.

I am not really the one to ask about winning and I know there is not one single formula to winning. That being said, I have made some observations of winning teams, they are:

Set Proper Expectations. I think this goes without saying, we’ve talked about it before — part of being successful means setting the right expectations and having all the right parties in the room to agree together on expectations. This doesn’t mean that at times we shouldn’t stretch ourselves to be better, but the rules of winning should be agreed upon before we try to win.

Be Consistent. Now, you can consistently lose, but we are not primarily talking about losing here. When I say “be consistent,” maybe I should say “establish consistency.” In that we should establish consistency in both the Demand side and Supply side of the delivery equation. Smoothing out peaks and valleys improves the likelihood of meeting shared expectations and making the act of setting shared expectations easier.

Try. If you don’t try, you can’t win — ’nuff said. Well, let’s be a little more specific. Teams that don’t try to set the right expectations, or teams that waiver from their end goal because they are not on board simply will not win. These are the teams that are often the superheroes and they’ll end up within Sprint Burndowns that never reach zero.

Rely on Strengths of Team Members. Everyone on a team has different skills and strengths — both tangible and intangible. Although it is sometimes good to work outside your comfort zone so that you can develop, when teams constantly do this or team members take on activities that reside more in their weakness zone, it’s really hard to get things done and meet expectations. Try creating a skills catalog for your team and try out Strengths Finder. Doing this together and making it visible helps team members to put themselves and others in positions to win.

Embrace Adversity and Diversity. We should not only embrace adversity (a.k.a. challenges and failures), we should also embrace diversity (a.k.a. team member differences). When something goes wrong, it should always be a learning experience, even if that learning is, “we shouldn’t do that again.” A team that picks themselves up after failure and moves on is generally a stronger team in the end. From a diversity perspective, with Agile teams we look to build cross-functional expertise. Well, it’s also good to build cross-cultural teams. I’m not talking about putting someone on a team who is totally a bad fit, but instead — introduce a team member that will bring new ideas and approaches to solve a problem. Plus, with diversity generally comes new experiences — which is usually a good thing.

Practice. It goes without saying, you don’t get good at something without some practice — look at professional sports teams. Yes, there are those of us who are naturals at certain things — but even the naturals need to practice. When we practice, we learn – we learn new ways, better ways, and ways not to do something. Yes, we might experience beginners luck every-now-and-then; however, by practicing what is often unnatural to us becomes natural — then we can focus on the next level of challenges.

Apply New Techniques and Tools. As we practice, we should always be looking to find a new way to do something or seek out opportunities to improve some aspect of what we do with tools. Although I work for a tool vendor, I’ll be the first to say that tools don’t always solve a teams problems — people have to solve the problem. But tools and new techniques can improve the way we do something or make a solution more reachable. Not to mention, new techniques and tools can make solutions more effective and efficient.

Put Our Best Foot Forward. Simply put, don’t put anything out there unless it is something you would put your name on and parade it in front of the world. A team I once worked with used to have “Good Code, Bad Code” sessions. This was a community of practice meeting where the developers would present examples of either good code or bad code. The author of the code generally remained anonymous; however, you were really nervous that it might be your code about to flash up on the screen as an example of bad code. This type of discussion really forced everyone to raise their game, but at the same time if your code was brought up — you had the opportunity to discuss the circumstances and understand a different way. Nonetheless, you always tried to put your best foot forward.

So those are the observations and experiences I’ve had around winning teams. In your experience, what makes up winning teams? Or, is something I said something you agree or disagree with? I really think teams want to be successful, it’s human nature. It doesn’t mean we always have to win, but it sure is fun and everything is easier.

Why I Love the Boards We Use In Agile Software Development

in Agile/Agile Engineering/Product Management/Project Management by Matt Badgley

I know this is going to sound weird, but I have an affinity for the boards we use in Agile Software Development (or Agile Project Management). I’m using the generic term board, but you may refer to them as Storyboards, Taskboards, Testing boards, kanban boards, or simply job boards.

If I get into the way-back machine; back to when I first started my career. I was in Valdosta, Georgia working my first job out of college, Trus Joist MacMillan, and I was being walked around the manufacturing facility. I was in awe and deep appreciation for what I was learning. As I went through the customer service area, I noticed a large magnetic board and on it were these color coded cards with lines that represented stages and dates running across the top. I inquisitively asked, “what is that?” They quickly and proudly gave me the run down that these things all represented our customer’s requests for goods (a.k.a. orders). The columns represented stages of the manufacturing cycle and the lanes represented weeks. This board was the central information hub for the plant — the board described everything going on from department-to-department. It gave the all plant team members indicators as to how well are we satisfying our customers, what’s coming up for the next week, and which customer orders were being held up. The other thing I noticed is that as you went throughout the facility, other departments had similar boards that were a reflection of the main board but for the specific customer requests they were working and the boards reflected each departments workflow.

Does this sound familiar?

If it doesn’t, then you are probably not working in an agile world, or living under a rock, or you are using agile by you are missing out on something great. Me and my colleagues talk about metrics a lot – with our customers and internally. Well, in case you didn’t notice, the boards themselves are living breathing metrics. The boards are the team’s communications central as well as a key way to see collectively “how are we doing?”

In fact, the boards often ask just as many questions that they answer. These questions can vary based on your role and whether or not you are an embedded team member using the board or a bystander looking to gain insights from the board. I took on the the challenge of looking at questions asked and answered by the boards and found 40, yes 40 — check them out on this mind map. I’m sure there are more; however, I figured 40 was a good start and stop.

Besides all the questions boards ask and answer, I also love boards for their ability to simplify the portrayal of real-time information while being able to have complex information tucked away in the details. Teams can easily innovate and adapt their boards with the use of colors, pictures, lines, layouts, materials, etc.

Finally, I love boards because they actually build the teams. They provide a place to recognize accomplishments, share challenges, and they are the place to stand around and talk about what is happening on the project. If you are using an electronic board, you can still gain these valuable intangibles by keeping the board up-to-date and driving all collaborations on what is happening on the boards. Boards give teams focus, while also given them information that empowers them to make decisions to get things done and meet commitments.

Please share your thoughts on why you like, love, or hate the boards your teams use. Share some examples. If you are not using a board today, why not? Trust me — you will be glad you did.

Checkout this gallery of board examples that I found: