Category archive

Agile Engineering - page 5

Another, Another Post About Agile Metrics

in Agile Engineering

Even if you are on the right track, you will get run over if you just sit there.
– Will Rogers

As you can see by the title of this blog (or shall we get fancy and call it an article), it is about Agile Metrics. And as you can also glean from the title, this is yet another blog about this topic. However, I plan to be different – I’ll come after this topic from purely personal experience and reflection. Now I’m not saying the other writers do not reflect upon their personal experiences, but over the last eighteen months the teams I worked with have done an excellent job at forming out and tracking metrics. These metrics do two things:

  • Provide the stakeholders with insight regarding to what the teams are doing, how well they are doing it, and what is coming when.
  • Provide the team with measurements that can be monitored to identify and understand how the process of software development and deployment is working.

These both lead to consistency, trust, and continuous process improvement. Now don’t get me wrong, metrics alone don’t provide these things – but, capturing metrics, measuring, and then taking stock in the measurements do help establish consistency, foster trust, and drive continuous improvement.

We chose six metrics to capture:

  • Roadmap Effort Achieved. This equates to enhancements, new features, or business value activities performed during the iteration versus defects and other technical tasks which although are required for sustaining the product, they don’t have the agreed upon value with the Product Owner.
  • Utilization. This is broken out into two metrics: (1) Ideal Utilization which considers actual hours booked against all project activities divided by the team’s ideal hour capacity; and (2) Planned Utilization which considers actual hours booked divided by the team’s planned capacity. The first measure is generally mid 60% to low 70% depending on the roles and the level of the team.
  • Defects Raised. If this isn’t clear, then nothing is – just kidding, this metric tracks those defects that are born out of all touch points including level 3 support, testing, and those raised by developers that should have been caught during continuous integration cycles.
  • Sprint Task Complete. This metric is similar to a sprint task burn down; however, it is a measure of how close was the team at completing tasks they signed up for. This measure did not include tasks introduced during the sprint – only those they signed up for. The challenge we find with this measure is that if a team is small, this number could be skewed. However, this metric clearly indicates either too much cross-winds hitting the team, lack of focus, or simply too large of stories/tasks.
  • Sprint Effort Complete. Effort relates to hours, so this gives a pretty good barometer of how close the team came to finishing the tasks. We measure this for two reasons – (1) smaller teams that may not complete but 1 of 3 tasks looked bad, but the reality was they finished 95% of the work and just need another day or two to get it across the finish line and (2) this allows for us to understand impact of estimates and cross-winds (a.k.a. support). The team still got recognized for the effort, but there was recognition that either through more focus or better break down of tasks may help complete what was committed.
  • Estimate Accuracy. This type of metric is not unusual in most traditional PMOs; however, the goal here was to drive the behavior of breaking down tasks more and not estimating something you didn’t understand. The result of this awareness helped the organization to build features consistently and follow best practices surrounding analysis and design. We also found it valuable with prototyping in that the team was able to break down the tasks in a fairly repeatable way and the estimates only got better during these periods.

How have these metrics worked out?

These metrics have served us well; however, some of these metrics can be frowned upon since they appear to be too focused on performance. But what we’ve found is that you adapt the metrics as the teams mature and the inevitable organizational change occurs. So for instance, Utilization was always a product of ideal hours – therefore, you generally saw 60-68% utilization for teams that performed project tasks and played a role in level 3 support. By plotting the utilization over time, we arrived at a median utilization rate of 67%. Once we felt this stayed level, we turned our focus to planned utilization – meaning are we planning our sprint capacity accurately and therefore filling up the buckets properly. We are an organization that uses story points; however, we’ve found velocity to be cyclical based on the phase of the project and possibly the opportunities brought forth by the business (a.k.a. several large deals sign and we find ourselves involved in implementations that are not managed via Scrum, thus velocity collapses). Nonetheless, now that we’ve moved to looking at planned utilization, we are able to establish a level of consistency and trust with the business that when we say we have X capacity and we can deliver by Y – the business is lock-step saying the same thing because they too see the metrics.

Lessons Learned

Well, we’ve learned that the data wasn’t useful in short cycles – the data was useful once you had five or six sprints worth of data points, then adjustments to process could be made. Also, by this time, the reasons you would expect things to go sideways were very clear there on the charts (e.g. defect rate went up prior to the release cycle – as airspeed is slowed on the feature development and the focus turned to technical debt). We also learned that tracking estimate accuracy and utilization altered the teams’ behaviors – and not in a good way. The team started finding ways to work the metrics in the favor. They did so even after being told that we weren’t measuring their performance based on these numbers, but using them for identifying opportunities for improvement and demonstrating successful delivery. We had folks that would sandbag their estimates, or underestimate and work late then log work for the exact hours they estimated.

We also learned the obvious and, frankly what we wanted to see, and that is that once a team started to understand the metrics – they matured as a team. Part of this was the equation of time and the improvement of the Scrum/Agile processes. The other part was the fact that we chose specific metrics to focus on – especially those surrounding productivity and quality. We started conducting group code review meetings and pairing up more on tasks that we deemed to have higher risk.

What to Look at Next

We need to start look at tracking business value better – looking at using earned value. The challenge we have is that we generally don’t put a cost on projects – we are a small software firm that generally runs with department budgets only and assess costs along the line of business only. It seems that we would make better decisions along the way if we kept cost and revenues in plain view during the development process.

Also, need to look at measurements surrounding quality automation (unit test coverage and LOC test coverage).

I’m sure there are other areas, but now that we have a good handle on consistency and trust in delivering what we said we would, it is time to focus on business value and look deeper into quality. Both of these will help make the products and the business more successful.

You might be interested in

Self-Organizing and Strengths Finder

in Agile Engineering/Product Management/Teams

Team WorkA key part of any good team, especially an agile team, is teamwork and more importantly a team of hardworking individuals that both care about their teammates and they understand the strengths of their fellow teammates.  In the sporting world you will often hear of situations where one player will attempt to play outside his strengths.  Or what is more prevalent, a player has an outstanding year due to either scheme or competition or simply a well formed team that emphasizes the strengths of the players.  Then that player is a free-agent and goes to another team, and the player’s career sags.  This is primarily due to familiarity, chemistry, and scheme that doesn’t fit the strengths of the player.

There is a concept that surrounds Scrum teams and that is the concept of “self-organizing” team.  Now this concept is often not clearly defined and depending on the Agile coach and or trainer, a team member might view this differently (see Does Self Organizing Team Imply Self Assembly?) .  Nonetheless, if we define self-organizing in the simplest form of a team that has the ability to take inputs and based on knowing certain attributes of their fellow team members, work is doled out, people are paired up, issues are worked out, team members are brought in and pushed out, and, in summary, things just start getting done and done effectively.

The “certain attributes” that a team uses for team self-organization often includes:

  • Team member workload
  • Technical background
  • Domain expertise
  • Team member availability
  • Understanding of the tasks-at-hand
  • Priority of tasks
  • Team member “likes”
  • Team member weaknesses
  • Team member strengths

Of course, all of these are important to consider; however, one would argue that the strengths are where all self-organizing should be focused.  As you guessed it, I’m the one to make this argument – and I encourage everyone who runs any type of team to take advantage of Clifton StrengthsFinder from the Gallup News organization (yes, that’s the same Gallup that does many politico and social polls).   StrengthsFinder came about from Dr. David Clifton – he is known as the father of Strengths Psychology.  The focus of StrengthsFinder is to help people identify their top five strenghts where they work best, understand these strengths, and see how focusing on using these strengths versus working on improving areas of weaknesses is much more productive, self-rewarding, and ultimately successful.

Some have grouped StrengthsFinder into the area of self-help or lumped it with personality tests such as Meyers-Briggs Type Indicator.  It is definitely not a personality test and if you did want to categorize it as self-help, you will find it to be much, much better and more effective than any other self-help book or tool.  When I was approached by a colleague that went through the StrengthsFinder exercise and he express the passion that this is for real and fulfilling, I was convinced I had to give it a go.  So after I took the 30 minute survey and got my results, I quickly learned and understood better why I say, do, and act certain ways.  It helped me rationalize and focus my energies within the areas of my strengths so that I can help my team and the company become more successful.  This didn’t mean I wouldn’t ever work on improving my weaknesses, but it did help me recognize where I can use my strengths to overcome or disguise my weaknesses.

So, you are now asking, why does this have anything to do with Self-Organizing agile teams.  Well, soon after going through the exercise myself, Dave (my colleague mentioned above) implored me to have our whole product development team go through the survey – we did.  After everyone had their top five strengths, we compiled the list and posted it on our team wiki.  We all sat down and discussed what we learned about ourselves and reflected on the strengths of others.

At the time when we did this exercise, we were about ten months into using a Scrum agile methodology.   We had earlier delivered a huge product release and we were coming off a really tough period where everyone was getting burned out by the product launch, supporting new customers, and having some frustrating issues largely due to incomplete quality processes.   Soon after experiencing StrengthsFinder, we saw that the team which would normally “mouse-up” during difficult team discussions, would start leaning on each other strengths.  This helped me as the team lead, to simply sit back and let the team work through challenges that when previously encountered, the team would simply brush under the rug.  Going through StrengthsFinder seemed to give the entire team more confidence and it helped those team members that would often simply focus on trying to improve their weaknesses to instead, embrace their strengths and take on the tasks that best suit them.   Not only would they have more self-confidence, but they also embraced the strengths of their peers.  When we would start new feature or enhancement development, the team members often signed quickly and they would ask a fellow team member that may have been stronger in another area to assist (e.g. using a person that has the Relator strength to interview subject matter experts).

Within two months after working through StregnthsFinders, we experienced measurable quality improvements and we moved closer to a self-organizing team where anyone on the team could lead within their strengths.

So if you are working within an Agile team and you are struggling with the “self-organizing” aspects or your team has hit a continuous improvement plateau, give Clifton StrengthsFinder 2.0 a try.  The books required can be obtained for less than $13 at Amazon or Borders.

You might be interested in

Five Ways to Establish Trust

in Agile Engineering/Product Management/Teams

As everyone knows, a key success factor of an agile process implementation, specifically Scrum, is establishing trust between the team delivering and those waiting on the delivery (basically between the product development team and the business).  In order to establish trust, a development team should be meeting their commitments made during the planning meetings; thus, delivering what they said they would.  Sounds like a pretty simple formula and for the most part — it is.

Never trust the teller. Trust the tale.
– DH Lawrence

Outside of simply delivering what you said you would in a sprint, here are five ideas to establish trust:

  1. Always act with integrity. It goes without saying, truth sets you free.  But what is more than simply being honest while communicating with someone, but it is more important to perform your job duties with integrity.  This means, no cutting corners, maintain the utmost quality, and always keeping the team informed when it comes to matters that may impact the core deliveries.
  2. Ask questions and share information. Make sure that if an idea is not understood, be sure to not be afraid to ask a question (sometimes, the first thing that comes out the mouth is “I don’t know”). Be sure not to assume you are all-knowing about an issue, so share what you know — this is an excellent way to facilitate conversation and potentially discover what you don’t know.
  3. Reasonably accurate expectations (estimates to complete). In the agile world, estimation can be challenging.  This is primarily due to the lack of data present at the beginning of tasks.  The easy way to overcome this and become more accurate with your estimation is to be sure to break down stories into common development tasks.  This may include an analysis, design, build out, unit test, etc type task.  I also like the proto-type, stub, or even the sub-functional task where a small piece of functionality is implemented.  Estimates generally get better the smaller the task and they are of course better when more than one person sets the estimate (leveraging the experiences of others will help).  Finally, always accurately update estimates to complete daily.
  4. Deliver within the boundries of the commitment. Simply stated, don’t always try to be a hero and deliver beyond the expectations of the original commitment. This can be viewed by management (be it product or development) as sand bagging estimates, or simply going rogue and “doing as you want to”. Not to say that finding a great way to do something that was much more productive is not a good thing or seeing that you can do a lot more within the context of the product functional area is not appreciated. But the right way to handle this is to disclose that you found a faster way to do something and your estimate-to-complete is much less than orginally anticipated, so you would like to work on another key capability that is similar to the functionality being implemented. Negotiating this within the context of the daily standup (and possibly within a follow up meeting) would be the correct course of action.
  5. Consistency is key.  Having one good sprint is not enough, a team needs to establish a trend of delivering on sprint tasks on a consistent basis.  This applies to the team as well as the individual.  Once a team is able to demonstrate a consistent trend surrounding capacity, story points delivered, hours worked, etc — the product stakeholders can start to reasonably expect that what the team says they will finish will in fact be delivered.

Remember the idea of trust should also exist within the scrum team and between scrum teams where there is a need to share deliverables or even if you all work within the same development organization. The five ideas above should be applicable to ensuring that all scrum teams work well together and within the overall scope of an organization.

What techniques or ideas do you have about how to build trust with the product management or project stakeholders?  What have you’ve done that works?

You might be interested in

Agile Software Manufacturing

in Agile Engineering

Early in my career, I was lucky to be able to work for a process manufacturing company that invented and produced engineered lumber products. Today I use many of my experiences on the manufacturing floor in my agile software development life. I know what you are thinking, how can you equate the manufacturing of engineered lumber products with development of software. Well you don’t have to look far to see that the whole product and software development communities have embraced efficient process manufacturing production line principles in the development of software projects and or products. This is a move from the world where building software was like building a house — lots of up front work regarding requirements and design, checkpoints and sign-offs along the way, and quality check offs at the end.

Scrum, Test Driven Development, Kanban, and eXtreme Programming — all of which are similar to the processes and best practices followed on a manufacturing floor. How are they similar? I’m glad you asked, over my next several posts I plan on exploring how they are similar and what we can learn from what transpires on the manufacturing floor.

Let’s first focus on what seems to be the most commonly implemented Agile practice – Scrum. I’m not going to get too much into what Scrum is – there are plenty of resources available for that and I’m sure one day that will be a good discussion for Agile.Agile.Agile.  So with Scrum, there are three areas which really stand-out:

  • Daily Standup (or Scrum) = Shift Production Meeting. All most all teams that practice Agile have daily meetings during which the team members answer the questions of “what happened since we last met”, “what is planned between now and the next time we meet”, and “what is preventing progress towards the goal”. Well these are the same questions answered during a shift production meeting at a manufacturing facility. They discuss the results from their previous shift, the happenings of the shift prior, planned production, and obstacles/challenges to meet the production goals. Much like the daily standup, issues are raised during the shift production meeting, but the discussion surrounding the actions to resolve are reserved until after the meeting and only those parties that can impact the resolution are involved. The primary reasons these meetings are similar are simple — it’s all about daily collaboration, team building, and ensuring the goals of the team are always in front of them.
  • Do what you said you would do. Well parents around the world have been saying this for a long time, but this is a core tenant of Scrum and working on a manufacturing floor. Actually, Scrum is about commitments and an agreement between the Product Development Team and the Product Owner (or Ownership Team). The crew that works on a manufacturing floor are committed to producing what was planned in the production schedule.
  • Plan in short periods that result in potentially shippable product. A key to meeting production goals is setting realistic goals that are derived from understanding production input constraints (raw materials and workforce) and an honest assessment of market conditions (understanding supply and demand of produced products as well as organization production goals). To do this successfully, planning is most effective when it is done within short time fences. Of course, all practicing agilistas know this to be true within most agile processes – and the sprint timeline is considered untouchable. This is because what was mentioned above – the team planned and committed to complete work within the specified sprint. It is also important to mention that using consistent short periods helps teams, production floor and or Scrum team, to get into a rhythm.

You might be interested in

1 … 3 4 5
Go to Top