I’ve always felt that I struggled when estimating projects. No, that’s not completely correct. The small tasks or “go do it” tasks, I can estimate those ones pretty well. What I have struggled with is estimating a big project that has many facets and is, of course, high priority and visibility.

TLDR: Estimating is only useful if it’s going to be used for some decision. Having developers estimate so that leadership can know when something is going to be released and then decide based on their gut feeling how effective the team is at delivering is not worthwhile. It’s important to know why you’re estimating and if instead, you can provide continuous improvements rather than work hard to an estimated release date. Agile has helped me to estimate and project plan by allowing me to abstract my estimates, but there can still be problems with hitting estimated release dates.

I’m going to talk about the progression of how I have learned to estimate and discuss those things along the way. My train of thought can be all over the place so I hope it will make sense. 

My First Experience Estimating

When I first started coding and was introduced to the idea of estimating, it was through the eyes and experience of the developers around me. When I asked for suggestions on estimating, they gave me decent advice I’m sure with the idea of looking at the task or project, thinking about what you have to do, and then making a gut call of how many hours it would take me to complete. The phrase that stands out from that initial conversation was

 “Once you have your estimate, double it!” 

Though this phrase does have merit to it, it still doesn’t hit on everything you will face when beginning to develop the project. If I think it will take 40 hrs, and then double that to 80hrs. How many days is that at work? How do I come up with an “estimated release date”? Do I think I work 8hrs a day and will get it done in 10 business days? No, there’s no way I wouldn’t be distracted or called to other things in those 10 days. So, then what about if you said “I think I do 80% of the workday as project work.” Okay, so let’s see, 8 * .08 = 6.4hrs. So 6.4hrs a day will be working on this project, and it should take 80hrs. 80/6.4 = 12.5 days. Okay great! I’ll tell them I can get it done in 2 weeks! 

Unfortunately, even though we think the project will take only 2 weeks, by telling a customer or leader that it will take that long, we get stuck in that number. What happens if you aren’t able to start the project right away? Could a bug get found and you are the developer to fix it? These things crop up and you have to adapt the estimate you said earlier. The customer will of course be understanding, at least the first or second time it happens. At some point, if you don’t get this task or project done when you said you would, people lose faith in you. This isn’t always accurate, but it’s something each developer can begin to think about when we focus on estimating and release dates. By not focusing on quality and only focusing on when the project gets completed, we begin to lose value for good code and start to value people getting projects done. 

An example of this that I have experienced shows up when a developer is ready to release their project. They put out a code review about a week before the release date or sometimes even shorter. Then, the expectation is that the reviewer gets it reviewed before that release date. And let’s say in an example, the developer puts the code review out on a Tuesday and wants to release it the next Monday. The reviewer is forced to review it the same day and try to complete it no matter the size of the project. If the reviewer has notes or things that should be fixed, the developer has only a short amount of time to fix them if they want to stay on target for their release date…Not to mention the testing that comes from making changes based on a code review. This can cause extra pressure and stress on both the reviewer and developer and does not facilitate good coding practices. It suggests that the “quick” way of doing things should be done instead of the right way or a more clever and efficient way. 

This way of doing estimates actually got me through my first couple of years at my job. We got projects or tasks and I could turn them around in a month or so. The customers were happy and we were progressing. No one seemed to mind that we’d say a date or an estimate then just end up giving status updates periodically. I ended up slowly stopping giving estimates and giving more status emails that mentioned where I was at and what I was doing at that time. It felt very relaxed and allowed creativity to try new things and learn how to make the project better. This could only work so long though, as the team I was on was getting noticed by higher-up leadership. 

Estimating Changes or Did I Open My Eyes?

I’m not sure if the change was gradual or sudden but to me, it felt sudden. I remember a moment in a weekly project planning meeting that caught me unaware. I had been working on a project for a while and had had some roadblocks throughout the development, there were external factors that I didn’t deal with correctly and it ended up adding time to the project. However, throughout this time, I was optimistic about the completion and I didn’t factor in the obstacles in the estimates I had been giving. I gave an update on where I was at and talk about needing to push back the release date due to some factors. The response from this was not what I expected. I had gotten used to people being understanding when it comes to long projects and unforeseen complications. This time, the response was to question why the dates had to keep getting moved and why the project wasn’t on track. I, unfortunately, got defensive somewhat and didn’t have any good answers which prompted an action to have a  meeting with the planning developer to go over my project and see how he could help get an accurate estimate. I understand it was not a way to try and humiliate me and suggest I’m not a competent developer due to having to move the release date multiple times, but it was something that I felt after leaving that meeting. This feeling stayed with me and woke me up to the change in the team’s priorities and focus. It made me wonder, has this always been there but I’ve gotten by because I’ve been lucky or a new hire?

That project was eventually released and life seemed to return to normal, but that moment stayed with me throughout the rest of my projects. I could start to see the questions and the way other developers started to talk about their projects. The focus switched to when will it get done, how fast can it be completed, will there be a risk to the release date and almost an apologetic reaction when someone had to move a release date due to things like time off or having to work on a support ticket. 

Whenever I am unsure how to handle a situation, I try to research how other people have handled it and try to see how to do the same. I know there are many smarter people out there that have struggled with the issues surrounding estimating before. My quest first led me to this video which many developers I am sure have run into before: #NoEstimates (Allen Holub) 

Basically, the video has Allen Holub describe the process of estimation and how it wastes the developer’s time and is not actually beneficial to the team. While a developer estimates, they could be out programming instead. Instead, the idea is to use projections and story maps to prioritize what needs to be done first. If we continue completing the most important tasks, we will always have something of value for the customers. This concept is interesting, but I didn’t know enough at the time to be able to successfully try to convince my leaders that we should throw out estimations. I remember showing my manager this video and though they thought it was interesting, it was not something that he wanted to try. So it’s back to the idea of needing to figure out a way to estimate again that will satisfy the leadership and the developer team. 

This video fueled my desire to not estimate though and I allowed myself to be drawn back into the normal routine and do my best to get projects out while failing my estimations. I realized that as long as I had a good reason for the delay, or I only moved the date a few times in the lifecycle, there were not as many questions or comments. This was due to everyone doing the exact same thing. It turns into a game of how many times you can get away with messing up your estimated release date before you were noticed. As you can tell, I was excited to go to the project planning meeting each week. 

My favorite time while doing any work is when I’m in the middle of a project and when I am in-between projects. When I’m in-between I can take time to research and learn if there are things I want to improve on from the last project. I decided in one of these moments to attempt to slay the dragon of estimating a project once again. 

There are many questions that I could lead to my thoughts and discovery. I want to try starting with the basics and that comes from asking questions. 

I set out to answer one question for myself, “how can I estimate better?” I didn’t think I could estimate very well and that moment from the project planning meeting in the past still came to my mind. I assumed there was something I was lacking rather than realizing the system itself is broken but that’s for another discussion. 

I knew that leadership wanted to have an estimated release date or some way of tracking a project. The team I was on had the capabilities to use Jira which is a software used for project planning and ticket tracking. We currently didn’t utilize it as much and so I looked into using it with estimating and project planning. It turns out Jira has a lot of articles that discuss this issue in great length. Here’s what I ended up coming up with from all of those articles. 

The new methodology that I wanted to begin using is called Agile. There are many different ways to use the Agile methodology and I will not claim that I follow it exactly. I enjoy the adaptations that can be made similarly to how one may take and adapt the coding standards from Microsoft to fit their own team.

Jira Stories and Epics

In Jira, there are different boards that can be used to fill with “issues.” Each issue can be designated as a different type. The new type of issue that I began to use was called a “story.” A story is basically a way of summarizing a requirement into a phrase that reads like: “As a {actor}, I want to {action} so that I can {benefit}.” This made a lot of sense to me because it grouped together a requirement for a project without making it too detailed and granular to lose the full picture. Inside the story was where a developer could add the more granular tasks such as “create this, update this build that” to keep track of what needs to be done to accomplish the story.

The other type of issue in Jira that I began to use was called an “Epic.” An Epic holds many stories together and groups the pictures together to form the backbone of the project. Some instances allow for something to hold together multiple Epics but in the Jira instance I had access to, this was not possible. 

The two new types, allowed me to structure my projects in a way I hadn’t realized I was missing. Now instead of breaking down tasks that were either too large or too small, it allowed me to group them by a picture, or a story of what the customer wanted. Not to mention it gave me access to the reports within Jira that could be used to show progress and track the project in new ways. 

Product Requirement Document 

With the new knowledge of how Epics and Stories worked, I was ready to begin a new project and get the requirements down. It turns out that Atlassian, the company that made Jira, has a template for creating an agile requirements document which in turn can connect with the stories in Jira. This is what I was looking for. After reading this document https://www.atlassian.com/agile/product-management/requirements, I was able to create requirements that are directly related to a Jira ticket. It was perfect in my head because people can see the progress on each requirement and it made the document come alive. 

The most important thing that the article mentioned is that with the document, the goal is to create a shared understanding between the stakeholders, developers, and all other parties associated with the project. The great thing with it as well as the flexibility of the document to be growing as time goes on which can allow for re-estimates and updates to the priority of the requirements. 

Story Points

Even though learning about the issue types allowed me to have a more focused and better understanding of a project, along with the help of the product requirement document, it still didn’t help me with getting to an estimated release date which I knew leadership wanted. The next step was to see how people estimated stories without using hours as I hadn’t had success with that estimation in the past. This introduced me to story points. 

For those who don’t know, story points are a huge topic that many articles discuss in length so I’m bound to explain it wrong in one way or another. This is how I view story points. Story points are a way to abstract an estimation for a specific story/task. The biggest things that are lost when someone estimates hours are complexity and uncertainty. With story points, you must take into account those two aspects and come up with the number. The goal is to get away from a set timeframe. 

Many teams will use a Fibonacci scale in order to show that estimates are not linear when it comes to uncertainty and risk. I think that works fine for now. So I began assigning those points to the stories I had just spent time creating and seeing how that looked in Jira. 

Will Sprints Help with Estimating?

The natural progression of Agile leads to finding a way to organize and focus the work. This led me to learn about sprints. Sprints are a set amount of time that the developers will put in stories they plan to complete in that time. Normally, the timeframe is two weeks as that gives enough time to get things done but not so long that the team isn’t getting regular feedback.

The goal for sprints is to forecast the work developers wish to complete. The more times you complete a sprint, the more consistent a developer can get when estimating with story points and what they can get done in a sprint. I’ve read that there are teams that have gotten very good at being consistent and knowing how much work they are able to get done in two weeks. 

Theoretically, using sprints may help me to be more consistent with my estimations.

Retrospectives

Something that relates directly to using sprints in development is the retrospectives that happen at the end of the two-week sprint. The retrospective is a meeting where the team inspects itself, including processes, tools, and team interaction. Typically, the team will go around and answer these questions: 

• What did we do well during the sprint?
• What could we have done better?
• What are we going to do better for next time?

Now, this meeting does not necessarily improve my estimation skills, but by having this meeting I can continue to make sure the most important tasks are getting completed and I can reflect on what I or a team member could be doing better in order to get more done or any other improvement that is thought of. 

Sprint Planning

This is another meeting that happens when using the Agile methodology. It’s important to plan out the sprint and decide what you want to get completed in it. The other benefit of sprint planning is that you can work with your team and see if they have a different perspective on the estimates you give the stories. It can be helpful such as if you don’t estimate something high enough, the other developers may mention that and have you be a little more accurate.

Getting an Estimated Release Date

Once you assign points to the stories, the focus circles back to the issue from the beginning; what is the estimated release date? Even though using story points abstracts the time it takes to complete a story, it doesn’t automatically absolve you from giving an estimate. This was my experience at least, and so I had to try coming up with a way to give an estimate that would be accepted. What I do now is trying to see how many story points I can accomplish in a sprint, take the total amount of story points from the project Epic, and do some math to come up with how many weeks it would take to complete the project. 

This calculation is not perfect and there’s more that needs to be thought through with it. Even though the story point estimations can abstract the estimate to some degree, it’s still important to factor in a contingency amount for when you may be on support or when something else comes up that takes you away from work. We as developers assume we can accomplish the same amount of work each week but it doesn’t end up happening for many reasons. 

Closing Thoughts

The final result of the new knowledge I’ve acquired with estimating has yet to be determined. At this stage, it has a lot of potential, but as I’ve read in other articles, there are still issues that can crop up when using Agile methodology. This still doesn’t solve the question of having a more accurate estimated release date and I am certain I’ll have to move back estimates in the future, but the change in mindset and the ability to track and abstract estimates in a way that I can understand and relate to make this process worthwhile. With the different way of doing the requirements document alone, I feel a lot better about project planning and tracking for future ventures. I will continue this journey to better estimations and hope to find a path that allows the flexibility and understanding that comes with less strict release dates.