Why Is It So Hard To Measure ROI?

When asking questions about what the Return on Investment (ROI) is for a project, regardless of whether it’s software development or data warehousing/business intelligence, people start to fidget.  The same folks who are passionate about the initiative, spent weeks on the requirements and are willing to fight a cage match to get their project to the top of the priority queue are suddenly at a loss for words when asked what the return is to the organization if this work gets done.

Some of the quotes I’ve heard to the question “What’s the ROI on this project/initiative once the solution is rolled out?”:

“There is no ROI”

“…and even if people can do their jobs faster we’re still paying them so it’s not like we’re saving the company money.”

“I don’t think we can put a dollar figure on it.  We just need to do it.”

“What do you mean?”

“Unless it’s something you can sell, it’s too hard to measure ROI on technology projects.”

Quite frankly, it is rare to find a technology project that has the ROI clearly stated that is measurable, believable and agreed upon.

However, there is no hesitation at marshaling a small army of very expensive resources, along with the infrastructure to support the effort, and lock them away on a project for three months or six months or longer.  Total costs for an initiative go well beyond a project time line, and it isn’t hard to crest the $1M mark in Total Cost of Ownership within a year or two even on seemingly small projects.  That’s one heck of an opportunity cost.

I am a firm believer that anything worth doing is worth measuring, but often it’s not.  What’s worse is that the success of most projects is measured as the combination of three things:

1. Was it completed?
2. Was it on time?
3. Was it within budget.

While those are important measures, they really don’t matter if the organization sees no value from the investment.

I’m not sure if there’s just one major hurdle that everyone has problems clearing when it comes to ROI or if it’s different for everyone.  These are some of the mental barriers I’ve seen in people :

• Some have a hard time with dollars and cents- they just don’t think that way.
• Math scares them and all they can think about is how much they hated figuring out the Internal Rate of Return (IRR) and Net Present Value (NPV) problems for their Finance course in college
• People are overly cautious and don’t want to over promise
• The value that their department, or functional area, provides to the company is not very clear
• They assume it’s the cost of doing business so no ROI needs to be computed
• No revenue will be generated from the work and that’s all they think ROI is
• There was never a need to define and argue initiative ROI so they really don’t know how

What I always find fascinating is that, even though ROI is rarely given and most people claim it’s too hard (if not impossible), everyone knows what the ROI is for a project.   They just don’t know they know.  All you need to do is help them uncover it.

“I saw the angel in the marble and carved until I set him free.”  – Michelangelo

In subsequent posts I’ll share with you the chisels that I use.

What Does IT Project Management and Jello Have In Common?

The ABCs of IT Project Management over at CIO Magazine starts off with a great intro:

Managing an IT project is like juggling chunks of Jell-O: It’s neither easy nor pretty.

It’s a good article on getting your arms around what project management is, what the challenges are and some approaches to help make sure you are more successful.  Although it does prepare you for the worst before you get too deep in the article.

According to a cited study, only 29% of IT projects are completed successfully, and the reasons for those that do fail can be traced upstream.  How do you know if you’re project is among the 71% unlucky ones?  Here’s a rule-of-thumb metric for determining if a project will fail:

…use an indicator such as the “15-15 Rule.” The 15-15 Rule states that if a project is more than 15 percent over budget or 15 percent off schedule, it will likely never recoup the time or cost necessary to be considered successful.

While the tone set is definitely gloom and doom there are some constructive pieces on how to help ensure the project is positioned, and managed, to succeed.  Overall a good read if you are either new to project management, looking for paths to improving how your projects are managed or just want vindicated for what’s been keeping you up at night.

How To Be An Expert In One Easy Step

In the old days the title of this post would be RTFM.

The Help menu is there for a reason and Jeff Smith writes a great piece that explains everyone can be an “expert” if they just used the Help feature built into the applications, systems and tools they use.  The information is there and, with often very little effort, the information is easy to find.

These are my experiences on why people rather go to the “experts” rather than building their knowledge themselves.

• Intimidated by technology
• Don’t have time to learn something new
• Don’t have the capacity to learn something new
• Easier to ask someone else to figure it out
• Faster to ask someone who already knows or who can find out quickly
• Delegation
• Generational differences
• Personality types - some people rather deal with people than with computers 
• Hierarchical in nature – not my job 
• Laziness

Albert Einstein was quoted as saying “I never memorize anything I can look-up.”  The key is being able to find it when you need it so you can apply it when you need it.  It’s a critical skill for any career and even more so for those in technology.    It’s one of the top skills I look for when I’m interviewing.

Brief Bits: Schedule Chicken, A Cup of Joe and Predicting the Future

URLs that came across my browser today. 

If you’ve been in a number of projects you’ve probably seen the game known as Schedule Chicken. Funny but true.  Peter Clark talks about the meaningless of percent completion (“we’re 90% of the way there”) and the need to be willing to deal with project challenges transparently and head-on.  Found the link from Dare Obasanjo’s blog post on the Top Ten Signs Your Software Project is Doomed.

Next on the list – I don’t know about you, but I’m always worrying if I’m getting my daily requirement of caffeine.  Now there’s a tool that can help.  Energy Fiend has a Caffeine Database that allows you to not only see what the caffeine content is in virtually every beverage out there, but it also allows you to calculate how much you’ve had.  I’m somewhere between “OK. That’s enough” and “Ever hear of moderation?”  Link courtesy of LifeHacker.

Once you’ve got all that extra energy to spare, head over to Andrew Moore’s Statistical Data Mining Tutorials.  This is awesome material.  These are all PDF’s of slides from lectures he’s given.  Normally slides are not a great learning device without the presenter, but these stand on their own very well.  He’s working for Google now and recruiting for the new office location in Pittsburgh, PA. Tip – always try to work with people that are smarter than you.  If I were still back in the Burg I’d be begging him for a job.

How To Be a Successful Project Manager – Part 3

Here is the final installment on ways to make sure you are firing on all cylinders as a project manager.  Looking back on these postings I realize that I’m really just touching the surface on these topics.  Each could easily have warranted a separate post each to provide a deeper dive.  The reality is managing software projects, and the teams that make them successful, is not something that can be summed up in a simple step-by-step recipe. 

Sometimes you’re on offense.  Sometimes you’re playing a zone defense and sometimes you’re playing man-to-man.  Hopefully this series will keep you out of trouble so you don’t find yourself throwing up a Hail Mary!

Without further delay, these are the final four:

Challenge #7 – Encourage the sponsor to approve deliverables informally (with nods, smiles, and verbal praise); never force sponsors to stand behind their approvals with a formal sign-off. (In other words, give ‘em plenty of room to weasel out of agreements!)

My experience is that sign-offs are never a done deal.  It doesn’t matter if it’s a hand-shake or a document signed in triplicate.  What will make a deliverable successful or not will depend on what you did upstream in the preparation and initial requirements and midstream as you kept the project on track and inline with the business needs.  If your requirements up front are fuzzy and your agile methodology is more like dancing in concrete boots then those deliverables are likely not going to go over too well.

Challenge #8 – Make sure project managers have lots of responsibilities and deadlines, but no authority whatsoever to acquire or remove people from the project; to get enough money, materials, or facilities; or insist on timely participation of SMEs and key reviewers.

If, as a project manager, you have no control over resources whatsoever then you are in a bad situation.  Getting thrown a project that’s already been scoped for you requires you to start using your communication and negotiation skills to get what you need to get the project done.  If that doesn’t get you anywhere, your task in this situation is to raise the flags on the risks and raise them often. 

That’s not a fun place to be.

A key part to being a project manager is scoping out what it’s going to take to get the job done. Being in this situation, the first place you need to look at is your ability to adequately map out the resources for a project.  You don’t have to know it all.  You do need to know the right people to interview and the right questions to ask.  It’s more important to know what you don’t know.

Challenge # 9 – Describe project deliverables in the vaguest possible terms so sponsors and reviewers have plenty of leeway to reinvent the project outputs repeatedly as the project unfolds.

This is related to #5 and #6 from Part 2 of this series and #7 above.  There’s nothing wrong with going into a project with less than crystal clear requirements if you have a good agile development methodology in place.  Re-read those parts of the series for more information.

Challenge # 10 – Get projects up and running as quickly as possible – don’t worry about documenting agreements in a formal project charter, clearly describing team roles/responsibilities, or doing a thorough work breakdown analysis. After all, we know what we’re doing and we trust each other. So let’s get to it without a pesky audit trail!

Defining roles, responsibilities and escalation procedures are critical to getting things done a project.  If you are already knee deep in the schedule without this then it’s time to establish them pronto.  Don’t assume you’ll work it out as you go.  At the very least you’ll need to establish who will give the go/no-go decision at the sponsor level and who the key decision maker is for each of the major swim lanes in your project.

Key decisions need to be made in a forum where everyone understands that a decision has actually been made.  E-mails by themselves are not a good choice.  These should be done in person or on a conference call, with an e-mail follow-up and documented in a formal document that shows the decision has been logged.  This is more to make sure the information is communicated than to hold anyone’s feet to the proverbial fire.

…and that does it for this series.  As I mentioned earlier I didn’t spend nearly enough time on each of these as they are warranted.  There is a common theme to many of these answers though – managing technology projects is about managing people, not technology.  All the tools in the world won’t help you if your soft skills aren’t up to par.

How To Be a Successful Project Manager – Part 2

This is part two in the three part series on how to improve your project management success based on the post by Michael Greer on 10 Guaranteed Ways to Screw Up Any Project. The goal is to give you tools and perspective on how to handle these situations when they come up.

Part 1 focused on challenges 1 -3. This part is focused on challenges 4, 5 and 6…

Challenge # 4 – Interrupt team members relentlessly

The project team is being nickled-and-dimed by tedious tasks and wild goose chases. At it’s worst, team members are expected to respond to e-mails and phone calls while they’re “off.” This isn’t during the occasional time during projects where extra effort has to given, but a constant trickle that means leaving work never means leaving work. The constant needling means the development team is going to have a hard time focusing and, if it happens beyond work time, they never get a chance to recharge.

As a project manager there are several ways to deal with this:

• Make sure the roles and chains of communication are clear. If there are not clear proxies setup (i.e. stakeholders should discuss issues with project or development managers and not the developers themselves) then create them.
• Create a structured forum where these issues can be aired, document them and prioritize them. Make it clear that if they aren’t in an issue log with a priority assigned then it doesn’t get done.
• Be the envoy for those that are being drubbed and talk with the offending parties to get it to stop. Part of project management is diplomacy, negotiation and persuasion.
• Raise the workload as a risk to the project on the status reports and make it clear how it threatens the project.
• Make sure you aren’t enabling this behavior by building in unnecessary work, expecting 60 hour work weeks or having a schedule that’s aggressively unrealistic.

If you are a development manager it is your responsibility to coach your team how to manage up and be the shield if needed.

Challenge # 5 – Create a culture in which project managers are expected to “roll over” and take it when substantive new deliverables are added halfway through the project.

The longer it takes for the deliverables to be, well, delivered the more likely the scope is going to change. And there are always going to be stakeholders, sponsors and influencers that continue to come up with features that are “must haves.”

Fact is that you are never going to get all the requirements and features documented adequately up front. People really don’t know what they want until they see it. They may have a conceptual model, but as you start making strides in the project the viewable progress will trigger “a-ha’s” that result in scope creep.

Plus, the business, industry and market keeps moving. Keeps changing. This means a nice-to-have can get promoted to a top priority and vice versa.

This is what agile development methodologies are meant to resolve. If you are not familiar with agile, scared of agile, what might fit and/or what might not, check out a two part series. Here’s Part 1 and Part 2. What’s nice is that at the end of the second part there is a nice series of tables that show pros/cons and high-level snapshots like these two:

Table 2. Methodology Comparison

Condition	XP	Scrum	Lean	FDD	AUP	Crystal	DSDM
Small Team	v	v	v	X	X	-	v
Highly Volatile Requirements	v	v	v	v	-	-	X
Distributed Teams	X	v	v	v	v	X	X
High Ceremony Culture	X	X	-	-	v	-	v
High Criticality Systems	X	-	-	-	-	v	X
Multiple Customers / Stakeholders	X	v	v	-	-	-	X

Table 3. High-Level Methodology Description

Methodology	Summarizing Phrase
XP	Simplicity
Scrum	Prioritized Business Value
Lean	Return on Investment (ROI)
FDD	Business Model
AUP	Manage Risk
Crystal	Size and Criticality
DSDM	Current Business Value

Bottom line is that project managers should anticipate changes. I once heard it put best that the ability to add requirements into a project mid-stream translates into a competitive edge. As I mentioned in Part 1 of this series, project teams need to be able to anticipate what the business needs before they realize it themselves. After all, the technical team knows all the great things that are possible with technology, which is way more than most business types know. Add a little bit of business perspective to the technical team.

Challenge #6 – Half way through the project, when most of the deliverables have begun to take shape, add a whole bunch of previously unnamed stakeholders and ask them for their opinions about the project and its deliverables.

Expect this one. If the project is a big one, people will be jumping on the wagon. Success will bring them too. Shifts in influence and strategy can also have added team members in the mix. The goal is to try to uncover them early in the process. They won’t be handed to you on a list titled “The People You Never Saw Coming.”

I wrote about how to manage for this risk before.

OK, only one more part to go in the series and that’s a wrap. Keep an eye out for it.