Why Is It So Hard To Measure ROI?

When asking questions about what the Return on Investment (ROI) is for a project, regardless of whether it’s software development or data warehousing/business intelligence, people start to fidget.  The same folks who are passionate about the initiative, spent weeks on the requirements and are willing to fight a cage match to get their project to the top of the priority queue are suddenly at a loss for words when asked what the return is to the organization if this work gets done.

Some of the quotes I’ve heard to the question “What’s the ROI on this project/initiative once the solution is rolled out?”:

“There is no ROI”

“…and even if people can do their jobs faster we’re still paying them so it’s not like we’re saving the company money.”

“I don’t think we can put a dollar figure on it.  We just need to do it.”

“What do you mean?”

“Unless it’s something you can sell, it’s too hard to measure ROI on technology projects.”

Quite frankly, it is rare to find a technology project that has the ROI clearly stated that is measurable, believable and agreed upon.

However, there is no hesitation at marshaling a small army of very expensive resources, along with the infrastructure to support the effort, and lock them away on a project for three months or six months or longer.  Total costs for an initiative go well beyond a project time line, and it isn’t hard to crest the $1M mark in Total Cost of Ownership within a year or two even on seemingly small projects.  That’s one heck of an opportunity cost.

I am a firm believer that anything worth doing is worth measuring, but often it’s not.  What’s worse is that the success of most projects is measured as the combination of three things:

1. Was it completed?
2. Was it on time?
3. Was it within budget.

While those are important measures, they really don’t matter if the organization sees no value from the investment.

I’m not sure if there’s just one major hurdle that everyone has problems clearing when it comes to ROI or if it’s different for everyone.  These are some of the mental barriers I’ve seen in people :

• Some have a hard time with dollars and cents- they just don’t think that way.
• Math scares them and all they can think about is how much they hated figuring out the Internal Rate of Return (IRR) and Net Present Value (NPV) problems for their Finance course in college
• People are overly cautious and don’t want to over promise
• The value that their department, or functional area, provides to the company is not very clear
• They assume it’s the cost of doing business so no ROI needs to be computed
• No revenue will be generated from the work and that’s all they think ROI is
• There was never a need to define and argue initiative ROI so they really don’t know how

What I always find fascinating is that, even though ROI is rarely given and most people claim it’s too hard (if not impossible), everyone knows what the ROI is for a project.   They just don’t know they know.  All you need to do is help them uncover it.

“I saw the angel in the marble and carved until I set him free.”  – Michelangelo

In subsequent posts I’ll share with you the chisels that I use.

The #1 Reason BI Isn’t Getting To That Next Level

The quality is bad.

Interesting quote by John Thompson of Kognito:

‘[Executives] are saying, “Are you certain that syndicated sales data and marketing data from last year is high quality data I can use to make decisions?” They’re still asking about that.’

When talking about the quality of any system there’s two parts to it:  real quality and perceived quality.   The prior are things you can eyeball, create QA/QC test plans around, foot to baselines, regression test and hand off to your development team to fix.  This type of quality is what most technology teams focus on.

Perception, however, is more powerful, influential and much-much-less scientific.   If someone doesn’t believe in the quality of your data, or your team, then it really doesn’t matter if the data is correct or not.  Often the perception is well warranted at some point.  However, once the real quality is corrected, that perception lingers.  What needs rebuilt at this point is your reputation and that doesn’t happen in a day.

You need to work harder at changing your image, and it needs to be at the top of the list.  If people don’t have faith in the quality of what you and your team do then you’re not going to be able to move forward with the things you really want to do and the business really needs.

Quality is job #1.  Nail that before moving onto the more innovative initiatives.  It’s your dial tone.

How To Be An Expert In One Easy Step

In the old days the title of this post would be RTFM.

The Help menu is there for a reason and Jeff Smith writes a great piece that explains everyone can be an “expert” if they just used the Help feature built into the applications, systems and tools they use.  The information is there and, with often very little effort, the information is easy to find.

These are my experiences on why people rather go to the “experts” rather than building their knowledge themselves.

• Intimidated by technology
• Don’t have time to learn something new
• Don’t have the capacity to learn something new
• Easier to ask someone else to figure it out
• Faster to ask someone who already knows or who can find out quickly
• Delegation
• Generational differences
• Personality types - some people rather deal with people than with computers 
• Hierarchical in nature – not my job 
• Laziness

Albert Einstein was quoted as saying “I never memorize anything I can look-up.”  The key is being able to find it when you need it so you can apply it when you need it.  It’s a critical skill for any career and even more so for those in technology.    It’s one of the top skills I look for when I’m interviewing.

Brief Bits: Linked Lists of Lists Patented

From URL:

Over at Slashdot it’s being reported that, finally, someone has patented linked lists.  My favorite comment in the wild and wooley discussion that’s going on over there is by tedgyz who states “I would show the prior art, but I can’t read the reel-to-reel tapes.“

And, following that linked list theme, over at Coding Horror is a top list of top programming lists.  Maybe Jeff should patent his post before someone else beats him to it.

Brief Bits: Schedule Chicken, A Cup of Joe and Predicting the Future

URLs that came across my browser today. 

If you’ve been in a number of projects you’ve probably seen the game known as Schedule Chicken. Funny but true.  Peter Clark talks about the meaningless of percent completion (“we’re 90% of the way there”) and the need to be willing to deal with project challenges transparently and head-on.  Found the link from Dare Obasanjo’s blog post on the Top Ten Signs Your Software Project is Doomed.

Next on the list – I don’t know about you, but I’m always worrying if I’m getting my daily requirement of caffeine.  Now there’s a tool that can help.  Energy Fiend has a Caffeine Database that allows you to not only see what the caffeine content is in virtually every beverage out there, but it also allows you to calculate how much you’ve had.  I’m somewhere between “OK. That’s enough” and “Ever hear of moderation?”  Link courtesy of LifeHacker.

Once you’ve got all that extra energy to spare, head over to Andrew Moore’s Statistical Data Mining Tutorials.  This is awesome material.  These are all PDF’s of slides from lectures he’s given.  Normally slides are not a great learning device without the presenter, but these stand on their own very well.  He’s working for Google now and recruiting for the new office location in Pittsburgh, PA. Tip – always try to work with people that are smarter than you.  If I were still back in the Burg I’d be begging him for a job.

Is The Object Oriented DBMS Making a Comeback?

Hardly.

Ted Neward is stumping for the object oriented database management system (OODBMS) db4o.  While I can understand his frustration as a Java programmer struggling with wrapping calls to a RDBMS, there are a number of reasons why OODBMS like db40 will not cut it for production systems.   The two big ones are accessibility and performance.

Accessibility

Or, more accurately, the lack of it when using an OODBMS.  On accessibility of data from other programs…

Well, that depends. If by “other programs” you’re referring to other Java code, then simply use the definition of the Person class in those other programs and pass them into the ObjectContainer just as I’ve done here. In an OODBMS, the class definition itself serves as the schema, so no other tools are necessary to fetch the Person objects. If, however, by “other programs” you mean other languages, then the story is trickier.

Exactly.  Every RDBMS system on the market enjoys virtually ubiquitous access no matter what language, or application, you happen to be using.  The data are the assets of an organization and it makes no sense to restrict it.

Performance

A well designed database in any of the main stream (including open source) RDBMS should blow the pants of an OODBMS.

db4o is more on scale with a MySQL

I seriously doubt that. 

I’m not sure what he’s basing this on so I checked out the benchmarks section of db40′s site.  They reference a PolePosition open source database benchmark that shows db40 beating out MySQL in various circuits.  All I find there are the results of the various tests, but nothing about the data that was used, what kind of configuration was used for each system and what kind of tuning was done.  When I downloaded the source for the benchmark I couldn’t find the datasets in the zipped file either.

Without that information I can only assume that the tuning of the MySQL database (and all the rest for that matter) was negligible.  A benchmark is worthless without knowing this info so these numbers are meaningless at best and misleading at worst.

I won’t even go into the agruments of adoption, encapsulation (that’s right – encapsulation), scalability, availability and reusability. 

UPDATE:  Found this thread on Slashdot that has some great points made by both sides of the aisle.  That link only shows the highest rated comments to weed out the noise.  Even though it’s from ’04 it’s still relevant.