Curing the Rampant Waste in Product Innovation

Why So Many Product Innovation Efforts Fail to Achieve Their Objectives

“We lack a focused, cohesive vision for our company. We want to do everything and be everything — to everyone. We’ve known this for years, talk about it incessantly, but do nothing to fundamentally address it. We are scared to be left out. We are reactive instead of charting an unwavering course. We are separated into silos that far too frequently don’t talk to each other. And when we do talk, it isn’t to collaborate on a clearly focused strategy, but rather to argue and fight about ownership, strategies and tactics. … I’ve heard our strategy described as spreading peanut butter across the myriad opportunities that continue to evolve in the online world. The result: a thin layer of investment spread across everything we do and thus we focus on nothing in particular.”

– Brad Garlinghouse, SVP at Yahoo! in a November 2006 internal memo leaked to the Wall Street Journal and later dubbed “The Peanut Butter Manifesto” (*)

“50% of all advertising is useless. We just don’t know which 50%.”

– David Ogilvy, legendary advertising executive and co-founder of Ogilvy and Mather

Product innovation is both a mantra and a mystical term. It is supposed to occur in one main organization – R&D, or Engineering – but also occurs in subsidiary areas such as product management, product marketing, professional services, outsourced/offshore developers, external partners, and sometimes even in internal end-user departments such as finance, legal, marketing, sales, or support. Unfortunately, the overall track record for innovation efforts in most tech companies is a spotty one, with horror stories everywhere of failed or failing efforts among the many development projects that do end in helping each company succeed.

Product Innovation – A Question of Strategy

As so often is the case when talented and dedicated people fail to achieve their objectives despite all their creativity and hard work, I think you have to start by assuming that everything starts with strategy – or lack thereof – because developers generally don’t fail through lack of interest or effort. What I most often see is unclear or undefined market and product strategies that cause management teams to flail in their attempts to cover all bases, participate in every much-hyped or fast growing product category within their grasp (or their mission), and/or try to fulfill every imaginable type of customer request. Very often engineering take on the challenge “because we can”. Or, sales, marketing or other functions suffer acute anxiety not to lose out to their closest competitors and bully product management and engineering into chasing after them at all costs; or even, in the case of young startups, because their CEO, venture partners or board members egg them on to jump in to new spaces wherever possible to attract new customers and increase the company’s valuation.

This phenomenon is happening every day among companies that are casting their fortunes in hot areas such as cloud, mobile, big data, cyber security, work management, collaboration, and IoT. What’s needed in these situations is much more stringent prioritization. And prioritization is virtually impossible without strategy – which as much as anything is about deciding what not to do as well as what to do.

Peanut Butter & Waste – Root Causes

The reason why the peanut butter image caught on so quickly in the industry and still resonates with management teams today is that it describes exactly the problem that afflicts many companies. Fast-growing young companies nurture big ambitions and even bigger appetites, and they fear being left behind by leaner meaner competitors if they don’t pursue every opportunity in front of them. The faster their growth, the harder they struggle to resource every opportunity; and if their growth stalls, as has been the case with former high flier Yahoo! for so many years now, their cumulative and collective anxiety drives them even harder to do a bit of everything in the attempt to win somewhere. Even under Marissa Mayer, CEO since 2012, Yahoo! appears to have been thrashing around, first recasting itself as a web content publisher and now emphasizing its role as a budding leader in mobility, through fifty or so acquisitions in less than 36 months.

Especially if companies have stalled and thus been lean on resources for a few years, they never have sufficient developers to pursue every product innovation opportunity successfully. This leads to “bubble-up innovation”: myriad attempts in various parts of the company to catch up, keep up, get ahead, fix bugs, get some custom code or scripting into the product to keep customers from mutinying or going elsewhere. The problem is that the more frenetic the action in different functions and divisions, the less effective the outcomes. In other words, what they get into is an extremely vicious cycle from which it is difficult to escape.

As for David Ogilvy’s comment about the waste in advertising, if he had been around to watch the evolution of the tech industry he could easily have made an identical claim about the waste in product development. With advertising, I guess we can all understand some of the vagaries that advertisers and agencies have to deal with; after all, even armed with today’s diverse targeted advertising technologies, how can you know for sure who among a mass of consumers will notice your ad and then take action to purchase your product as a result? Surely engineering, a discipline founded on using mathematics to produce precise answers to thorny problems, when armed with at least some amount of market research and knowledge of customer requirements, should be more predictable and more effective in its use of resources? Well, it turns out that software engineering can often be virtually as much of a black art as advertising, marketing, and sales seem to be to those who work in other fields.

In a nutshell, peanut buttering results from a lack of strategy (= prioritization) and leads directly to waste, in fact sometimes astronomical amounts of wasted resources. This problem is not made easier by the relative lack of transparency in product development organizations. Often it’s almost impossible to identify the waste and reduce or eliminate it without carrying out a forensic examination of every so-called innovation project. Companies routinely plunge into ill-defined development projects in the attempt to make up lost ground against their competitors, or throw an immature new offering over the wall to their marketing and sales teams. Sometimes they try to address the demanding custom requirements of a few major customers without every managing to develop a repeatable solution to solve mission-jeopardizing problems of any one group of target customers. Another frequent sin is sticking with products that almost everyone in the company acknowledges should be end-of-lifed except that there’s a small but vocal minority of customers or salespeople who can’t let go of them just yet.

So, how can we put some order in this chaos? Why is this problem not just about tactics (i.e., which development tools or techniques to adopt, how to obtain the right skillsets to apply to each project, etc.), or execution (i.e, managing projects tightly to get teams to complete development projects on time, on spec, and on budget)?

My vote goes to strategy, first and foremost. Once we get the strategy right, the talents and hard work of engineering teams can take over. Despite the rapid growth in today’s tech markets there is no global lack of dedicated, gifted, and experienced engineers, product managers, and executives to drive for results once they have a clear direction and guideposts to follow.

The Three Types of Innovation Investment

First, we need to agree on what we mean by innovation. For starters, when left to their own devices engineers and engineering-centric organizations tend to intuitively equate innovation to differentiation. “If we produce this innovative product or feature on time, our customers will buy our product/service, and our revenue growth objectives will be fulfilled”. However, our empirically developed models tell us that, perhaps not surprisingly, innovation is a multi-faceted activity. It turns out that, in business terms, there are actually three types of innovation investment that can result in positive outcomes, all of them requiring a distinct, different strategy.

These three innovation types are Differentiation, Neutralization, and Optimization (AKA Productivity). They each have their unique characteristics and resourcing requirements:

1. Only one of these three, Differentiation, creates competitive separation in the minds of customers (and partners), and thus leads to premium pricing, preference (i.e., winning more deals than your competitors), and higher growth and/or margins than your closest competitors.

2. Of the other two, Neutralization is the type of innovation that enables you to overcome competitive disadvantage, allowing you to stay in – or get back in – in game, however fleetingly. It’s important to keep in mind that as soon as you (think you) have neutralized a reference competitor’s advantage, they may already have launched, or be about to launch, another version or new offer that puts you behind again. So you need to always be ready to launch new neutralization efforts, as the need crops up.

3. Optimization, the third innovation investment, provides for cost reduction which allows you to either lower your prices or raise your margins, or a combination of both. An example of an optimization innovation may be the use of new open source technology to complete a new version of an existing offer more quickly and inexpensively than if you had continued to use your existing tools and approach to the project.

Taken together, these three innovation goals account for no more than 50%-65% of most tech company R&D expenditures. That said, if waste can be brought under control, the situation can be significantly remedied. One important note is that deciding which innovation investments are actually differentiating as opposed to neutralizing or optimizing (or none of the above) requires an important strategy exercise unto itself, not to be neglected. But this topic is for another day.

The Damage Wreaked by Waste

There is a green-headed monster called Waste which often accounts the remaining 35%-50% of total development expenditures throughout and outside the company. Discounting a reasonable percentage of legitimate Failed Experiments, which are the only benign form of waste, you might find that all the malignant forms of waste account for “only” 20%-35% of total development expenditures. How come such a significant slice of the pie?

When you consider the following four criteria as major contributors, you will quickly see how waste can become such a significant drag on a company’s innovation resources and effectiveness:

• Differentiation projects that don’t go deep or far enough, and either just achieve neutralization or miss that outcome too. This can be extremely costly to the company in terms of opportunity cost because the customer preference and premium margins will never be realized. Since differentiation projects typically require a concentration of quality resources, failing to achieve the objectives also costs heavily in terms of opportunity cost.

• Neutralization projects that go beyond good enough, but take too long to do so. This occurs with great frequency because some or all of the engineers on the project – and maybe their product management colleagues as well – are misguidedly thinking that they will succeed in differentiating their offer against those of their closes competitors, when no such possibility exists.

• Optimization projects that fail to cost-reduce development. It is painfully common in tech that optimization goes neglected or even completely ignored, especially in younger companies where everyone wants to believe that every innovation effort is aimed at achieving differentiation.

• Bubble-up innovation projects that end up creating (sometimes massive) redundancy, and/or never coming to fruition. These typically exist in multiple functional areas of every tech company, and the waste is often enormous though very hard to calculate accurately.

Organizing Resources for Multiple Innovation Investments

So far so good, in terms of description; but what does this tell us about why companies struggle so mightily to manage their development projects to successful outcomes, as measured in customer adoption or preference, premium pricing and margins, customer retention, and so on? As implied earlier in this post, one of the main misconceptions among engineering teams is to treat all innovation efforts as differentiating. After all, the desire of every engineer is to work on “cool” projects, by which one might infer that the feature or product they are working on is bound to differentiate the company’s offering(s). But one secret of the R&D business is that not all engineers are ‘inventors’ focused on new creating forms of differentiation from scratch; many more are ‘deployers’ whose best work is performed when adding new features that result in improving their products dramatically or incrementally; still others are more talented at ‘optimizing’ spaghetti code, adopting new tools and techniques, or simplifying a code base that has become top-heavy and unwieldy for maintenance purposes.

Because so many management teams don’t have a model that distinguishes in this way between innovation investment types, the next mistake is to ask each development team to pursue multiple innovation objectives in the same project. In other words, ask the team to differentiate, neutralize, AND optimize all in the same body of work. Bad Idea!

The organizational strategy that we have found to work best to avoid waste, and instead succeed in enabling differentiation where it is feasible to do so, while also neutralizing and optimizing effectively, is actually to separate teams by their different innovation objectives. Most often this requires that you deconstruct large projects into their component sub-projects based on which parts are differentiating versus neutralizing or optimizing. In other words:

1. Teams working on a Differentiation (DIFF) sub-project will be asked to go as deep and far (and sometimes as long) as needed to achieve the differentiation they are seeking;

2. Teams working on Neutralization (NEUT) sub-projects will be tasked with getting to ‘good enough’ quickly and economically, and at all costs not to go beyond that;

3. Teams working on Optimization (OPT) sub-projects will focus on employing continuous improvement tools and techniques – such as open source software components – to continually cost-reduce the company’s code base and offerings.

Call to Action

Getting the DIFF / NEUT / OPT mix right and thus significantly reducing Waste can lead to astounding returns on innovation for companies. The reality is that, if you deconstruct all of the development/engineering projects in motion in the typical tech company, you are likely to find that more resources are aimed at achieving Neutralization than Differentiation, with the smallest percentage of resources allocated to Optimization work. The good news is that no single company generally needs to differentiate in more than one or two vectors – whether in specific key capabilities or functionality – where they genuinely possess crown jewels in potential or in actual form.

But in order to attack this major problem, light needs to penetrate the dark caverns of engineering, and lessons need to be acted on decisively. This requires that the R&D executives and product managers expose their black box secrets to other functional groups in the company. Otherwise the problems that Brad Garlinghouse described at Yahoo! in 2006 and that persists in so many companies today – multiple siloes and very little collaboration, proliferation of different types of development organization (new product engineering, sustaining engineering, professional services, outsourced but uncoordinated offshore efforts, etc.), lack of ownership and accountability for product innovation outcomes between different functions such as engineering, product management, product marketing, sales, pre-sales, and PS, all resulting in unfocused efforts, unfinished projects, and exploding development costs – will continue for many more years. A true, widespread peanut butter nightmare.

Among present-day Saas, Paas, and Iaas companies I know many where product innovation efforts are spread throughout the organization and outside its four walls, among myriad offshore organizations as well as other development partners. In most of these situations, no distinction is made between differentiating, neutralizing, and optimizing innovations. As a result, massive waste is occurring on a daily basis. The main reason that it escapes closer scrutiny is that engineering is so often a powerful and relatively unquestioned constituency and executives in other functions feel unsure about how to challenge it, and where to start.

One thing for sure: much remains to be examined and improved in the often opaque world of R&D.

* Today Brad Garlinghouse is president & COO at Ripple Labs, original developer of the Ripple Protocol for secure funds transfer.