the thoughts and opinions of dan schultze

Correlation vs Causation

2011-09-23T14:56:00.001-07:00

Product Development Methodology

2011-09-20T13:47:00.001-07:00

Product Development Methodology

Plan

Purpose

Understandthe business objectives for the project and the constraints the team must workwithin.

· Identify business goals

· Define scope

· Identify critical datesand budget constraints

· Create timeline

· Define: roles,responsibilities and reporting relationships

· Determine projectframework

Deliverables

· Work Plan

Results

Achievableand measurable criteria for success are established and the project managementis defined.

Discover

Purpose

Understandthe brand, the business environment, user archetypes and their goals in orderto define the correct design requirements.

· Understand businessenvironment (internal & external)

· Understand brand andproduct(s)

· Research users /user-types - observations, , task analysis and usability testing

· Define personas (usermotivations)

· Identify user goals

· Evaluate the currentsolution

Deliverables

· Brand Assessment

· Use Goals

· Current SolutionEvaluation

Results

Theteam has the necessary information to imagine the right solution for thechallenge.

Concept

Purpose

Assimilate the findings from the Discovery Phaseand envision a solution that gracefully bridges business goals with User needs.

· Articulate preciseproblem to be solved

· Describe/Illustrate visionfor solution

· Create lists of designimperatives and new features

Deliverables

· Product Brief

Results

Thestrategy and vision are set to guide the subsequent design activities.

Architecture

Purpose

Identifyall of the necessary design components, define, their purpose, behavior andproperties, model their relationship to one another and arrange them into acohesive, logical solution.

· Create scenarios,conceptual models and process models

· Identify/describe elementsand logic

· Create data and object models

· Illustrate framework

Deliverables

· Scenarios

· Site map

· Object model &definition list

· Content taxonomy models

· Content matrices

· Wireframe diagrams

Results

The activities and deliverable in this phasedefine the structure, meaning and behavior.

Design

Purpose

Buildon the architectural foundation to develop and test a design that achieves thevision articulated in the Product Brief.

· Develop messages

· Develop media content

· Design prototypes

· Validate prototype withUsers

· Create detailed designspecification

Deliverables

· Conceptual Prototype

· Creative copy

· Design Spec

Results

Creativemessaging, mood, persona and style are applied to the architectural frameworkand the solution is validated with key stakeholders. Further, thespecifications to implement the designs are described in detail.

Build

Purpose

Thedesign specs guide the production of the final, working product. Material isproduced to help transition the organization into the new business capability.Rules and policies are documented to help understand and maintain the solution.

· Optimize assets

· Produce code

· Conduct performancetesting

· Prep final files

· Createdocumentation/policies

Deliverables

· Versioned Adjustments toDocumentation and Prototypes

· Performance Test Results

· User Manuals

· Product Documentation

Results

Atthe completion of this phase, the product is ready for launch and theorganization is prepared to function in the new business capability beingcreated by the solution.

Deploy

Purpose

Launchthe product and help the organization get the most out of the solution.

· Deploy product

· Conduct training

· Deploy businesspolicies/procedures

· Administer clientsatisfaction survey

· Gather user feedback

· Monitor and measureresults

· Create strategies forfuture enhancements

Deliverables

· Final Tested Product

· Copy of Final Documents

· Analytics

· Surveys

Results

The completion of this phase marks the point atwhich the organization is up and running with their new business capability.The solution is monitored and measured and refined.

Android Development Best Practices

2011-01-04T14:51:00.000-08:00

A very helpful presentation in developing your first Android Application. I also recommend installing the API Demos App from Google.

SCVNGR Game Dynamics Playdeck

2010-09-01T09:26:00.000-07:00

Guide To This Document: This list is a collection of game dynamics terms, game dynamics theories that are interesting, useful and potentially applicable to your work here at SCVNGR. Many of them have clear applications within the SCVNGR game layer (progression dynamic, actualization), many of them don’t… yet (status, virtual items). Many of them are just interesting for your general education on game dynamics theory (epic meaning, social fabric of games). Many of these game dynamics concepts are well known and are sourced from all over the internet and from researchers such as Jane McGonigal, Ian Bogost and Jess Schell and articles on gamasutra (which I highly recommend reading). Others are used exclusively internally here and won’t make any sense outside of HQ. Along with a link to this document, you will have received these dynamics in a set of flash cards. Please memorize those. If you’re on the engineering / game-design team you can access our internal game dynamics visualizer (with the most up to date dynamics) through your account. Download the SCVNGR app for iPhone& Android (if you haven’t already) and start playing. Find places where these game dynamics exist or places where you could implement them by building on the game layer using our tools, or others.

1. Achievement

Definition: A virtual or physical representation of having accomplished something. These are often viewed as rewards in and of themselves.

Example: a badge, a level, a reward, points, really anything defined as a reward can be a reward.

2. Appointment Dynamic

Definition: A dynamic in which to succeed, one must return at a predefined time to take some action. Appointment dynamics are often deeply related to interval based reward schedules or avoidance dyanmics.

Example: Cafe World and Farmville where if you return at a set time to do something you get something good, and if you don’t something bad happens.

3. Avoidance

Definition: The act of inducing player behavior not by giving a reward, but by not instituting a punishment. Produces consistent level of activity, timed around the schedule.

Example: Press a lever every 30 seconds to not get shocked.

4. Behavioral Contrast

Definition: The theory defining how behavior can shift greatly based on changed expectations.

Example: A monkey presses a lever and is given lettuce. The monkey is happy and continues to press the lever. Then it gets a grape one time. The monkey is delighted. The next time it presses the lever it gets lettuce again. Rather than being happy, as it was before, it goes ballistic throwing the lettuce at the experimenter. (In some experiments, a second monkey is placed in the cage, but tied to a rope so it can’t access the lettuce or lever. After the grape reward is removed, the first monkey beats up the second monkey even though it obviously had nothing to do with the removal. The anger is truly irrational.)

5. Behavioral Momentum

Definition: The tendency of players to keep doing what they have been doing.

Example: From Jesse Schell’s awesome Dice talk: “I have spent ten hours playing Farmville. I am a smart person and wouldn’t spend 10 hours on something unless it was useful. Therefore this must be useful, so I can keep doing it.”

6. Blissful Productivity

Definition: The idea that playing in a game makes you happier working hard, than you would be relaxing. Essentially, we’re optimized as human beings by working hard, and doing meaningful and rewarding work.

Example: From Jane McGonical’s Ted Talk wherein she discusses how World of Warcraft players play on average 22 hours / week (a part time job), often after a full days work. They’re willing to work hard, perhaps harder than in real life, because of their blissful productivity in the game world.

7. Cascading Information Theory

Definition: The theory that information should be released in the minimum possible snippets to gain the appropriate level of understanding at each point during a game narrative.

Example: showing basic actions first, unlocking more as you progress through levels. Making building on SCVNGR a simple but staged process to avoid information overload.

8. Chain Schedules

Definition: the practice of linking a reward to a series of contingencies. Players tend to treat these as simply the individual contingencies. Unlocking one step in the contingency is often viewed as an individual reward by the player.

Example: Kill 10 orcs to get into the dragons cave, every 30 minutes the dragon appears.

9. Communal Discovery

Definition: The game dynamic wherein an entire community is rallied to work together to solve a riddle, a problem or a challenge. Immensely viral and very fun.

Example: DARPA balloon challenge, the cottage industries that appear around McDonalds monopoly to find “Boardwalk”

10. Companion Gaming

Definition: Games that can be played across multiple platforms

Example: Games that be played on iphone, facebook, xbox with completely seamless cross platform gameplay.

11. Contingency

Definition: The problem that the player must overcome in the three part paradigm of reward schedules.

Example: 10 orcs block your path

12. Countdown

Definition: The dynamic in which players are only given a certain amount of time to do something. This will create an activity graph that causes increased initial activity increasing frenetically until time runs out, which is a forced extinction.

Example: Bejeweled Blitz with 30 seconds to get as many points as you can. Bonus rounds. Timed levels

13. Cross Situational Leader-boards

Definition: This occurs when one ranking mechanism is applied across multiple (unequal and isolated) gaming scenarios. Players often perceive that these ranking scenarios are unfair as not all players were presented with an “equal” opportunity to win.

Example: Players are arbitrarily sent into one of three paths. The winner is determined by the top scorer overall (i.e. across the paths). Since the players can only do one path (and can’t pick), they will perceive inequity in the game scenario and get upset.

14. Disincentives

Definition: a game element that uses a penalty (or altered situation) to induce behavioral shift

Example: losing health points, amazon’s checkout line removing all links to tunnel the buyer to purchase, speeding traps

15. Endless Games

Definition: Games that do not have an explicit end. Most applicable to casual games that can refresh their content or games where a static (but positive) state is a reward of its own.

Example: Farmville (static state is its own victory), SCVNGR (challenges constantly are being built by the community to refresh content)

16. Envy

Definition: The desire to have what others have. In order for this to be effective seeing what other people have (voyeurism) must be employed.

Example: my friend has this item and I want it!

17. Epic Meaning

Definition: players will be highly motivated if they believe they are working to achieve something great, something awe-inspiring, something bigger than themselves.

Example: From Jane McGonical’s Ted Talk where she discusses Warcraft’s ongoing story line and “epic meaning” that involves each individual has motivated players to participate outside the game and create the second largest wiki in the world to help them achieve their individual quests and collectively their epic meanings.

18. Extinction

Definition: Extinction is the term used to refer to the action of stopping providing a reward. This tends to create anger in players as they feel betrayed by no longer receiving the reward they have come to expect. It generally induces negative behavioral momentum.

Example: killing 10 orcs no longer gets you a level up

19. Fixed Interval Reward Schedules

Definition: Fixed interval schedules provide a reward after a fixed amount of time, say 30 minutes. This tends to create a low engagement after a reward, and then gradually increasing activity until a reward is given, followed by another lull in engagement.

Example: Farmville, wait 30 minutes, crops have appeared

20. Fixed Ratio Reward Schedule

Definition: A fixed ratio schedule provides rewards after a fixed number of actions. This creates cyclical nadirs of engagement (because the first action will not create any reward so incentive is low) and then bursts of activity as the reward gets closer and closer.

Example: kill 20 ships, get a level up, visit five locations, get a badge

21. Free Lunch

Definition: A dynamic in which a player feels that they are getting something for free due to someone else having done work. It’s critical that work is perceived to have been done (just not by the player in question) to avoid breaching trust in the scenario. The player must feel that they’ve “lucked” into something.

Example: Groupon. By virtue of 100 other people having bought the deal, you get it for cheap. There is no sketchiness b/c you recognize work has been done (100 people are spending money) but you yourself didn’t have to do it.

22. Fun Once, Fun Always

Definition: The concept that an action in enjoyable to repeat all the time. Generally this has to do with simple actions. There is often also a limitation to the total level of enjoyment of the action.

Example: the theory behind the check-in everywhere and the check-in and the default challenges on SCVNGR.

23. Interval Reward Schedules

Definition: Interval based reward schedules provide a reward after a certain amount of time. There are two flavors: variable and fixed.

Example: wait N minutes, collect rent

24. Lottery

Definition: A game dynamic in which the winner is determined solely by chance. This creates a high level of anticipation. The fairness is often suspect, however winners will generally continue to play indefinitely while losers will quickly abandon the game, despite the random nature of the distinction between the two.

Example: many forms of gambling, scratch tickets.

25. Loyalty

Definition: The concept of feeling a positive sustained connection to an entity leading to a feeling of partial ownership. Often reinforced with a visual representation.

Example: fealty in WOW, achieving status at physical places (mayorship, being on the wall of favorite customers)

26. Meta Game

Definition: a game which exists layered within another game. These generally are discovered rather than explained (lest they cause confusion) and tend to appeal to ~2% of the total gameplaying audience. They are dangerous as they can induce confusion (if made too overt) but are powerful as they’re greatly satisfying to those who find them.

Example: hidden questions / achievements within world of warcraft that require you to do special (and hard to discover) activities as you go through other quests

27. Micro Leader-boards

Definition: The rankings of all individuals in a micro-set. Often great for distributed game dynamics where you want many micro-competitions or desire to induce loyalty.

Example: Be the top scorers at Joe’s bar this week and get a free appetizer

28. Modifiers

Definition: An item that when used affects other actions. Generally modifiers are earned after having completed a series of challenges or core functions.

Example: A X2 modifier that doubles the points on the next action you take.

29. Moral Hazard of Game Play

Definition: The risk that by rewarding people manipulatively in a game you remove the actual moral value of the action and replace it with an ersatz game-based reward. The risk that by providing too many incentives to take an action, the incentive of actually enjoying the action taken is lost. The corollary to this is that if the points or rewards are taken away, then the person loses all motivation to take the (initially fun on its own) action.

Example: Paraphrased from Jesse Schell “If I give you points every time you brush your teeth, you’ll stop brushing your teeth b/c it’s good for you and then only do it for the points. If the points stop flowing, your teeth will decay.”

30. Ownership

Definition: The act of controlling something, having it be *your* property.

Example: Ownership is interesting on a number of levels, from taking over places, to controlling a slot, to simply owning popularity by having a digital representation of many friends.

31. Pride

Definition: the feeling of ownership and joy at an accomplishment

Example: I have ten badges. I own them. They are mine. There are many like them, but these are mine. Hooray.

32. Privacy

Definition: The concept that certain information is private, not for public distribution. This can be a demotivator (I won’t take an action because I don’t want to share this) or a motivator (by sharing this I reinforce my own actions).

Example: Scales the publish your daily weight onto Twitter (these are real and are proven positive motivator for staying on your diet). Or having your location publicly broadcast anytime you do anything (which is invasive and can should be avoided).

33. Progression Dynamic

Definition: a dynamic in which success is granularly displayed and measured through the process of completing itemized tasks.

Example: a progress bar, leveling up from paladin level 1 to paladin level 60

34. Ratio Reward Schedules

Definition: Ratio schedules provide a reward after a number of actions. There are two flavors: variable and fixed.

Example: kill 10 orcs, get a power up.

35. Real-time v. Delayed Mechanics

Definition: Realtime information flow is uninhibited by delay. Delayed information is only released after a certain interval.

Example: Realtime scores cause instant reaction (gratification or demotivation). Delayed causes ambiguity which can incent more action due to the lack of certainty of ranking.

36. Reinforcer

Definition: The reward given if the expected action is carried out in the three part paradigm of reward schedules.

Example: receiving a level up after killing 10 orcs.

37. Response

Definition: The expected action from the player in the three part paradigm of reward schedules.

Example: the player takes the action to kill 10 orcs

38. Reward Schedules

Definition: the timeframe and delivery mechanisms through which rewards (points, prizes, level ups) are delivered. Three main parts exist in a reward schedule; contingency, response and reinforcer.

Example: getting a level up for killing 10 orcs, clearing a row in Tetris, getting fresh crops in Farmville

39. Rolling Physical Goods

Definition: A physical good (one with real value) that can be won by anyone on an ongoing basis as long as they meet some characteristic. However, that characteristic rolls from player to player.

Example: top scorer deals, mayor deals

40. Shell Game

Definition: a game in which the player is presented with the illusion of choice but is actually in a situation that guides them to the desired outcome of the operator.

Example: 3 Card Monty, lotteries, gambling

41. Social Fabric of Games

Definition: the idea that people like one another better after they’ve played games with them, have a higher level of trust and a great willingness to work together.

Example: From Jane McGonicgal’s TED talk where she suggests that it takes a lot of trust to play a game with someone because you need them to spend their time with you, play by the same rules, shoot for the same goals.

42. Status

Definition: The rank or level of a player. Players are often motivated by trying to reach a higher level or status.

Example: white paladin level 20 in WOW.

43. Urgent Optimism

Definition: Extreme self motivation. The desire to act immediately to tackle an obstacle combined with the belief that we have a reasonable hope of success.

Example: From Jane McGonical’s TED talk. The idea that in proper games an “epic win” or just “win” is possible and therefore always worth acting for.

44. Variable Interval Reward Schedules

Definition: Variable interval reward schedules provide a reward after a roughly consistent amount of time. This tends to create a reasonably high level of activity over time, as the player could receive a reward at any time but never the burst as created under a fixed schedule. This system is also more immune to the nadir right after the receiving of a reward, but also lacks the zenith of activity before a reward in unlocked due to high levels of ambiguity.

Example: Wait roughly 30 minutes, a new weapon appears. Check back as often as you want but that won’t speed it up. Generally players are bad at realizing that.

45. Variable Ratio Reward Schedule

Definition: A variable ratio reward schedule provides rewards after a roughly consistent but unknown amount of actions. This creates a relatively high consistent rate of activity (as there could always be a reward after the next action) with a slight increase as the expected reward threshold is reached, but never the huge burst of a fixed ratio schedule. It’s also more immune to nadirs in engagement after a reward is acheived.

Example: kill something like 20 ships, get a level up. Visit a couple locations (roughly five) get a badge

46. Viral Game Mechanics

Definition: A game element that requires multiple people to play (or that can be played better with multiple people)

Example: Farmville making you more successful in the game if you invite your friends, the social check-in

47. Virtual Items

Definition: Digital prizes, rewards, objects found or taken within the course of a game. Often these can be traded or given away.

Example: Gowalla’s items, Facebook gifts, badges

Credit:Erick Schonfeld

FEED Presentation

2009-11-19T14:58:00.001-08:00

I think this is one of the best presentations I've seen this year.

FEED is Razorfish’s annual study charting how technology is changing the way consumers engage with brands. The report, and the blog, are written by Garrick Schmitt, Group Vice President, Experience Planning.

FEED: The Razorfish Digital Brand Experience Report 2009 Key Findings

View more presentations from Razorfish Marketing.

Google interview questions

2009-11-19T14:54:00.000-08:00

Knowing what I know about product management, I found the product manager questions in this article to be totally on point when it comes to problem solving and thining outside the box. I think I could hack it no prob.

Product Development a la Google

2009-10-07T10:54:00.000-07:00

Taken from here (thanks Google!)

1. Focus on the user and all else will follow.

2. It's best to do one thing really, really well.

3. Fast is better than slow.

4. Democracy on the web works.

5. You don't need to be at your desk to need an answer.

6. You can make money without doing evil.

7. There's always more information out there.

8. The need for information crosses all borders.

9. You can be serious without a suit.

10. Great just isn't good enough.

LowFi Prototyping

2009-10-05T14:17:00.000-07:00

Low-fidelity prototyping versus high-fidelity prototyping

In my discussions with designers, one of the interesting recurring conversations is the tools and process they use to prototype and mock up experiences. In particular, there’s a lot of divergence on how high or low-fidelity to go with a prototype.

For designers that primarily come from agency backgrounds, I’ve found that there’s an emphasis on quickly getting to a near pixel-perfect mockup, and the variations are minor in detail. In that worldview, the ideal deliverable is a single version of something that feels high-quality and gets minor feedback from clients. In the client-agency model, if you give your clients a bunch of rough mockups that seem low-fidelity, then you risk looking unprofessional. Or worse yet, you might get a ton of diverging comments that you then have to work out and iterate – in some cases that’s the last thing you want to do.

This is especially not a good situation for companies that focus on products delivered to customers – in that case, you mostly want your product to be the right one, no matter how many iterations it takes. As a result, low-fidelity prototyping can be really useful because it aids an iterative, customer-focused approach rather than one where the Great Designer comes up with something directly from his brain.

Here’s a couple of the main advantages:

Get better and more honest feedback
It’s great for A/B testing
Make the cost of mistakes cheap, not expensive
Refine the page flow, not the pages
Figure out the interaction design rather than the visual design

In addition, after I’m done arguing my point, I’ll recommend some of the tools that have been useful for me in doing low-fidelity prototyping.

Get better and more honest feedback

The first time I really undestood the power of low-fidelity prototyping was when I started doing usability tests on consumer products I had built (This was years ago). Initially, I wondered why anyone did paper-prototyping? I immediately concluded that it was due to a deficiency of many designers that they couldn’t write code, and thus couldn’t do HTML mockups of the products they wanted to build.

But once I started getting people to view and interact with my prototypes, I realized that one of the big problems was that people didn’t give good feedback when the prototype you present to them is too perfect. Rather than telling me about the really high-level things, like “does the value proposition make sense?” instead they would focus on colors, fonts, the layout of the page, etc. And furthermore, they didn’t feel that they could really jump in and build on top of the ideas you showed them, because it was far beyond their capability to duplicate.

Compare this to a simple exercise where you are using hand-drawn cards or drawing paper and are literally sketching stuff out during a customer interview – you’re much more likely to try something out, and have the person you’re interviewing grab your pencil and say “no, more like this!” And that’s exactly the kind of interaction you want.

It’s great for A/B testing

As for as a metrics-driven approach goes, you have to remember that techniques like A/B testing fundamentally thrive off of variety. In particular, it thrives off of variety at the UI layer, where many small UI changes that cost very little technically can be tried out and optimized. As a result, you don’t want 2-3 pixel-perfect mockups, you actually want 10 or 20 rough mockups where you can select only the most high-variance candidates.

Some of the highest variance stuff has to do with changing the order in which you do things, or opt-in versus opt-out, or richer AJAXy interactions. These are all things where it’s easier to generate many candidates through low-fidelity prototypes since you’re often looking at things form a systems level.

Make the cost of mistakes cheap, not expensive

One of the hidden benefits of having a low-fidelity prtotyping process is that it makes changing directions much easier, which naturally facilitates a collaborative design discussion. When you’re using a customer-driven product philosophy that incorporates a lot of outside metrics and qualitative feedback, you’ll probably get multiple people involved in the design process. If it’s done by one person or a small group, and is polished significantly before it reaches the greater group, one of the problems is that it discourages collaboration. It’s very hard for people not to get defensive when they’ve spent a lot of time polishing something only for it to get changed significantly. Using a low-cost process makes it so that you can try a lot of variations cheaply, without any of the emotions involved.

Refine the page flow, not the pages

One of the highest leverage design decisions you can make is not about the look of an individual page, but what happens before and after it. For example, you can take a multi-step process and condense it onto one page, or change the ordering of something so that you do something and then register, rather than the other way around. These kinds of design decisions ultimately focus on the order and flow of the user, rather than the look or interactions of any specific page. If you go with a low-fidelity, then it’s easy to draw lots of small pages and link them up in a flow, and do things like cross pages off, change the ordering of a funnel, and lots of things that feel natural when the prototype is very rough. Otherwise, it’s too easy to get caught in the details on the “right” way something works without exploring the options.

Work your way up from low-fidelity to high-fidelity

Of course, you want to make sure that you use the right process for the right job. So there’s nothing wrong with high-fidelity prototyping, especially when you are in the later stages and thinking about issues like branding, look and feel, and all those other details. One way to keep this process going is to have multiple rough prototyping checkpoints so that design decisions are constantly getting refined – maybe the first step is a sketch on a paper, the next is a rough mockup on the computer, then a detailed mockup, then a rough built-out version, and then iterate to the final product. These steps make it so that all the design decisions are well understood, refined, and debated all the way through.

Tools I recommend

Finally, a couple recommendations on tools for paper prototyping:

Number 2 Pencil
Giant art pad for drawings – you can get these at an art shop or office store
Balsamiq (check out the video on the linked page)
Macromedia Fireworks

In general, nothing beats pencil and paper, but that’s just me. I’ve been told that for people who aren’t comfortable drawing, using tools like Balsamiq helps quite a bit.

Credit Andrew Chen

Designing A Viral Loop

2009-10-05T12:20:00.000-07:00

Designing a viral loop has multiple stages
Viral loops have been featured in mainstream media and there’s even a book coming out on it – but the step-by-step design of creating a new loop remains obscure, and for good reason. I’ve come to believe that creating viral loops is akin to building a software project – at best, it still comes down to a great team, a strong understanding of the tools available, and relentless iteration. There’s no recipe at the heart of it which guarantees a viral process every time, the same way that you can’t guarantee that any software project will result in market success.

There are no silver bullets in viral marketing
In fact, the core of virality ensures that there will never be a dominant “recipe.” If everyone knows how to build a viral loop around social network invites, then everyone will do it, resulting in consumers will become desensitized, which finally leads to lower response rates. Thus this causes the viral loop to unwind, which leads to long-term disaster.

The only way to combat this is to build a viral loop around the core of your product – something that no one will seek to duplicate, unless they are a direct competitor. These viral loops are incredibly effective because they are lasting and sustainable.

I wanted to jot down a couple thoughts on the different stages that viral loop design go through, so that the entreprenurs reading through this can imagine deeply ingrained, user-aligned ways for their products to gain distribution.

Strategize: Stage 1
The first stage of a viral loop is developing the core strategy around the loop. This requires the viral loop designer to think through, step-by-step, how a user will come to find their product and how they will ultimately pass it along to their friends. If you’re lazy, there are lots of recipes to follow from the Facebook ecosystem like quizzes, “find your friends,” and gifts. As discussed above, these opportunities are already becoming less effective every day.

Even if you decide to use an existing recipe, here are some higher-level strategy questions that should be answered before proceeding:

How does this viral loop fit into your core product?
What is the fundamental value proposition you are presenting to your users?
If your loop is successful, will users transition to your core product or will they bounce when reaching the switchover point?

As you might imagine, most of the discussion here is qualitative and there’s very little A/B testing involved.

Implement: Stage 2
The next stage is the rapid development of the core viral loop. This part should hopefully take days or weeks, not months. It will also certainly be wrong. The best advice I can give here is to follow agile development models and to build the smallest number of features and pages to create the initial flow of pages.

As mentioned before, the best implementations are strongly tied to the core product – as a result, if you’re a video site, it’s best if you can somehow involve videos. If you’re a dating site, you probably want to involve dating.

The other implementation advice I’ll give is to treat the viral loop code as an iterative, protoyping process. So copy and paste all you need, keep it in a separate codebase, and make it easy to refactor. You’ll need to do a lot of messy stuff like changing the order of pages or page elements later, and once you develop your own recipe, it’s easy to rewrite it in the “right way.”

Launch: Stage 3
The next step is to beg, borrow, or steal traffic. The easiest way is often to pay for it, $50/day or so, just so you have a trickle of traffic coming in.

Optimize: Stage 4
As you get a flow of incoming traffic, this allows you to deeply optimize the experience. This will involve building out some basic infrastructure to do A/B testing, or using Google Web Optimizer, and otherwise. The key thing here, of course, is to measure whether or not the $50/day you’re spending results in traffic above and beyond what you’re paying for – the more the better, and eventually you’ll cross the threshold where traffic scales infinitely.

In this stage, there are a lot of common fixes that you’ll want to consider:

Shortening the flow of pages (can you shrink a 5 page funnel down to 2?)
Rearranging UI elements to emphasize next steps
Testing different value propositions for going through the flow
Increasing the # of people invited

This optimization stage creates great conflict for product and customer-oriented people. Oftentimes, to get a number to move from 10% to 30%, there’s temptation to do things that users may not be happy with. This might include things like asking for invites multiple times throughout the initial session, presenting an opt-out process for selecting friends, etc. These are all bad and need to be fixed in order to create a long-term sustainable viral loop.

This optimization step can take a very long time (months is not uncommon) as you zero in on the dozens of small and large changes needed to create a viral loop.

After months of work, two outcomes can result:

You don’t reach your goal, and you’re stuck on traffic
You reach your goal, and your traffic is going bananas!

If you don’t reach your goal, then it’s time to stop your optimization process. Often the changes that result are just too small to drive substantial increases in metrics. Instead, you’ll have to rework your entire value proposition, which means to either go back to Stage 2 or possibly Stage 1. This means you’ll want to stop A/B testing and start building out a deeper featureset.

Refine: Stage 5
If your optimization step was successful, your work is probably not done. The final step is polishing your viral loop.

This includes figuring out issues like:

Making your loop as user-aligned as possible
Building a pleasant user experience and removing unnecessary flows or page elements
Refactoring the code to move it from prototype to production
Integrating it into your core product in a way that makes sense

A lot of people are tempted to skip this polish step, but don’t do it! Skipping this step means that your initial product experience will suck, or be offensive.

In fact, when there’s “excess” virality, that’s a great opportunity to make changes to the viral loop that make it nicer or friendlier. In general, if you are getting exponential growth, it’ll be great even if it’s a slower exponential. What’s more important at that point is spendfing your extra growth towards changes that positively impact long-term retention.

On the other hand, if your product is just meant to be short-term mad money, then by all means skip this step

Credit Andrew Chen

Persistence Mathematically Works

2009-09-08T16:28:00.000-07:00

Peep this:

Google, P&G, 3M, and every other ridiculous innovator knows that every product they release won't be profitable.
They know that every product comes with a high probability of failure.

So assume this:

Your first try has a 10% chance of succeeding.
Two tries? 31.6%
Three tries? 46.4%
10 tries? 79.4%

(The equation: Your chance of success = 0.10^(1/X) ------ where X is your # of tries, based on the assumption that there's a 10% chance of succeeding)

Social design explosions

2009-08-26T12:31:00.000-07:00

Why do social products tend towards clutter?
One of the toughest design problems for people working on social products is the inevitable path towards cluttered interfaces and diluted brands, as you try to build more social activities and richness within your product. As a result, these products tend to drift towards “portals,” “hubs,” or “platforms,” rather than clean, single-purpose destinations. This might be good or bad, depending on your viewpoint, but it certainly introduces a number of design challenges when every central “entity” on your site (be it a video page, or profile) has dozens of jumping off points for more complex interactions. Or when you have a “tab explosion” as you bolt on common social application paradigms like blogs, chat, or whatever.

For MySpace, this manifested itself as a massive top menu detailing all the different ways to interact with the site, including Classifieds, Music, Games, Video, Forums, and others. For Facebook, they had to build the Windows-like application bar that shows up on every page and allows access to chat and commonly used applications. It seems as though this clutter is almost inevitable as you try to centralize a wide variety of social activities.

Users push you towards more social activities, not less
The central driver, I believe, for this social activity explosion is that people want to have LOTs of different ways to interact with their friends. These different activities let them have very nuanced interactions that have deep and meaningful social signaling.

Let’s take an offline activity, for example, an invitation to a date – there are lots of nuances that can be read into asking someone to:

have a quick mid-day coffee
come over and have a nice dinner
go to a movie
have a drink mid-week versus Friday/Saturday
go out with a group of friends to a music show
having brunch with your parents
etc.

All of the above activities provide different social signals based on how big of a time commitment it is, who’s involved, what time of week it happens, how expensive it is, etc. And if you were to build online equivalents of these types of activities, it would be better at each step, since it allows for richer interactions.

As a result of this, your users will always like any new social interactions you push out, and will often suggest/demand new activities.

The social web laundry list
As a result of the demand for new social activities, you inevitably get a series of bolt-on design patterns that recur across many different social products. An incomplete list might include:

polls
quizzes
reviews
comments
forums
chat
blogs
videos, photos, and other multimedia
avatars
leaderboards
private/public messaging
status messages
etc.

What else am I missing? Suggest some other ones in comments, and I’ll be happy to continue extending this list

Either way, there’s probably some rule that if a new social product these days decides that, “hey! What our product needs is polls!” then the design philosophy of the product probably should be reevaluated. It’s a powerful indicator that the product roadmap is overly focused on short-term user engagement versus a long-term market position.

Drawing the line between “core” versus other
These mechanics are so easily bolt-on-able that it destroys the differentiated value of a product – this happens through clutter, confusion, and overduplication of features relative to other sites. It becomes a trap that weakens the brand long-term, while producing higher engagement in the short-term – quite the devilish dilemma.

Ultimately, to avoid this fate, every product needs to draw a line in the sand on what is core, and what are extraneous social activities that should happen off the site. Or, if not off the site, in a carefully cordoned-off area. Either way, these choices need to get made, otherwise clutter ensues.

Potential solutions
Several companies have dealt with this design problems in different ways – let’s go through all of them:

Solution 1: Build everything
In the MySpace example, the site ultimately decided to incorporate a very large chunk of all the functionality they could think of. Just explore the top menu bar, and I think you’d be surprised by how much product is sitting inside of there.

Solution 2: Open up the CSS/HTML layer
Interestingly enough, MySpace also used another method of allowing users to extend their profiles by allowing people to just copy and paste arbitrary CSS/HTML. Another company that did this is eBay, as well as many blogging sites. Outside of the obvious security issues, the nice part about this is that this is a really simple integration that works with many different kinds of tools and widgets.

Solution 3: Provide a rich onsite platform
This is the Facebook/OpenSocial approach, where applications exist on a site rather than off of it

Solution 4: Create off-site APIs and activities
To some extent, this can happen by itself with an API or not, as passionate users will create forums, mailing lists, blogs, and other social structures about your product. But as Twitter and blogs show, you can build an API which allows off-site applications and websites to build richer functionality. It will be interesting to see if Twitter eventually creates an on-site API a la Facebook, or if they will always make their onsite experience very simple and clean.

Solution 5: just focus on one thing
And the final solution is just to ignore your users, and focus on the main value that your product provides. This certainly has a nice charm to it, but obviously few companies follow this – more ambition leads to more features, typically, even though the user experience might suck as a result.

Credit Andrew Chen

Benefit-Driven Metrics

2009-06-11T11:28:00.001-07:00

Measuring value created rather than optimizing for yourself
In my last blog post, I talked about the idea that value creation generates revenue, traffic, and other metrics, not the other way around. This is a particularly interesting idea to implement because it goes against much of the standard analytics reports that are out there.

The reason is that ultimately, most metrics tend to focus inwards, on self-interested gain, rather than outwards on the value you’re creating for your customers. Let’s take a couple examples of inward-focused metrics that people often cite:

account registrations
pageviews
unique visitors per month
revenues

I’m sure you measure many of the above, as I do as well. It’s OK to measure this stuff, but if you start to optimize for it, you are starting to focus on the business of value extraction, not value creation.

Measuring # of life preservers sold versus the # of lives saved
Thus we come back to the title of the post. Most people are using standard analytics packages that are commoditized to focus inwards on metrics like pageviews and revenue. To use an analogy, that’s akin to the idea of measuring the # of life preservers that you sell, and trying to optimize for that, rather than optimizing for the benefit, which is the # of lives saved.

If you focus primarily on selling life preservers, then you’ll tend to do all sorts of stuff like:

making them cheaper to build
adding doo-dads to them that are flashy to customers
using aggressive sales tactics
etc.

These things might generate revenue in the short to medium run, but if you prioritize this at the expense of actually delivering on the product benefit, then that’s a bad optimization in the long term.

Now contrast that to the idea of trying to save as many lives as possible. You might still want to make them as cheap as possible, so that every ship in the world can have as many of them as needed. You might still want to add upgrades to them, but only if they help save lives. etc. While these changes may be similar in execution, they are different in spirit than the changes you’d make when optimizing for sales.

Introducing “Benefit-Driven Metrics”
So ultimately to start this exercise, you should throw out all the standard metrics (conversion rates, pageviews, etc.) and just focus on one thing:

What are your customers measuring?

By looking at how they define value, then you get yourself aligned to them as closely as possible. Answering this question sets your company up for value creation, which then unlocks the ability to gain something from that value, then you have to start here.

I’ll deem these quantitative measurements as “Benefit-driven metrics.”

How do you measure it?
Here’s the interesting part - everyone’s benefit-driven metrics will be completely different, because most people’s customers and value proposition and product are ultimately very different. Unfortunately, you don’t have the crutch of standardized numbers like pageviews or uniques to lean on.

But let me give you some examples for reference:

For dating sites:
Why do customers join dating sites? To find their soulmates. Thus, measure the quantity of successful matches you make, not the lifetime value of the customer. Focusing on LTV can easily lead you to do things like creating fake accounts to make people come back, or optimizing it so that they find their best matches several months down the line, or trying to get everyone to pre-pay for the service rather than making the product experience awesome.

For marketplaces:
Why do customers sell on a marketplace? To make money and get rid of their stuff. Why do customers buy on a marketplace? So that they can get things cheaply and quickly, and are happy with their purchase. Thus, measure the quantity of how much your sellers take home, and how many buyers are happy with their experiences. Contrast this with overfocusing on listing fee revenues, which might get you into a spiral of raising prices rather than creating the best commerce experience.

For social networks:
Why do customers use social networks? To “connect” with their friends - let’s boil that down to communicating (though it’s obviously much richer than that). Then ideally, you might want to focus on the number of messages/comments/posts that end up getting replies from their friends. If you overfocus on something like user registrations, then you might get a ton of users, but maybe they won’t be getting to experience the benefits of the product.

For online publishers who sell to advertisers:
Why do advertisers buy ads on websites? To generate traffic to their own sites, which in turn leads to revenue. In this case, you should quantify the amount of revenue you generate for your advertiser customers, or at least the number of conversions they receive. Contrast this with the approach of measuring and optimizing your own CPMs as a publisher, which results in potentially delivering a lot of crappy traffic to advertisers who will drop their payments in the long term.

Special note for ad-driven startups
Now ad-supported startups have a particularly interesting issue in this, because I keep using the words “benefits” and “customers” and perhaps it’d be easy to think this refers to the users of the product. But maybe not, as I’ve outlined before in Your ad-supported Web 2.0 site is actually a B2B enterprise in disguise. The reason is that your customer may actually be the advertiser on the site, not your user!

And in fact, if you overfocus on pleasing your users to the detriment of your advertiser customers, which is very easy - then that leads to very bad things.

Start this benefits-driven approach now, not later, so you can learn the right things
Finally, I want to emphasize that I believe it’s important to start thinking about these benefit-driven metrics from the beginning of your business, not later. The reason is that every learning that a startup makes is often hugely applicable to a specific context, but not at all applicable to other variations.

If you’re going to start a website that churns users like crazy but hits massive user goals, you will build an entire organization to optimize for those metrics. And once you’ve gone far on this, it’s not clear that you’ll have the DNA, the technology, the ideas, or the willpower to execute in a different direction.

Credit Andrew Chen

Product & Marketing Metrics Optimization

2009-04-10T10:58:00.000-07:00

How to be a Web 2.0 Metrics Jedi (Web 2.0 Expo, April 2009)

View more presentations from Dan Olsen.

Vertical Markets are DIFFERENT

2009-04-08T13:48:00.000-07:00

Steve Blank: “What I began to realize is that we teach entrepreneurship like every vertical market and industry has the same set of rules. So the first heuristic I want to offer is that — even in this class — there really is no common ‘these rules work’ for all vertical markets and industries.”

Listen to this excerpt from the second class of Steve’s customer development course for the rest of this wonderful lesson.

Audio: Vertical markets (mp3)

Slides: Vertical markets (pdf)

Here’s a transcript of the lesson.

We teach entrepreneurship like every vertical market has the same set of rules

Steve Blank: How many are in Web 2.0? Like the social something web. How many are in enterprise software? Anybody in semiconductors? EDA? OK. Is there a biotech guy still here? Oh!

One of the interesting things about when I put up the fact that there are different industries or markets, everybody goes, “Well, yeah. Of course.” I’m going to tell you a very funny story.

When I started teaching in the engineering school of the school not-to-be-named down south, but it starts with an “S,” I formed teams just like you guys are going to do for projects. And I’d always say, “Listen, anybody can start a company. All you need is a half a million bucks.” OK, yes, sir, a half a million bucks. Write that down.

Next week there’s always a group that looks like these three people, that says, “Startup, half a million dollars.” It’s a divide by zero problems here, because in our business they’d come back and say, “Hey, Professor Blank, in our business the common wisdom is $100 million.”

And then I’d go, “Oh, well, yes of course, you’re in the life sciences, that’s completely different.” The next week I’d say, “Except for these guys who need $100 million, you ought to get out and start selling your product on day one, because you don’t need to worry about any IP at all. Web 2.0. Just go out and get out there.”

The next week someone raises their hand and says, “Professor Blank, in our industry there’s a ton of patents and stuff and people tell us we shouldn’t be out there unless we start patent protecting all our stuff.”

And I went, “Oh, oh, oh, you’re in a different vertical market. In that vertical market you’re absolutely right. But OK, let’s keep going on with the class. The rest of you guys can keep going out because you don’t need to worry about anything, about government regulation or anything. It’s a startup. Just go out there. There’s no regulation to worry about.”

The next week it’s a group that comes up and says, “Professor Blank, we’re doing a medical device and there’s something called the 510K, and that’s a two-year process.” And I go, “Oh, oh, for those of you in medical devices…”

And what I began to realize is that we teach entrepreneurship like every vertical market and industry has the same set of rules. So the first heuristic I want to offer is that even in this class, there really is no common “these rules work” for all vertical markets and industries.

Vertical markets

Steve Blank: And the first heuristic I want you to think about is, when you hear common advice from friends or other people who’ve done startups, always ask what industry were they in, and was that particular advice relevant for me or not.

So for example, here’s a checklist of — I just randomly picked these. Web 2.0, enterprise software, enterprise software, communications software, communications software, consumer electronics, games software, semiconductors, EDA, clean tech, medical devices, life sciences, and personalized medicine.

I think, I’ve probably screwed up a startup in almost every one of these. That was a joke.

Did I miss anybody’s vertical market? Anybody here who I didn’t kind of get? All right. So this is not meant as a comprehensive list, but usually it takes about 95% of those students in the room.

Technical risk vs. market risk

Now what’s interesting is if I ask you, Eric, in your biotech startup, what’s your greatest risk? What is it?

Eric: Our team is working on an asthma inhaler.

Steve: Right. So what’s the biggest risk?

Eric: The efficacy.

Steve: Right. The efficacy of what?

Eric: The efficacy of the drug and its impacts. That’s a big risk.

Steve: So whether the product, as envisioned, works at all.

Eric: Right. And even if it works, are the adverse effects….

Steve: Does it kill you?

Eric: Yeah. Not to put too fine a point on it.

Steve: It’s a very nice clinician’s way of saying did it kill him or did he grow a third arm. How about you guys, do you have a particular drug or product in mind?

Student: The technology similar to some medical devices, the bio-monitoring… interactions.

Steve: So whether biomarkers are predicted, for a predictor from an assay you’re thinking about making.

Student: Yes.

Steve: So it’s a technical risk, right?

Student: Yes.

Steve: How many of you are thinking about a Web 2.0 startup? Great. What’s the risk, Josh? What’s the product?

Josh: It’s a digital media company.

Steve: Perfect. What’s the technical risk?

Josh: Finding the engineers.

Steve: Right. Is that a risk in Silicon Valley?

Josh: There’s not a lot of risk.

Steve: What’s the risk?

Josh: Getting people to use it.

Steve: Interesting. If our drug works for asthma, or your friends drug, does he have a customer problem?

Josh: No.

Steve: Why?

Josh: Because there are a lot of people that need….

Steve: If you’re running out of air, you’re going to probably want your drug, right? But you have a different problem. You could almost say, unless we really are stupid, we’re not going to screw up the technology. Wouldn’t you love it, if you were these guys, to be able to say that?

Josh: Yeah.

Steve: Big idea here. It’s a big idea, one that I’ve never heard articulated before at all with startups, yet world-class VCs know this on day one. There are some industries where the risk is purely customer in market.

And by purely I just mean, in Silicon Valley we take for granted digital media and web, with all due respect, for the hard work your software engineers are going to do getting it up, it’s not invention.

It’s, gee, did, they do it efficiently or did the Oracle salesman convince them to buy half a million bucks of software they didn’t need. But it’s not invention.

There’s a whole other set of industries where it truly is invention. Where it truly is, we should be so lucky to get this product working. Because if we get an asthma drug or an oncology for cancer curing drug, our only problem is how big is the licensing deal going to be? And not whether customers are going to want this.

Is this distinction clear? When you start a company, question one to self. Memo to self. Am I in a market risk company? Or am I in an invention risk company?

Hybrid risks

And by the way, I said this in the first class, I’ll remind you again, though. I’m happy to have every one of you in the class, but if you are in an invention risk company, now I’ll talk about hybrid companies in a second. Invention risk company, this class can offer you nothing.

To the extent that, what customer development is about, is how to dramatically reduce market risk. It is not how to reduce invention risk. So if I lose three of you next week. But you’re more than welcome, I just want to understand…

Student: I’m a hybrid.

Steve: I’m sorry?

Student: I’m a hybrid.

Steve: You’re a hybrid. And I’ll talk about hybrids in a second. Is that clear so far? Memo to self, duh! Can we assume the technology works and our problem is whether the product and market fit is correct? Or is in fact, the product itself the risk factor. But I have something says both.

Give me an example of a vertical market or an industry that has both risks.

Student: Semiconductor?

Steve: Perfect. Why?

Student: Because the technology is fairly advanced. There’s a lot of new insight there. But then the customers are very finicky, maybe, in the type of devices…

Steve: Let’s get specific. What kind of semiconductors do you have in mind? That has technology risk.

Student: Say, consumer electronics or…

Steve: Give me more specific than that. Anyone else in the semiconductor business? You raised your hands in the beginning, now you’re hiding. Yes?

Student: Yes, the industry I work in, we target what the customers want.

Steve: Right, so give me a specific case of technology risk in semiconductors.

Student: Like a high speed serial interface.

Steve: Perfect. OK. Or better, a new graphics architecture, or a new CPU architecture, or you’re making a new IBM cell architecture. Yeah, that’s on the bleeding edge. We don’t even know if the architecture is going to work. Right? I just want to be clear.

Semiconductors, if you’re just making a faster version of some one else’s chip, you’re not taking too much technical risk, are you? I mean, whether you can push it faster.

But typically in semiconductors, if you’re taking architectural risk, if you really have some insight you believe, or communications hardware.

Pushing the envelope is usually about how deep you can go into packet inspection, to how fast and et cetera. Those are some pretty serious trade-offs. You don’t know if this stuff works until you get first use out of the way. Is that fair?

Student: Yep.

Steve: That’s the technical risk. What’s the market risk in that kind of semiconductor business? What’s the customer risk there? Anybody? You don’t even have to be in the semiconductor business. Yes.

Student: For example, they could really mean technology. They could build a chip.

Steve: Yes.

Student: But for some reason, in benchmarking for the system provider assistance is another. Another vendor. Or even more, the old standard just doesn’t pick up.

Steve: Right. So you could have a neat, new architecture, but your competitor could kick your butt. By convincing all the platform people who have to design your product in, “Oh, listen. That’s so incompatible. It uses 62 Hz, rather than 60 Hz of electricity. You’ll never be able to use it.” By the way, I once convinced an entire industry of that, but that’s another story.

So you could win on technical risk, and lose in market risk in hybrid technology. Give me another example. I picked Semiconductors. What’s another one?

Student: I used to do R&D groups with Blu-ray.

Steve: Perfect, talk to me.

Student: I don’t know much about it, but basically from a technical stand point, it seems like you’re pretty similar technology. But it obviously, Sony and company convinced the company prior to movie studios that they’re better off just shipping their content with Blu-ray rather than the DVD medium.

Steve: So this was the next standard for DVD’s. Right? For the last three or four years. Huge battles over who would be the supplier. Lots of chess games, lots of technical risk.

Because even at the end, they were still playing games with the spec and adding more security layer and what ever. At the end of the day, Blu-ray won. Didn’t win on technology, it won because they finally got a critical mass of people to design in the product.

Now the irony is, who do you think might actually, ultimately win? Who may undercut?

Student: Streaming Hi-Def?

Steve: Streaming Hi-Def. Right? It might be that the current DVD standard might have been the last one that sold upon these that it is. Most people, certainly my kids, don’t go out and ever buy DVD’s.

They download stuff to their iPods or Macs or some thing else, or to streaming video. Oops. They’re all an investment. You might have just built the product that no one else wants.

So I just want to point out that when you look at a startup, ask those fundamental questions on day one. What problem do we think we have besides who are we, what business are we in, and what ever. It’s like, are we going to be risking trying to understand our customers and we ought to try to focus on that.

Or is the focus truly inside the building. Because Steve, it doesn’t matter what customers think unless we really nail this technology. None of this matters.

Credit VentureHacks

Tim's Design Slideshow

2009-03-27T13:42:00.000-07:00

Experience Design: Why Pages Are The Last Thing You Need To Worry About

View more presentations from Tim Richards.

For those of you who don't know Tim, hes a UX guru @ Razorfish, and a very good friend on mine.

Don't Launch.

2009-03-16T11:09:00.000-07:00

First off, what does it mean to launch? Generally, we conflate two unrelated concepts into the term, which is important to clarify right up front.

Announce a new product, start its PR campaign, and engage in buzz marketing activities. (Marketing launch)
Make a new product available to customers in the general public. (Product launch)

In today's world, there is no reason you have to do these two things at the same time. In fact, in most situations it's a bad idea for startups to synchronize these events.

Launching is a tactic, not a strategy. In the right situation, it's a very useful tactic, too. In particular, a marketing launch can help you do three things (courtesy, as is most of my marketing advice, of The Four Steps to the Epiphany):

Drive customers into your sales pipeline. This is the usual reason given for a marketing launch, but for most early stage startups, it's a failure. That's because a marketing launch is a one-time event, and rarely translates into renewable audiences. Worse, if you are not geared up to make the best use of those customers when the launch sends them your way, it's a pretty big waste. And, as we'll talk about in a moment, you don't get a second chance.

Because this reason is so often used as an excuse, I recommend giving it extra scrutiny. Are you really choosing to engage in marketing in places where your potential customers pay attention? Do your customers really read TechCrunch? If not, do not launch there. Even if you must launch to your customers, avoid the urge to also launch in extra places, just because your PR firm can do it at the same time.
Establish credibility with potential partners. In some businesses, especially in certain industries like traditional enterprise software, you simply cannot bring a new product to market on your own. You need to combine your product with others, and this requires partners like OEM's or system integrators. A marketing launch can help you get in the door with those partners, if you're having trouble getting their attention. Again, it's critical to focus your marketing launch on those publications, venues, and channels that your potential partners are paying attention to. If you don't know who the partners are, what they pay attention to, or what kind of message they are open to receiving - it's too early to launch. Do some Customer Development instead.
Help you raise money. If you are having trouble raising money, sometimes a little PR can help. But don't be too sure. When VC's and other investors see PR activity, they are going to expect to see significant traction as a result. If you launch and see only mediocre results, it may actually make it harder to raise money. Sometimes, it can be easier to raise money pre-launch, if the launch is not imminent and there is some fear on the part of investors that they might lose the deal when the launch drives awareness of your company to all their peers.

Those are the potential goals of a marketing launch, but those are not its only effects. It also has causes other tectonic shifts that many startups don't consider:

A marketing launch establishes your positioning. If you don't know what the right positioning is for your company, do not launch. Figuring this out takes time, and few entrepreneurs have the patience to wait it out, because the business plan does such a good job of explaining what customers are going to think. The problem is that customers don't read your business plan.

When you launch with the wrong positioning, you have to spend extra effort and money later cleaning it up. For example, we did some early press (in Wired, no less) for IMVU that called us the next generation of IM and compared us positively to AOL. At the time, we thought that was great. Now, I look back and cringe. Being compared to AOL isn't so great these days, and IM is considered a pretty weak form of socializing. When we finally launched for real, we had to compensate for that early blunder.

Of course, we didn't realize it was a blunder at all. We were actually really proud of the positive coverage. In fact, at that time we were auditing Steve Blank's class at Haas, since he was an early investor. Since we hadn't shown him much in the way of progress recently, we actually brought in the article to show off. I won't recount what happened next (although your can hear us recount it in audio). Suffice to say I can trace my understanding of what it means to launch to that day. We're lucky we had a mentor on board who could call us on the bad strategy before it was too late. Most startups aren't so fortunate.
You have to know your business model. Most startups launch before they've figured out what business they're in. Pay attention to your fundamental driver of growth. If the product needs to be tweaked just a little bit in order to convert users into customers, you want to figure that out before the launch. If the viral coefficient is 0.9, keep iterating until it's 1.1 before you launch. And if your product doesn't retain customers, what's the point of driving a bunch of them to use it? Spend your time with renewable sources of customers and iterate.
You never get a second chance to launch. Unlike a lot of other startup activities, PR is not one where you can try it, iterate, learn, and try again. It's a one-way event, so you'd better get it right. Remember the story about IMVU's early encounter with Wired? When we finally did launch the company, even though our product had grown and changed significantly, Wired didn't cover it.

I wrote a little bit about the epic launch we had at a previous startup in my post Achieving a failure. We really did it well, with a great PR firm and great coverage. New York Times, Wall Street Journal, CNN, the works. But it turned into a crushing defeat, because we couldn't capitalize on all that attention. The product didn't convert well enough, the mainstream customers we were driving weren't ready for the concept, and the event fed expectations about how successful the product was going to be that turned out to be hyper-inflated.

Worse, we tricked ourselves into thinking that what the press said about our success was actually true. And even worse, we'd cranked up the burn rate in order to be ready to handle all those millions of mainstream customers we anticipated. When they failed to materialize, the company was in big trouble.

Why do startups synchronize marketing launch and product launch? I think it has mostly to do with psychology.

Investors push for it. Many investors have a desire to see their companies lauded publicly. This actually makes a lot of sense, if you see the world from their point of view. Third-party validation is one of the few forms of feedback they have available to them. Most investors in startups have a 3, 5 or even 10 year horizon for liquidity. That means they don't really know if they made a good investment for a very long time. Seeing the press talk about what a great investor they are is a great form of feedback. As a bonus, it gives them something to show their partners and LP's.

This trend is so strong, this is actually a question I recommend to screen potential investors: "How do you know it's time to launch the company?" See if their answer is about tactics or strategy.
Founders push for it. Who doesn't want to see their name in print? Investors aren't the only ones with ego invested in the company. In some ways, founders are even worse. How do theyknow they are making progress? They spend so much of their time trying to convince everyone around them that their idea is great and the company is doing well: employees, investors, partners, friends, family, significant others - it's a long list. But when they go to sleep at night, who's there to convincethem that they are making progress? My experience is that many founders actually have a deep anxiety that maybe they are not succeeding. Sure, they are keeping everyone busy, but are they really working on the right things? A marketing launch is a temporary salve for these kinds of worries. Plus, it gives you something you can send home to mom (hi, mom!). Unfortunately, it's not a long-term solution, so it can become a bit of an addiction and, therefore, a huge distraction.
There is also fear of the accidental launch. Companies that are thinking strategically sometimes reason like this: "if we do a product launch, members of the public will see our early product. They'll form their own opinions, maybe see our wrong positioning, and maybe talk to members of the press. By the time we're ready for a marketing launch, it will be too late. Better to launch now and get ahead of the story, or stay in closed beta until we're ready."

In most situations, this fear is misplaced. Here was our experience at IMVU, which I have seen replicated at many other consumer internet startups. We did alienate and mis-position to our early customers. Luckily, if your product isn't good enough to have traction, you simply cannot alienate very many customers - because you can't get them engaged with the product. When you finally do get traction, the millions who see the right positioning will dwarf the few who saw the wrong one. And you can get an astronomical amount of traction before anyone will write about your company of their own accord. IMVU was a top-1000 website in the world, with millions of customers and making millions of dollars without getting any significant press coverage.

In fact, we often felt frustrated when new startups with a fraction of our success got terrific write-ups in Silicon Valley-centric venues. We had to resist the urge to launch just to make that frustration stop. And, more often than not, we'd watch those companies flame out and die while we continued to grow steadily every month. If we'd wasted energy chasing their PR coverage, we'd probably have died too.

So don't combine your product launch with a marketing launch. Instead, do your product launch first. Don't chicken out and do a closed beta; get real customers in through real renewable channels. Start with a five-dollar-a-day SEM campaign. Iterate as fast and for as long as you can. Don't scale. Don't marketing launch.

How do you know you're ready for marketing launch?

When you have a strategy for the launch, which means knowing why you're doing it. Make sure it's solving a problem you actually have, and not one that you think you might have some day.
Know what the success metrics are for the launch. If you know what the strategy is, you'll know how to tell it was a success. Write it down ahead of time, and hold yourself accountable for hitting those objectives.
Know what your fundamental driver of growth is. Make sure the math for your model makes sense. That way, you'll be able to predict the future. When customers come in from your marketing launch, you'll know exactly what they are going to do and how that benefits your business.
Know where, when, and how to launch. If you know what your strategy is, and you know your target well (customers, partners, investors) you will also know where they are paying attention, and what messages they are able to absorb. Hold yourself and your PR agency accountable for developing a high level of understanding of these questions ahead of time.

One last suggestions. Think about the psychological motivations that are driving you to want to launch earlier than makes sense for your company. See if there's anything you can do to address those underlying needs that does make sense. For example, if your employees are feeling frustrated that they don't get much third-party validation for their work, use a board of advisers to fill that role. Bring in people that they (and you) respect to evaluate your progress and make suggestions. In my experience, this has provided an effective boost to morale and also helpful guidance.

When you're ready, enjoy the launch. Until then, resist the urge.

Credit Eric Ries

I want this!

2009-03-02T13:55:00.001-08:00

sickkkk!

also an AWESOME article on Facebook gaming from VentureBeat!

5 warning signs: Does A/B testing lead to crappy products?

2009-03-02T13:37:00.000-08:00

The dangers of the metrics-driven design process
Many readers of this blog are expert practitioners of metrics-driven product development, and with this audience in mind, my post today is on the dangers of going overboard with analytics.

I think that this is an important topic because the metrics-driven philosophy has come to dominate the Facebook/OpenSocial ecosystem, with negative consequences. App developers have pursued short-term goals and easy money - leading to many copycat and uninspired products.

At the same time, it’s clear that A/B testing and metrics culture serves only to generate more data-points, and what you do with that data is up to you. Smart decisions made by entrepreneurs must still be employed to reach successful outcomes. (Thus, my answer to the title question is that no, A/B testing does NOT lead to crappy products, but poor decision-making around data can absolutely lead to it)

So let’s talk about the dangers of being overly metrics-driven - here are a couple of the key issues that can come up:

Risk-averse design
Lack of cohesion
Quitting too early
Customer hitchhiking
Metrics doesn’t replace strategy

Let’s dive in deeper…

#1 Risk-averse design
The first big issue is that when you design for metrics, it’s easy to become risk-averse. Why try to create an innovative interaction when something proven like status/blogging/friends/profiles/forums/mafia/etc already exists? By copying something, you’re more likely to quickly converge to a mediocre outcome quickly, rather than spending a ton of effort potentially creating something bad - but of course this also eliminates amazing, ecstatic design outcomes as well.

This risk-averse product design can lead to watered down experiences that combine a mish-mash of features that your audience has already seen elsewhere, and done better too. So while it’s an efficient use of effort, it’s unlikely that your experience will ever be a great one. It’s a recipe for mediocrity.

Risk aversion is responsible for a whole bunch of bad product decisions outside of the Internet industry as well: Why do Hollywood sequels get made, even though they are usually much worse than the original? Why do companies continually do “brand extensions” that dilute the value of their brand position? The reason is that it’s an efficient thing to do, and it’s pretty easy to make some money even if the end product is not that great. But it’ll hurt in the long run, since these products will inherently be mediocre rather than great.

In my opinion, the only way to avoid this is to never get lazy about design, and to always take the time to create innovative product experiences. Of course you’ll always have parts of your product which will borrow from the tried-and-true, yet I think it’s always important that the core of the experience is differentiated and compelling.

#2 Lack of cohesion
As hinted above, the next issue is that A/B tested designs often create severe inconsistency within an experience. The bottoms-up design process that results from lots of split testing is likely to come up with many local effects, which may rule global design principles.

Here’s a thought experiment to demonstrate this: Let’s say you tested every form input on your website, with different labels, fonts, sizes, buttons, etc. You’re likely, if you picked the best-performing candidate, to have wildly different looking forms across the site. While it may perform better, it also makes the experience inconsistent and confusing.

Ultimately, I think resolving this has to do with striking a balance between global design principles and local effects. One great way to do this is to split out the extremely critical parts of your product funnel to be locally optimized, and keep the rest of the experience the same. For a social gaming site, the viral loop and the transaction funnel should be optimized separately, whereas the core of the game experience should be very internally consistent.

#3 Quitting too early
Another way to get to uninspired products is to quit too early while iterating an experience because of early test data. When metrics are easy to collect on a new product feature, it’s often very tempting to launch a very rough feature and use the initial metrics to judge the success of the overall project. And unfortunately, when the numbers are negative, there can be a huge urge to quit early - this is a very human reaction to wanting to not waste a bunch of time on something that’s perceived to fail.

Sometimes a product requires features A, B, and C to work right, and if you’ve only done A, it’s hard to figure out how the entire experience will work out. Maybe the overall data is negative? Or maybe it inspire dynamics that go away once all the features are bundled? Interim data is often just that - interim. But people are great at extrapolating data, but sometimes the right approach is just to play out your hand, see where things go, and evaluate once the entire design process has completed.

#4 Customer hitchhiking
A colleague of mine once used the term “customer hitchhiking” to describe how it’s easy to follow the customer on whatever they want to do, rather than having an internal vision of where YOU want to go. This can happen whenever the data overrules internal discussion and resists interpretation, because it’s so uncompromising as hard evidence. The important thing to remember, of course, is that the analysis is only as good as the analyst, and it’s up to the entrepreneur to put the data into the context of strategy, design, team, and all the other perspectives that occur within a business.

Today, I think you see a lot of this customer hitchhiking whenever companies string together a bunch of unrelated features just to please a target audience. This reminds me of what is often called the “portal strategy” of the late-90s. Just combine a bunch of stuff in one place, and off you go. The danger of that, of course, is that that it leads to incoherent user experience, company direction, and numerous other sources of confusion.

In the Facebook/OpenSocial ecosystem, of course, this manifests itself as companies that have many unrelated apps. You can dress this up as a “portfolio” or a “platform” but at the same time, it can be a recipe for crappy product experiences.

#5 Metrics doesn’t replace strategy
What do you think it would be like to write a novel, one sentence at a time, without thinking about the broader plot? I’m sure it’d be a terrible novel, and similarly, I bet that testing one feature at a time is likely to lead to a crappy product.

Ultimately, every startup needs to decide what they want to do when they grow up - this is a combination of entrepreneurial judgement, instinct, and strategy.

Every startup has to figure out how big the market is, they have to deliver a compelling product, and they need a powerful marketing strategy to get their services in front of millions of people. Without a long-term vision of how these things will happen, an excessive amount of A/B testing will surely lead to a tiny business.

To use a mountaineering analogy: Metrics can be very helpful in helping you scale the mountain once you’re on top of the right one - but how do you figure out whether you’re scaling the right peak? Analytics are unlikely to help you there.

Conclusions
My point on this - nothing is ever a silver-bullet, and as much as I am an evangelist for metrics-driven approaches to startup building, I’m also very aware of the shortcomings. In general, these tools are great for optimizing specific, local outcomes, but they need to be combined with a larger framework to reach big successes.

Ultimately, quantitative metrics are just another piece of data that can be used to guide decision-making for product design - you have to combine this with all the other bits of information to get it right.

Credit Andrew Chen

The Social Technographics® Of Business Buyers

2009-02-26T09:45:00.001-08:00

From Jeremiah Owyang

The Social Technographics® Of Business Buyers

View more presentations from Oliver Young. (tags: oliver_young laura_ramos)

More Multiplayer Game Atoms

2009-02-26T09:38:00.001-08:00

Another Great Diagram

Skill Chain Mechanics & Atoms

2009-02-25T11:00:00.000-08:00

A very interesting breakdown of the Skill Chain in Tetris. It seems very applicable to the streams of Casual Gaming, and since the market is moving toward a much more reward and achievement based model (via an API), it seems even more relatable.

Download the Full PDF Here

Creative destruction … Google slayed by the Notificator?

2009-02-09T11:51:00.000-08:00

The web has repeatedly demonstrated its ability to evolve and leave embedded franchises struggling or in the dirt. Prodigy, AOL were early candidates. Today Yahoo and Ebay are struggling, and I think Google is tipping down the same path. This cycle of creative destruction — more recently framed as the innovators dilemma — is both fascinating and hugely dislocating for businesses. To see this immense franchises melt before your very eyes — is hard to say the least. I saw it up close at AOL. I remember back in 2000, just after the new organizational structure for AOL / Time Warner was announced there was a three day HBS training program for 80 or so of us at AOL. I loath these HR programs — but this one was amazing. I rememberKotter as great (fascinating set of videos on leadership, wish I had them recorded), Colin Powell was amazing and then on the second morning Clay Christensen spoke to the group. He is an imposing figure, tall as heck, and a great speaker — he walked through his theory of the innovators dilemma, illustrated it with supporting case studies and then asked us where disruption was going to come from for AOL? Barry Schuler — who was taking over from Pittman as CEO of AOL jumped to answer. He explained that AOL was a disruptive company by its nature. That AOL had disruption in its DNA and so AOL would continue to disrupt other businesses and as the disruptor its fate would be different. It was an interesting argument — heart felt and in the early days of the Internet cycle it seemed credible. The Internet leaders would have the creative DNA and organizational fortitude to withstand further cycles of disruption. Christensen didn’t buy it. He said time and time again disruptive business confuse adjacent innovation for disruptive innovation. They think they are still disrupting when they are just innovating on the same theme that they began with. As a consequence they miss the grass roots challenger — the real disruptor to their business. The company who is disrupting their business doesn’t look relevant to the billion dollar franchise, its often scrappy and unpolished, it looks like a sideline business, and often its business model is TBD. With the AOL story now unraveled — I now see search as fragmenting and Twitter search doing to Google what broadband did to AOL.

Video First

Search is fragmenting into verticals. In the past year two meaningful verticals have emerged — one is video — the other is real time search. Let me play out what happened in video since its indicative of what is happening in the now web. YouTube.com is now the second largest search site online — YouTube generates domestically close to 3BN searches per month — it’s a bigger search destination than Yahoo. The Google team nailed this one. Lucky or smart — they got it dead right. When they bought YouTube the conventional thinking was they are moving into media – in hindsight — its media but more importantly to Google — YouTube is search. They figured out that video search was both hard and different and that owning the asset would give them both a media destination (browse, watch, share) and a search destination (find, watch, share). Video search is different because it alters the line or distinction between search, browse and navigation. I remember when Jon Miller and I were in the meetings with Brin and Page back in November of 2006 — I tried to convince them that video was primarily a browse experience and that a partnership with AOL should include a video JV around YouTube. Today this blurring of the line between searching, browsing and navigation is becoming more complex as distribution and access of YouTube grows outside of YouTube.com. 44% of YouTube views happen in the embedded YouTube player (ie off YouTube.com) and late last year they added search into the embedded experience. YouTube is clearly a very different search experience to Google.com. A last point here before I move to real time search. Look at the speed at which YouTube picked up market share. YouTube searches grew 114% year over year from Nov 2007 to Nov 2008!?! This is amazing — for years the web search shares numbers have inched up in Google favor — as AOL, Yahoo and others inch down, one percentage point here or there. But this YouTube share shift blows away the more gradual shifts taking place in the established search market. Video search now represents 26% of Google’s total search volume.

The rise of the Notificator

I started thinking about search on the now web in earnest last spring. betaworks had invested in Summize and the first version of the product (a blog sentiment engine) was not taking off with users. The team had created a tool to mine sentiments in real-time from the Twitter stream of data. It was very interesting — a little grid that populated real time sentiments. We worked with Jay, Abdur, Greg and Gerry Campbell to make the decision to shift the product focus to Twitter search. The Summize Twitter search product was launched in mid April. I remember the evening of the launch — the trending topic was IMAP — I thought “that cant be right, why would IMAP be trending”, I dug into the Tweets and saw that Gmail IMAP was having issues. I sat there looking at the screen — thinking here was an issue (Gmail IMAP is broken) that had emerged out of the collective Twitter stream — Something that an algorithmically based search engine, based on the relationships between links, where the provider is applying math to context less pages could never identify in real time.

A few weeks later I was on a call with Dave Winer and the Switchabit team — one member of the team (Jay) all of a sudden said there was an explosion outside. He jumped off the conference call to figure out what had happened. Dave asked the rest of us where Jay lived — within seconds he had Tweeted out “Explosion in Falls Church, VA?” Over the nxt hour and a half the Tweets flowed in and around the issue (for details see & click on the picture above). What emerged was a minor earthquake had taken place in Falls Church, Virginia. All of this came out of a blend of Dave’s tweet and a real time search platform. The conversations took a while to zero in on the facts — it was messy and rough on the edges but it all happened hours before main stream news, the USGS or any “official” body picked it up the story. Something new was emerging — was it search, news — or a blend of the two. By the time Twitter acquired Summize in July of ‘08 it was clear that Now Web Search was an important newdevelopment.

Fast forward to today and take a simple example of how Twitter Search changes everything. Imagine you are in line waiting for coffee and you hear people chattering about a plane landing on the Hudson. You go back to your desk and search Google for plane on the Hudson — today — weeks after the event, Google is replete with results — but the DAY of the incident there was nothing on the topic to be found on Google. Yet at http://search.twitter.com the conversations are right there in front of you. The same holds for any topical issues — lipstick on pig? — for real time questions, real time branding analysis, tracking a new product launch — on pretty much any subject if you want to know whats happening now, search.twitter.com will come up with a superior result set.

How is real time search different? History isnt that relevant — relevancy is driven mostly by time. One of the Twitter search engineers said to me a few months ago that his CS professor wouldn’t technically regard Twitter Search as search. The primary axis for relevancy is time — this is very different to traditional search. Next, similar to video search — real time search melds search, navigation and browsing. Way back in early Twitter land there was a feature called Track. It let you monitor or track — the use of a word on Twitter. As Twitter scaled up Track didn’t and the feature was shut off. Then came Summize with the capability to refresh results — to essentially watch the evolution of a search query. Today I use a product called Tweetdeck (note disclosure below) — it offers a simple UX where you can monitor multiple searches — real time — in unison. This reformulation of search as navigation is, I think, a step into a very new and different future. Google.com has suddenly become the source for pages — not conversations, not the real time web. What comes next? I think context is the next hurdle. Social context and page based context. Gerry Campbell talks about the importance of what happens before the query in a far more articulate way than I can and in general Abdur, Greg, EJ, Gerry, Jeff Jonas and others have thought a lot more about this than I have. But the question of how much you can squeeze out of a context less pixel and how context can to be wrapped around data seems to be the beginning of the next chapter. People have been talking about this for years– its not that this is new — its just that the implementation of Twitter and the timing seems to be right — context in Twitter search is social. 74 years later the Notificator is finally reaching scale.

A side bar thought: I do wonder whether Twitter’s success is partially base on Google teaching us how to compose search strings? Google has trained us how to search against its index by composing concise, intent driven statements. Twitter with its 140 character limit picked right up from the Google search string. The question is different (what are you doing? vs. what are you looking for?) but the compression of meaning required by Twitter is I think a behavior that Google helped engender. Maybe, Google taught us how to Twitter.

On the subject of inheritance. I also believe Facebook had to come before Twitter. Facebook is the first US based social network — to achieve scale, that is based on real identity. Geocities, Tripod, Myspace — you have to dig back into history to bbs’s to find social platforms where people used their real names, but none of these got to scale. The Twitter experience is grounded in identity - you knowing who it was who posted what. Facebook laid the ground work for that.

What would Google do?

I love the fact that Twitter is letting its business plan emerge in a crowd sourced manner. Search is clearly a very big piece of the puzzle — but what about the incumbents? What would Google do, to quote Jarvis? Let me play out some possible moves on the chess board. As I see it Google faces a handful of challenges to launching a now web search offering. First up — where do they launch it, Google.com or now.Google.com? Given that now web navigational experience is different to Google.com the answer would seem to be now.google.com. Ok — so move number one — they need to launch a new search offering lets call it now.google.com. Where does the data come from for now.google.com? The majority of the public real time data stream exists within Twitter so any http://now.google.com/ like product willaffirm Twitter’s dominance in this category and the importance of the Twitter data stream. Back when this started Summize was branded “Conversational Search” not Twitter Search. Yet we did some analysis early on and concluded that the key stream of real time data was within Twitter. Ten months later Twitter is still the dominant, open, now web data stream. See the Google trend data below - Twitter is lapping its competition, even the sub category “Twitter Search” is trending way beyond the other services. (Note: I am using Google trends here because I think they provide the best proxy for inbound attention to the real time microbloggging networks. Its a measure of who is looking for these services. It would be preferable to measure actual traffic measured but Comscore, Hitwise, Compete, Alexa etc. all fail to account for API traffic — let alone the cross posting of data (a significant portion of traffic to one service is actually cross postings from Twitter). The data is messy here, and prone to misinterpretation, so much so that the images may seem blurry). Also note the caveat re; open. Since most of the other scaled now web streams of data are closed / and or not searchable (Facebook, email etc.).

Google is left with a set of conflicting choices. And there is a huge business model question. Does Ad Sense work well in the conversational sphere? My experience turning Fotolog into a business suggests that it would work but not as well as it does on Google.com. The intent is different when someone posts on Twitter vs. searching on Google. Yet, Twitter as a venture backed company has the resources to figure out exactly how to tune AdSense or any other advertising or payments platform to its stream of data. Lastly, I would say that there is a human obstacle here. As always the creative destruction is coming from the bottom up — its scrappy and and prone to been written off as NIH. Twitter search today is crude — but so was Google.com once upon a not so long time ago. Its hard to keep this perspective, especially given the pace that these platforms reach scale. It would be fun to play out the chess moves in detail but I will leave that to another post. I’m running out of steam here.

AOL has taken a long time to die. I thought the membership (paid subscribers) and audience would fall off faster than it has. These shifts happen really fast but business models and organizations are slow to adapt. Maybe its time for the Notificator to gopublic and let people vote with their dollars. Google has built an incredible franchise — and a business model with phenomenal scale and operating leverage. Yet once again the internet is proving that cycles turn — the platform is ripe for innovation and just when you think you know what is going on you get blindsided by the Notificator.

Note: Gerry Campbell wrote a piece yesterday about the evolution of search and ways to thread social inference into search. Very much worth a read — the chart below, from Gerry’s piece, is useful as a construct to outline the opportunity.

Credit : John Borthwick