DIRTY LITTLE LIARS: WHY ARE THERE PRODUCT FAILURES? PRO TIP ! the recording is available if you click on the title link // Mahin Samadani, Charlie Burgoyne, Charles Marcus, Tracy Thirion #dirtyliars
Charles Marcus – Co-Founder, Vidlet (moderator)
Charlie Burgoyne – Founder/CEO, Valkyrie Intelligence (was an astro physicist first, and then left and went to work at Frog)
Mahin Samadani - Partner, McKinsey and Company (started on Newton team at Apple - one of biggest product failures of our lifetimes)
Tracy Thirion - CEO, Bamboo Worldwide
Session Description (directly from SXSW website):
Google Glass and Heinz Purple Squirt Ketchup. How can these products be such market failures? We are swimming in data, yet it's a fact that over 50% of products fail every year. Why is this happening despite having so much data easily accessible to us? Of course, much of the answer is that humans are messy and unpredictable. How do we cope with unpredictability and find meaningful insight? In this session, we will explore the rise of quant analysis and the fall of qual analysis. Are we better off?
Real life blatant example of when quantitative methods/research/analysis : 2016 election. Michael Moore was gathering qualitative data in rural parts of the US and the writing seemed to be on the wall, but we latched onto what we feel like is safer data (quantitative).
Techniques and machine learning thoughts:
- In one of Charles Burgoyne's jobs, the company dropped traditional terms like gender, race, etc. and instead created new architypes
- building tools that allow us to build things that will push you places you can’t even fathom, we'll be designing for 45 architypes
- The next 10 years in AI : we will be creating products that conform to people vs. expecting people to conform to products because products need to be designed using iterative steps and AI can be programmed to do this without a huge manual time investment
- bringing a human element to data
- Duolingo example : instead of being obsessed w/ average person, they built a platform that adapts to each user. Changing pedagogies and taxonomies and changing entire experience based on the individuals
- The products can and will fail, but they will heal when they do
- Good designers are those who both understand customers and data and can empathize with both customers and data and data scientists (or be both in one)
- The designer and the scientist become a more fluid, evolved idea.
- Tracy said she doesn’t think adults can learn empathy (I almost entirely disagree with this, but I could also be wrong)
- we're loosing empathy in our society largely because of our digital tools we're building
- example of woman cleaning house: quantitative analysis would provide things like; how many hours per week she cleans, what types of products she uses, her patterns. What would be missed is something important that was only revealed during conversation: she has deep rooted emotion tied to cleanliness and her mother, whenever she spoke about her mother, she would grab her neck.
Qualitative vs. Quantitative : qualitative is the spark, quantitative is the fuel. The qualitative research is the why, it’s the influencing and emotional impact. There’s a reluctance for clients to want to pay for the qualitative research because it’s more nuanced and uncertain, however it builds confidence through depth and understanding. It’s a harder sell to the boss. Or the bosses bosses boss. Problem in market industry: there are not a lot of good qualitative moderators available.
What’s the process of bringing qualitative research into quantitative research? It’s the blending of insight with analogies. Doing so can allow the processes to get directionally dangerous, allow space to fail, and working toward a solution. This will require a massive cultural shift to combine these methods and be ok with failing, to see it as a necessary part of the process and not as a negative.
Training people on how to sell ideas: create an emotional connection to the idea through compelling stories.
Why do products fail?
- Because they lack empathy or
- There’s a lack of diving more deeply into the data
Contextual research: co-creating products alongside users. Story: Maheld’s firm has a project of working with a utility company to transition from analog maps/highlighters/binders to a digital system. The designers rode along in the trucks with the drivers to gain insight in how they use their tools to perform their job. The boss was certain that the designers wouldn’t be able to tell him anything he didn’t already know, but he was able to.
The most basic machine learning is the survey.
Qualitative, quantitative, qualitative, quantitative
Tension of a business decision vs. streamlining a process
The ethical layer – responsibility of the designer to help make sure the machines don’t do EVERYTHING and mess it up.
Why retro packaging works: it’s a safe bet, unlikely to offend anyone.
Tina Koyama: product manager @ Twitter. She’s swiss/Japanese (she pointed this out)
Jill Nussbaum: design manager at Instagram in NY. Worked at R|GA, worked on Nike and has been a teacher for 9 years. She teaches a class called Design in Public Space.
John S Couch: currently at Hulu, previously ebay, Magento, Wired Japan, then startup. SF – LA
Their brands are in the spotlight currently.
Particular project with strong audience reaction:
Jill talked about the Instagram rebranding project: biggest logo and UI change to date. started with research specifically focusing on the core element of the brand that people connect with (rainbow and camera lens). There will always be lovers, haters, and mime creators. The UI changes improved user experience a ton.
IG did a good job of PREPARING THE COMPANY as a whole for the rebrand. The involved everyone in the company all through the redesign process so everyone was excited and felt a proud ownership when the new UI launched. The truth for how successful the product is is always somewhere in the middle between the SUPER negative and SUPER positive feedback (outliers). Don't get stuck in overhype or in trolls.
Tina talked about Twitter's intentional redesign and had a significant shift from a social media app to a news app. This change happened about a year ago. On the tweet, there's a retweet, like, comment and significant change is real time for like and retweet counts (animation). Realtime user feedback. When things break on the internet, things break on TW also, so they have immediate feedback.
- No one on the team is ever surprised by feedback on TW because they design with their users along the way and focused on the most important part of the brand
Why did IG change the design?
Instagram hadn't had significant change since launch in 2010, community had grown and products had grown, so they decided they needed a change as well, it was time for the brand to also grow. It's was scary!
Why did TW change the design?
For TW, it was more about focusing on the most important piece of their brand and focusing efforts to communicate. Tina would describe it as a focus rather than a product change.
Why did Hulu change?
Ostensibly to bring live into the product. Also, they hadn't changed fro 10 years. John saw an opportunity to bring OTT (over the top) and also an opportunity to entirely change the user experience and marketing/brand focus. They made a video of people using the mobile product. They got the new product launched in 14 months. Launched with new show Handmaids Tale. He says it’s important for the WHOLE COMPANY to be an advocate for the end user and that this core piece sometimes gets lost.
Narrative, continuous storyline is important for designers to keep sight of this so it doesn't get lost in all the KPIs, data, etc.
Focus of Tina's class she teaches is the narrative. Her students create what she calls design fiction. It's important to tell the story of the product in the near future to get buy in and excitement for everyone. Can also show a few different paths they can take – create stories and films of people using the product.
3 ways they learn at Instagram:
- Foundational research (in the field, in homes, businesses, understand the challenges) or lab research
- Building quick prototypes and dog fooding it in their internal test app and then run experiment in real app (maybe 1% of users)
- Run experiment in real app (maybe 1% of users)
^^ go into understand phase quickly so they can understand if they're moving in the correct directly.
Twitter has a similar process to Instagram. Design is always partnered with research. – challenging to design products from end to end in a big company. Shifted entire company to a jobs-to-be-done framework, very user-centric framework. This gets everyone thinking about the user experience as a whole vs. individual products. They ask themselves:
- When do people hire twitter? And, just as importantly or maybe more importantly
- When do people fire twitter?
Is there anyone in the company who owns overall user experience end to end?
- One example of a brand who does this well is Apple: they have continuity throughout their entire user experience from someone passively observing a billboard to someone browsing their website
- At Twitter, the CEO Jack Dorsey is the visionary who aligns everyone and evangelizes all of the principles really well throughout company
- At Instagram, they have a slogan that says “nothing is someone else’s problem” so the expectation is that everyone is responsible. They have a series of internal reviews and a weekly design review. Design leadership from across company, not just your product team. It's the responsibility of each design lead to know what is going on across the company. So, even though there are 40 designers at IG, the end user experience feels like there's one designer.
Companies are very design-centric, do you ever have to explain what DESIGN is?
- There are a lot of different designers at Instagram who have different skillsets. Instagram is a design lead company and one of the most important questions they area always asking is who should we bring in at which point in the project.
- HULU is a mix of Hollywood + technology + design which makes evident the omnipresent tension between Hollywood and silicone valley. Hollywood is established (100 years) and more static while Silicone Valley is agile. With the different players converging at HULU, the company had to define what design means for them. Hulu defined roles of UX vs. product design vs. information architecture vs. motion designers vs. communication design
Did you anticipate the reactions to the redesign when launched?
Instagram : Knew there would be a strong reaction, prepped the company as a whole and tried to be as transparent as possible (w/things like videos). When feedback came out, they listened, but gave more weight to the analytics or anecdotal and how customers were actually using the product
Twitter: anticipated reactions and they work so closely with users for the entire process of the redesign. They wanted to bring the brand to life and The big things they did were changed typography for headlines to make them more readable and be able to easily differentiate the content being shared vs. personal
- Changed reply icon to be more universally recognizable
- differentiate content vs. person behind it (shapes)
- Iconography changes; changed reply from back arrow to comment bubble (universally recognized)
- All changes informed by user feedback, so user responses were quite positive
- Twitter prepped users/were transparent by launching blog posts about their process
- They recently launched their design blog, about 6 months ago
- RADICAL TRANSPARENCY translates well
Hulu got ahead of it with videos and how-to use the new products. "Disruption that doesn't serve the user is just indulgent." - John. So make sure the improvements help the customer and communicate the why and the process.
- Hulu used videos as teasers ‘this is coming’
- TV is a more passive medium compared to TW and IG, the mobile experience has an opportunity to bring more interaction than the living room hulu experience
Most Valuable Product (MVP): for each redesign, did you launch what you wanted?
Twitter : design systems project let them launch in a couple months (WHAT). Because they built a strong foundation so it was easier to launch from there.
Instagram : they have a design systems team (SO IMPORTANT) so they can solve problems without having to solve the entire UI for the whole product. They looked at how users in Indonesia were hacking IG as a platform for ecommerce. They were using hashtags to group like products. They hacked search by putting the word ‘pants’, for example, in as the name so their pants for sale would come up easily when someone searched pants.
Hulu: it was a year-long launch in which they interated and grew as they went.
They all spoke about the importance of getting offsite and starting initial design phase on paper.
Closing remarks, one piece of advice, mantra:
Jill, Instagram: one of IG's design principles; do the simple thing first. Design all of the things and all of the possible solutions and then figure out (through data, research, rigor) and then, from there, iterate and simplify.
Tina, Twitter: so important for us to understand the people who use the product, especially international and those not around us. Empathy. Designing for 10 people is very easy, designing a universal experience that works across countries is. Soft skills: very important to work in a large organization; how to negotiate, persuade, present well.
John, Hulu: courage. A lot of designers are introverts, so it can be hard to stand up for design. Being exposed while creating; the data as well as the craft. And then stand with a design through critiques. The greatest designs are when everyone initially thinks you're completely out of your mind, but then they start to tip and see that you're absolutely right. Mantra: be courageous about what you're doing and being ok if it fails.