#Build 7: 3 Questions Every Product Team Needs to Ask w/ Sandhya Hegde
#Build 7: 3 Questions Every Product Team Needs to Ask w/ Sandhya Hegde
Maggie: Welcome to another episode of Build. On Seeking Wisdom, this is Maggie, and today I'm so lucky and excited to have Sandhya Hegde with me, doing something a little different than usual, because she's actually the Global VP of Marketing at Amplitude, a leading product analytics software and was in products before. She was a General Partner at Khosla Ventures, a Product Manager, and the list goes on. So Sandhya, welcome to Seeking Wisdom.
Sandhya Hedge: Thanks, Maggie. Delighted to be here.
Maggie: So there's so much I want to dig into. Especially in product analytics, since you're at the epicenter of that over at Amplitude. But first I have to ask about your switch to marketing. I honestly haven't seen a ton of people make that leap, but I'd love to hear your journey to Amplitude and why as part of that, you've switched from product into marketing.
Sandhya Hedge: Yes. When I joined Amplitude first, I was already not your typical Product Manager, but I joined as the Lead Product Manager in Amplitude because I was really passionate about product data science. As an early stage investor, I worked with so many companies that get something right and lock their way into some product market fit. But the first time that something they ship does not work, they can't understand why. And often they struggle to invest in the right analytical frameworks needed to really understand customer set scale when you go past that early fit where, you know who your 100 power users are, and they are your compass, and now you have too many of them and you don't understand what's changing. So I've always been really passionate about making sure that you have the right kind of data investments made in a company that you can really take advantage of that moment when you find fit, and continue to take advantage of it as your customer base changes, as your company changes. So that's what brought me into Amplitude and I worked on a lot of our products. I also started working on building out our product marketing team. And that's when I realized a couple of things. So as a product marketing team, we designed the go- to market strategy for what we were building. And it was pretty obvious that a lot of our target customers, our buyers, who are essentially product managers, product leaders, people who run digital businesses.
Sandhya Hedge: They didn't want to be sold their traditional way. And to be honest, there's so much about traditional marketing right now that no one really enjoys, right?
Sandhya Hedge: No one wants the kind of, click- bait, cushioned, as content marketing, and random events that are just thinly [ inaudible 00: 02: 56 ] sales meetings that don't really value to your life. There's so much happening in traditional enterprise marketing right now that needs to be disrupted. And I think Drift is also one of those companies that really sees that. Allow a lot of what you are bringing to the marketing world in terms of modernizing it.
Maggie: I mean, we're all about making things human, and making things real. And I'm just nodding along with what you're saying, because in a past life, I did have to purchase some analytic software. And I remember going through this process and thinking, I've already done all the research, I've done the Googling, can we just have a chat, face- to- face. One person to one person and actually solve this problem.
Sandhya Hedge: Yeah, absolutely. And so our buyer is probably at the forefront of the modern buyer who is really looking for a partner, not like a vendor, and a checklist of features, but someone who will help them do their job better. And we wanted to completely transform our marketing team to meet our customers where they wanted us to be. Which is, how can we help you as opposed to, how can we collect your email ID and phone number? So a very different approach to take. So we are building out an extremely product led marketing team, which uses all of those principles that product managers love. Which is very much, what's the job required to be done? What's the job that we need to complete with any investment we are making? How do we make sure that anything we do actually solves some customer pain and actually addresses a gap in their workflow that they need to get addressed? And long- term how do we make sure that we keep helping them do their job better with every investment and everything we are doing in marketing? So that has manifested itself in almost... It has changed how we are executing on marketing strategy, even though the things that we are doing are essentially look the same. The kind of work that goes into it and the value that our customers and prospects get out of it as dramatically different. So we're extremely focused on helping them.
Maggie: Yeah. So it sounds like you're taking the traditional product stuff and bringing it over to marketing, but then it also kind of sounded like you're staying with the customer longer. What are the big differences from traditional marketing into what you're doing outside of just bringing in the product thinking?
Sandhya Hedge: Yeah. I think the big, important difference is that whenever we look at the goal for a certain marketing investment, right? Whether it's a conference, it's a meet up, it's just like what we're doing with our website. We're aligning everything to specifically help our target buyer and our audience be better at their jobs. So for everything we invest in the question for us is, how does this help this person be a better product manager? How does this help out to get this person to be a more effective VP of digital, if that's what their job is. How does it help them get better? So the actual activities that we perform can look very different. And that also helps us understand why should this content be gated. Does that actually help our customer to gate this content, versus, if we're doing a conference, does it help our customer for this session to be about how this particular team uses Amplitude, versus, what is the latest and greatest growth product strategy that would help this person accomplish your job better? So a lot of the choices that you make get simplified because of the overarching where you define the goal of our marketing team. And also it's very product first. So we get out of the way, we make sure that our customers get their hands on the product and effectively become better product managers while they are interacting with our marketing function, as opposed to having the more traditional marketing metrics and goals, which are very focused on lead and driving leads.
Maggie: Yeah. So do you not use those traditional MQL, SQL numbers at all?
Sandhya Hedge: Not quite. It's a transition in progress. So we started doing this about a month or so ago, and eventually we want to move away from all of those traditional definitions to better align it, to what our customers would describe their journey as. Which is, right now do I feel like I know what my product analytics strategy should be? Do I understand what stack I need to implement? Do I feel like I can make the best possible use of all the tools that my team is using? Have I been able to change my team culture? So there is a much better list of qualifications that your customers would ask if you were asking them, how far along the journey you are, and we want to align all of our work to that customer journey.
Maggie: Right. Okay. I want to get into the product analytics in a second, but first I just have to ask. You're coming in from product into marketing. You're having this more product focused marketing role and team, but how did you get up to speed at least on the original marketing situation in the first place? How did you? Did you have a crash course in what marketing was, or were you already familiar with that? How did you learn that so quickly?
Sandhya Hedge: Yeah. Great question. I'm still learning. I'm a lifelong learner. One of the things that really gets me really excited and makes me feel fulfilled is having to learn a completely new set of skills, and acquiring knowledge really rapidly about something. And some of that is of course, just reading the right books over time. And as an investor, I did want to have enough familiarity with modern marketing that I had built up enough context around what are companies doing well and not around me. But I think the most useful thing that I invested in was just finding the right advisors. And having the right advisors, and mentors who have really done this and can be a good sounding board for you, and make sure you don't make obvious mistakes, was the upfront of few months of investment that I made that really helped me out.
Maggie: Okay, awesome. So you've been in it for a month. Now you're sort of full on in this marketing role, product analytics. So what's the secret? You're at the center at Amplitude, what should we all be doing with our marketing and our metrics?
Sandhya Hedge: That's a great question. I think the beautiful thing about Amplitude is that though we usually go and work with product teams to enter a company, before long as we are successful in a company. We see that pretty much the whole company has adopted Amplitude. So it goes from product, to the analytics team, to marketing. Especially growth led marketing, people are focused on full life cycle ROI of our user acquisition. And then it's...
Maggie: Sorry, is that because all of these teams are all of a sudden starting to care about the same metrics? Or is that because they're putting more different types of data into your platform?
Sandhya Hedge: I think it's actually a lot the former. The fact that we have more companies realizing that they have these siloed functions that are suboptimal in terms of how they are operating. So for example, we had this one really large fortune 50 customer. Where there are about 2, 400 people using Amplitude right now. And at first it was very much, the PMs need something to understand what features are working, right? That was the initial use case that they could imagine that they would use Amplitude for, and very much a core value prop of our platform. And once they started using it, they realized that they had a lot of data and insights that other teams had been dying to access. So for example, they looked at their data set and said, we have these marketing campaigns that seem to be working really well to get new registered users into the platform. But none of them are converting. So this was data that the marketing team at that point didn't have access to. The product team had access to. And they went and quickly did some analysis to realize that all of that traffic was coming from click farms. And they were essentially just sucking millions of dollars of marketing budget by clicking and registering. And then they just simply didn't convert. So that kind of insight kicked off more collaboration where now Amplitude is becoming the source of truth for their entire cloud team. And with that shared source of truth, really unlocks a huge amount of innovation, because now we have a lot of different functions that can align on, what are the priorities. And many more people have the ability to pitch ideas that are going to push things to the next level. And that's what has surprised them, and really surprised us too. The difference between 10 people in a big company, really knowing the data, and 2000 people in the company, knowing the data. If you have a 100X more ideas on the table that are actually good ideas. And people can look at the data to qualify the ideas themselves. There's just such amazing democracy of initiative that gets created when you have the shared source of truth that empowers people who don't have political clout. Who can't get the attention from a VP or something, but just can genuinely defend the greatness of their idea and get their team excited about it because they all see the same thing. So that has been what has really changed Amplitude's journey as a company as well. Which is that we realized that when originally we were thinking of the problem we are solving as just... Product needs more data science, there are not enough data scientists in the world, and kind of solve that problem. And what we realized was that we were really changing the way businesses could operate, and the speed at which they could operate if they were trying to modernize themselves. This was a fundamental investment that they needed.
Maggie: Right. So then having been on both sides of this problem, how do you recommend that people go about starting this analytics journey, and bringing more data into product, because I think I've worked at environments where it's all the way on one side where everything is a test, and everything is data- driven and there's almost no qualitative research. And then also I think the bias, or the stereotype for super early stage companies is that, like you were saying before, there's so few customers, there are so few people, they don't really need analytics or data. So how do you think about that and where do you think the sweet spot is? And how can we start to think about this problem?
Sandhya Hedge: Yeah, that's a great question. I think as a company, Amplitude does a lot of qualitative research, too. So when we talk about data, it's not to the exclusion of qualitative research at all. But even having great data that everyone can use helps us do better qualitative research. So for example, when our design and UX team reaches out to customers to talk to them, they target customers based on what those customers have already done in the product. So they will say, look overall, we expected that at least, X- 1000 people would actually interact with this new feature that we launched in snow version one, but we expected these many people to interact. And we released it to this group and only X percent of them interacted. So they can actually target, how many people do we want to send UX interview requests to from the group that did interact and succeeded, versus those who did not interact, or interacted but didn't succeed. They can actually be super targeted in terms of who they go after and how they frame their qualitative research. And it's really easy to then customize it and say, hey, we saw that you did A, B, and C and we'd love your feedback on why you did not do D.
Sandhya Hedge: And the kind of response rates we get for our qualitative research are extremely high because of that, because it's not just like here's a standard survey that doesn't actually apply to you. It's very, very targeted, and based on the behavior that that customer did. So I do think that there's a lot of value in that. I think if you only have a thousand customers and you don't have the capital to make these investments, sure, it doesn't make sense. But I think at the point where you have an initial idea of what product market fit looks like, and what you're trying to measure. You do need to make this investment because without that, the first time you get into trouble, you are lost.
Maggie: Right. And I imagine you might not even know when you get into trouble because you're not even paying attention to it, right?
Sandhya Hedge: Yeah, absolutely. I think if you are paying attention just to traffic, instead of leading indicators, that a lot of your product roadmap is not getting the kind of results that you want from it. You are going to be in trouble because that takes so much time to turn around.
Sandhya Hedge: That takes a year turn around. So one of the things that we do is we also talk about how making investments in analytics tools is not going to solve the problem. You have to employ a lot of the best practices that come with it to really take advantage of data the right way. And we invest heavily upfront in that part of our solution, just making sure that we have workflows and best practices aligned with our larger customers. So for, for example...
Maggie: Oh yeah, I was just going to ask you, for, if someone were to say, okay, all right, I'm convinced. Analytics, I've got to do it, or do better. How do you tell them to start?
Sandhya Hedge: Yeah.
Maggie: How do you approach that problem?
Sandhya Hedge: Yeah, that's a great question. And so, in short I think the way we approach it is, we run our workshops around best practices when it comes to the process of product decision- making. Starting from simple things like goal setting, whether you have the right north star metric for your entire product organization and what the key of those metrics look like, right? What's each branch is, are you sure everything you're investing in has some clear quantitative metric at the end that can be measured? Because if not, you're not going to know whether you made the right investment or not. And how do all the smaller projects add up to an overall goal that you're trying to drive? So for instance, even with fairly large and well- organized companies, we often find that they have some overall high level goal that has come down from corporate. And then they have some four or five projects that they've decided a long time ago they are going to do. And there's no connective tissue between those two. It's not clear what was the hypothesis behind saying we are going to build out this particular notifications feature. Say, your overall goal, which is to increase month one retention by 20%. What was the hypothesis behind why one would actually drive the other? So being very clear about what are the goals? What are the hypotheses behind each goals? And then breaking it down into what are the leading indicators that your hypothesis is turning out to be true. And that's where your measurement comes in. If you measure the leading indicators of future success, you can keep your eye on whether everything is moving in the right direction or not.
Sandhya Hedge: So we are working a lot on helping people understand that so that they are measuring the right things and we can help them automatically keep an eye on, whether their leading indicators are actually strong or at risk.
Maggie: How do you help someone identify what a leading indicator even is? Because this makes a ton of sense to me. You've got your north star metric, you have your individual product metrics, you should probably connect them through your hypothesis. All sounds good. But then how do I know actually, that something is a leading indicator or not?
Sandhya Hedge: Yeah, that's, that's a great question. So I can walk you through how we think about that for Amplitude, right? So the way we think about Amplitude is very much, we typically land really small, right? We land with a small product team that really needs this and wants full visibility into customer data and behavior. And then we ended up expanding pretty rapidly into accounts as other teams, and other product units see this initial success that one team has had and start coming to us saying, okay, how can we do the same thing? How can we get our data into Amplitude? Which means that even if that process takes say, 12 months from our revenue perspective, the leading indicator for us is how much adoption of our critical workflows are we getting early in a team? So one of the leading indicators for us is just number of users per account, and how many accounts do we have where the number of users has crossed a certain threshold where we know that, that means this is really taking off in the company. And what we have found is, we did a pretty big one time analysis that gave us a very clear indication that this early adoption metric was heavily correlated to future growth and revenue. And if the product team just used revenue a new as a metric, we wouldn't be able to make any long- term investments, right? Because it's a lagging metric that only is realized after maybe 12 to 18 months after you have made those high quality investments. So it was super important for us to find the highly co- related leading metric. So what we advise companies to do is really use a lot of logic and data science to come up with that model for their business, which is, if we invest in X today, what drives revenue 12 months from now? And make sure that they have sufficient data. They should have about couple years of data to say, okay, this is all very clear that these are the leading indicators. And that's the point when you can just double down on driving those leading indicators and the rest just takes care of itself.
Maggie: Okay. That's amazing. I'm just thinking about the different metrics that we're using and how we can better tie those two things together. So before we run out of time, I wanted to ask you just a couple more questions. I think now that you've been on both sides of products and marketing, what do you think the role is of the product and marketing teams and how we can work better together? And then another question would be, if you were to go back into products, what would you bring with you that you've learned now that you're more in a marketing role.
Sandhya Hedge: Yeah. That's a great question. And I think at the stage that Amplitude is in right now as a company, every six months the kind of customers we are working with changes a lot, right? So we launched Amplitude as a platform just four years ago. And we started working with a pretty small tech startup, social gaming companies. And now we work with the biggest companies in the world in media, and retail, and e- commerce, and a lot of the largest consumer tech companies. So every six months, the kind of person we work with changes a little. So the way that I think I can best represent what is the difference between product and marketing philosophically right now at Amplitude is that product is like the voice of the end customer. They own the problem set that the current customers are trying to solve and marketing is the voice of the prospect. We are very focused on what does tomorrow's customer look like? What are the pain points? How are they different? And what is it that we need to do to help them get better at their jobs? What do we need to do to create superpowers for them? And while it's a small nuance, it's a really important nuance if you have no ownership of the former or the latter, you are in trouble as a company.
Sandhya Hedge: And to your second question of what I would bring back to the product team, I think one of the biggest realizations I have had is the amount of noise when it comes to communication in the market. And just the amount of noise that anyone who largely works online the whole day has to deal with. And one of the things that our product can do a much better job of is to assume less, right? So not assume that someone who has landed in a particular spot has all the context that you need them to have, but that they might need to be able to easily educate themselves on exactly how did I land up here and what the hell am I supposed to do next? Just being much more thoughtful about what that customer experience needs to be for novice users.
Maggie: Yeah. I feel that pain all the time, especially because I think it's getting easier, and easier to ship faster. And, at Drift, we ship all the time and it's something we're focused on constantly, but then it's sort of like... How are we making sure that our customers are coming along with us and how do we make sure that we're not overwhelming them and we're cutting through the noise and making something that's simple and easy for them to use, and not just throwing more products at their face, assuming that they're going to figure it out.
Sandhya Hedge: Yeah, absolutely. I think if you look at the degree of product complexity between what we used to use 10 years ago and what we use now, it's just a crazy order of magnitude higher. The way talk about it is, we say, look, almost all product managers think about your metrics, and their product as a funnel. You go from step one, to step two, step three. But today, what your products have become is more like a maze for your customer. And your customer is trying to figure out, if I go down this path, will I get to the thing I'm trying to get to? And often they are just in dead ends and they don't even know exactly what their destination is in your product. So you have to start thinking about it more as a maze and say, okay, what are the breadcrumbs I can add here so that they know where to go? As opposed to, it's a funnel. And obviously the next thing you do is click X.
Maggie: Right. Exactly. Okay. So if you had to give one to three pieces of advice to the people who are listening. Who want to be more data- driven or data informed, and who want to come along on this shift, but may or may not have the control to implement a new analytics offered in the snap of their fingers, what advice would you give to people who want to operate in this way?
Sandhya Hedge: It's a great question. I think the first big piece of advice is just having alignment as a team on what your goals and hypotheses are. Especially if you have a leadership team that seems unwilling to invest in data, being able to clarify that will definitely help you drive the conversation for why you want to invest in analytics platforms, because you can't really be goal- oriented without having good measurement, right? You only really drive what you can measure well. That would be the biggest thing, is make sure that you can have a conversation with your leadership team and ask them, okay, what exactly are our goals? So if you're doing an app redesign, how do we know the app redesign is successful at the end of this app redesign. Really have any goals for everything. Number two would be not sticking with demographics, but really moving to behaviors, and talking about customer behavior, and using behavior as the foundation of your product roadmap and your analytics. Demographics is just... That what people used to rely on because they did not have any behavioral data. So I still see a lot of people leaning heavily on surveys, and demographic data, and stood off just using actual behavior. You don't need to run a survey to understand if customers like some feature, you can just see whether they come back to it. And you can be completely fact oriented.
Sandhya Hedge: That would be number two. And number three would just be around, make sure that you are investing in experimentation and iteration culture, because without that, it's impossible to survive today's fast- paced age of product where over a weekend your industry could completely change because someone had an amazing launch of a new platform.
Sandhya Hedge: You have to have a very agile approach to decision- making, and you have to be able to iterate and experiment really fast. Yep. Awesome. Well, thank you so much, Sandhya. This was incredible. I really appreciate you coming on the podcast and giving us all the secrets on how to be more data- driven.
Maggie: Awesome. Thank you so much, Maggie. It was a real joy.
Sandhya Hedge: So you heard it here. You cannot be goal- oriented without measurement. Start to look at behaviors, and not just demographics. And you have to invest in experimentation, iteration, or else you're going to fall behind. So shout out Sandhya and reviews. Six stars only, obviously. And let me know what you think at maggie @ drift. com or @ maggieprobably on Twitter. Thanks.