Even the best customer discovery tools require great customer discovery. Here's how we do it.
The Pragmatic Application of Artificial Intelligence
Even at the time of writing this post - Artificial Intelligence is a tool best applied making humans better and/or ...
Even at the time of writing this post - Artificial Intelligence is a tool best applied making humans better and/or faster at their jobs. There are many areas of human performance where employing AI isn’t yet worth the effort. We’re just not there yet - but as the technology improves, we’re getting closer, quickly.
How we apply AI to our work at BMNT is continually evolving – we started testing its use in a small way and are currently working to develop a method for deriving significantly more detailed insights that once tested, will be able to describe precisely how an organization can make the greatest impact on a specific community by solving their specific problems.
In our day-to-day work, we regularly interact with others, in meetings, interviews and conversations in a synchronous way, and via text, social media, email and various collaboration tools asynchronously. We’ve seen AI-powered techniques creep into both synchronous and asynchronous communication streams for years now (decades, in some cases). While it’s been a gradual transition, it’s had a big impact, significantly changing the way we communicate with each other.
As practitioners of Design Thinking and Lean Startup methodologies, our team does a ton of customer discovery - getting out of the building and talking to people. These valuable conversations with end users are the single most important thing we do to help our customers understand their problems more clearly.We won’t ever stop doing it - but we’ve been asking ourselves, “What if we could scale this beyond the capabilities of our human team? Is that a good idea? Would we sacrifice some degree of fidelity in exchange for scale?”
Given the pervasiveness of AI in our conversation streams, we knew this was possible in a proof-of-concept sense, but it was unclear if the results of our experiments would pass muster. What was clear was that any application of AI into such a vital part of our business would need to be done with great care.
We interact with people in very specific ways to help our customers make decisions. We expect that any technology we apply to interactions adheres to the standards we set for ourselves. Even with all the recent advances in Natural Language Processing (NLP) and Natural Language Understanding (NLU), we know that AI currently can’t meet our user experience standards in terms of holding meaningful conversations and directing those conversations in ways that uncover novel insights.
We decided to begin testing our concept by using a highly scripted conversational UI that could take the work out of important, yet time-consuming, introductory conversations. Our hypothesis at the time was that we could leverage a conversational AI product front-end to capture this information and that the user would be open to doing so. We then used this UI to gather data about the problems people face in their day-to-day work. To our surprise, users loved it. In fact, two-thirds of people who engaged with the technology ended conversations saying things like, “It was nice chatting with you, have a great weekend!”
As we iterated on this approach to improve the resolution of data we could gather, we continued to learn how to fine-tune the user experience and quickly identified where to further apply AI to augment our human capabilities.
Even as we integrate NLP, we still control the narrative – we author every comment or question our technology will ask. We set the tone, and guide the flow of the conversation and control that narrative tightly to ensure that the conversations are similar to the ones we have every day with customers. . However, we’ve now started to allow the UI to make choices about which things to ask about or follow-up on for clarification based on what it receives from the user. It’s not sentient, or really anywhere close, but it’s a starting point.
Anyone who works with a large dataset knows how far Machine Learning (ML) technology has progressed and understands the massive impact it’s had on data analysis and understanding. Data analysis is where AI really shines - we’ve found it far more effective than people at analyzing the large sums of data we collect, and we’ve invested heavily in this capability. We leverage an AI-powered analysis platform to perform several critical functions that surpass what’s possible with human analysis:
- Interpreting everything exchanged in natural human conversations, and cross-referencing with all previous conversations
- Identifying unknown unknowns - the connections an analyst couldn’t even know to ask about when gathering data
- Understanding the data its interpreting without training or context
- Outputting groups of similar problems, or problem topics, that have high semantic similarity and high cluster coherency.
- Topic breakdown that includes a description of the group, the number of problems related to that topic, and the list of people who identified the problems.
The way we’ve employed AI allowed us to continue to talk to and learn from conversations with people -- only we can now do it at a scale that would take years to replicate with human-to-human interaction. We haven’t reached truly game-changing levels of machine intelligence just yet, but our approach is careful and pragmatic and as long as we maintain focus on ensuring the technology aims to replicate our best work product, we’re confident that we’ll find what we’re looking for.