Conversation design for LLMs
SeekOut Recruit: B2B SaaS platform used by 6 of the 10 most valuable companies in the US. Impact: Increased feature engagement by 30% with conversation design.
Improving candidate matching with natural language search
Conversational Search allows recruiters to search for candidates using natural language queries. I introduced conversational design strategies to boost feature engagement by creating natural, user-centered conversations that made the search process easier and saved time.
My role: Sole content designer, user interviews & testing, cross-functional collaboration.
Product context
SeekOut Recruit works like a search engine for recruiters. It uses AI to help users find and connect with candidates using publicly available information about them from LinkedIn, GitHub, and other online sources.
Challenge: Overcoming user frustration with AI responses
The soft launch of Conversational Search revealed that users were abandoning the search process. They found the feature difficult to follow, were unaware of what it could do, and received unhelpful responses like "sorry, I didn’t understand" when the AI failed to interpret their requests. These issues led to frustration and decreased usage.
Solution: Improve AI responses and guide users
Improve the AI's response clarity and guide users in their searches by using conversational design strategies. Make conversations more intuitive and effective, provide helpful suggestions, and enhance the overall user experience.
How I helped
User feedback analysis: Gathered detailed feedback from users to understand their pain points, such as misunderstood requests and unclear responses.
Collaboration with engineers: Worked with engineers to test and refine the search model, improving accuracy and relevance.
Intent discovery and utterances: Defined user intents and created a list of potential user utterances to train the search model for diverse inputs.
Testing and iteration: Analyzed user interactions, made adjustments based on feedback, and measured engagement rates before and after changes.
Template responses: Developed AI template responses for various user scenarios to enhance natural and effective interactions.
Understanding user intents and phrasing
I began by identifying and grouping user intents to understand how recruiters phrase their search queries. Next, I created “utterances” to represent these different phrasings. These utterances helped train the AI to understand and respond appropriately to diverse and natural language inputs.
Effective template responses
To improve the AI's interaction quality, I created tailored template responses for various user input scenarios. This ensured that the AI could provide more context-specific and helpful replies, enhancing the overall user experience.
Initial message to guide and inform
Limitations
Uninviting: Doesn't encourage interaction.
Lack of guidance: Users may not know how to refine their search.
Ambiguous: Misses the chance to demonstrate tool capabilities.
Improvements
Engaging: Uses friendly and inviting language, encouraging interaction.
Guiding: Provides specific examples and instructions, making the process intuitive.
Clear: Outlines available options clearly, reducing confusion.
Effective error response to get users back on track
Limitations
Technical language: Including and excluding values isn’t user-friendly language.
Unclear: Emphasizes what users cannot do instead of suggesting helpful alternatives.
Negative focus: Lacks specific guidance. Does not provide concrete examples or alternative solutions.
Improvements
Clear and direct: Clearly explains the limitation in simple language.
Helpful guidance: Offers a practical alternative with specific examples.
Positive language: Focuses on what the user can do, making the interaction more positive and supportive.
Clarify misunderstandings and redirect the conversation
Limitations
Unclear explanation: Doesn't explain why the query wasn't recognized or provide any guidance on how to fix it.
Lack of alternatives: Fails to offer any suggestions or examples for how to rephrase the query.
Improvements
Tailored: Adaptive responses based on type of error.
Clear and direct: Briefly explains that the query wasn't understood and asks for rephrasing.
Helpful examples: Provides specific examples to guide the user on how to structure their query.
Learning and impact
Addressing scalability challenges
As our Conversational Search user base grew, response times slowed. I learned that concise, efficient messaging is crucial for maintaining performance. By streamlining responses and removing unnecessary details, we improved the efficiency and user experience.
Metrics
Baseline: We tracked candidate searches initiated, profiles viewed, and interactions before implementing changes.
Post-Implementation: We monitored the same metrics after improving responses and achieved a 30% increase in user interactions and engagement.