Winning with AI

Muzaffar Borker, Managing Director, Software Engineering, has a semi-annual hackathon. The engineering teams and the product managers team up to come up with impactful ideas that will move the JPMorgan Asset Management business forward through solving some user pain point, or coming up with something radically new. In 2023, a simple, bare bones, natural language processing prototype used to retrieve funds using plain English through a search interaction paradigm wins the hackathon.

It also dovetailed nicely with the shift occurring in interaction design for Information retrieval, which moves from UI filters to natural language enabled by the rise of LLMs (Large Language Models).

I began working at JPMAM (JPMorgan Asset Management) in 2023. I heard about the hackathon shortly after I started working at the office through conversations with my managing director, Matt Lesle. I raised my hand, so to speak, because I wanted to work on the rapidly accelerating AI and LLM space shortly after the launch of GPT-3.5 and after leaving The Wall Street Journal, where I had worked on a machine learning project using knowledge graphs.

With JPMorgan’s resources, I believed I could make a huge impact by transforming the UX of its products with a new UX paradigm shift enabled by conversational interfaces.

Project Team and Roles

Edith Chu - Product Manager

Matt Lesle - Managing Director, Global Head of Digital Product

Corey Hill - Executive Director, Portfolio Insights

Obinna Izeogu - VP, UX

Goods & Services - Design Agency

Jason Brown - Compliance Director

Jung Kang - UX Researcher

User Zoom - UX Research Agency’

Gabe Tan - Executive Director, Business Intelligence (Data Science)

AMOC - C-Suite Executives operating committee

Muzaffar Borker - Managing Director, Software Engineering

"Smart Search" fund comparison by way of Copilot

I initially began work on a Copilot, but the AMOC wanted to launch the winning hackathon called “Smart Search” immediately. Morgan Copilot, which was a virtual assistant, would require a more complex approval process due to LLM hallucinations, so an NLP solution that was fact-driven and queried a relational database with SQL was a safer bet and could have an immediate impact by showing value through flows—a leading KPI for the marketing department—however, the task of taking a bare bones idea to an actual product needs product thinking.

Research produces observations

Before I began working at JPMorgan, there had been some early discovery work on the investment comparison tool with advisors who seemed content to look for funds by ticker symbol or by investment name with some common attributes like investment type, Morningstar rating, asset manager, and expense ratio.

Searching by ticker symbol or investment name was table stakes and didn’t create a differentiated experience from other investment comparison tools, and thus did not differentiate JPMorgan Asset Management from other fund managers.

A simple ticker symbol search with a few data points returned.

Personas

I took the initiative to look through personas that had been created by User Zoom, the user research agency, a few years before I started working at JPMAM. It gave me the tools to understand the needs of financial advisors. I look for clues from research data that can inform a perspective and lead to an observation that can address not only user needs but also nurture the opportunity for a business to differentiate itself from its competitors.

The white space created by a hypothesis

I started with the understanding that there was white space with LLMs. So I thought of ways to increase value by taking clues from the day-to-day of a financial advisor and the precedents being established from conversational interfaces. This would be the key to a solution hypothesis:  how to not only create a differentiated experience for financial advisors by addressing their mental model of how they do fund research through queries in their head and translating those queries into a clunky UI filtering experience on Morningstar. But what if we could use natural language processing to generate SQL to query the Morningstar relational database and tie it to conversion for the business?

Searching with a phrase and querying a relational database through NLP (natural language processing.

UX Orchestration

I introduced some key elements to orchestrate the experience: bifurcation, system transparency, personalization, dynamic columns, and high-value actions (fact sheet download, viewing fund commentary, and market outlook summaries), all presented as local actions for every fund result.

UX flow that leads to value that goes beyond just performance data to providing portfolio insights, tying together the natural language search, data, and context, going beyond just ticker symbol and investment name, and initiating a richer user experience.

The user's mental model is to think in terms of outcome not just ticker symbols in a black box. In this natural language search, the use of dynamic columns to reveal a high sharpe ratio using smart search which is a bifurcated experience keeping it separate from those that want the old experience. Transparency with what criteria was actually computed by the smart search and the ability to "See all supported criteria" enhancing their context for discovery.

Cross-functional teams get work done

There was the belief by the product manager, Edith Chu, that financial advisors didn’t need natural language search. I created prototypes and presented them to Corey Hill, the head of portfolio insights. He loved it. I later presented it during the weekly investment tools meeting, which included a cross-functional team of engineers, product management, and occasionally other key stakeholders like compliance.

All projects that wish to make an impact need to receive buy-in from all parts of the organization. The cross-functional team agreed to do testing on my concepts. Data-driven design is important in generating enthusiasm and buy-in, especially if it comes from clients or prospective clients.

Journeys, sketches, and prototypes

During the process of exploring what the prototypes could be, how to extend them and touch more areas and connect my work with those on other teams, I met with Gabe Tan, the Executive Director of Business Intelligence (Data Science), to better understand the journeys of financial advisors.

Journeys essentially lead to advocacy and user satisfaction or churn; understanding the sales process and what the business wants to achieve enables a more robust solution to get buy-in by the business. Understanding how data flows is key because it leads to attribution.

During my design sketching and wireframing phase, I met with Jason, the Compliance Director, to de-risk some of my value-added features around generative AI that did not return a single number but used language that could be non-compliant due to LLMs notrious habit of hallucinating and providing incorrect information with gumption and certitude. A firm like JPMorgan cannot have client-facing experiences that give incorrect information. In fact, that is a huge risk that needed to be addressed if those other features like market outlook were going to be part of the MVP.

Aha! moments during testing lead to the first JPMAM AI product shipped to clients.

The team ran 3 rounds of concept testing, and the concepts were rated highly, with the final iteration standing out from the rest after previous iterations of trial and error. There were several “aha!” Moments, because the core experience of using plain English to discover funds to compare against each other fits with the mental model of financial advisors. 

The personas created by User Zoom enabled the white space for features and design to come into view which allowed us to test those mental models to see if they resonated with financial advisors.

After the testing, the product manager engaged Goods & Services, the visual design agency,  to provide the visual design based on my wireframes, high-fidelity prototypes, and design.

JPMorgan Asset Management has a contract with a design agency, Goods & Services, so all final design deliverables are outsourced for production. I provided guidance and feedback during the visual design process, and I led the design effort from observation from user research, to problem framing, hypothesis generation, prototyping, testing with an experiment, analysis of the data, and reporting the results from the research back to all stakeholders. I provided the design leadership from concept creation to shipping the first client-facing AI-enabled experience for JPMAM.