Seller Inventory Recommendations Enhanced by Expert Knowledge Graph with Large Language Model

Recommender systems are extensively utilized within the e-commerce sector to assist consumers in discovering complementary products aligned with their interests or to discover unique items that may spark a new purchasing journey. However, the application of recommender systems for inventory providers (vendors or sellers) remains an area yet to be fully explored. At present, major e-commerce entities such as Amazon, Etsy, or Walmart do not employ recommender systems designed to aid vendors in augmenting their inventory in response to current market demand within their specific niche.

Within the marketplace economic framework, sellers or inventory providers frequently encounter a scarcity of market trend data, often due to limited resources. Conversely, marketplace platforms amass extensive data from buyers and possess the resources to assess and forecast inventory demands and trends. This disparity in data access is known as information asymmetry, where one party holds more information than the other. This imbalance can result in sellers being reluctant to expand their inventory, while marketplace facilitators attempt to bridge this gap by equipping sellers with relevant data. A notable example of resolving such asymmetry occurred in 2020 when Etsy addressed the surge in demand for masks, prompted by the onset of the pandemic, which far exceeded the available supply. Etsy mitigated the information gap by informing their sellers, enabling them to not only contribute to societal needs but also to reorient their inventory in light of the lockdown and the initial shift in purchasing behavior. By providing sellers with relevant information, the quality and relevance of the inventory listed can be enhanced, and the introduction of new inventory can attract additional buyers to the platform.

C2C sellers and smaller B2C sellers on eBay often lack market information due to limited resources for market research. This paper proposes an item recommender system for sellers that could potentially address market inefficiencies stemming from the insufficiency of market information available to sellers. The recommender system is designed to suggest specific items for listing that could expand a seller’s inventory, taking into account their preferences, buyer demand, and economic projections. The proposed solution integrates various methodologies to deliver optimal item recommendations: an expert knowledge graph at the leaf category level for complementary suggestions, an expert knowledge graph at the aspect level for in-category and subject-related recommendations, and a buyer behavior graph reflecting purchasing behaviors and patterns.

An expert knowledge graph is a new term introduced in this paper and is capable of discerning real-world connections in a specific niche category which generic knowledge graphs are not able to identify. Ebay, as a marketplace, is viewed by buyers seeking rare and niche-related items. An expert knowledge graph is therefore an essential tool for identifying and understanding connections based on category preferences. Behavioral patterns and preferences vary significantly between communities, such as those collecting sports trading cards and those indulging in fashion. This paper specifically concentrates on the sports trading card category as a representative sample for other niche communities.

The primary aim of this paper is to address market inefficiency arising from information asymmetry between sellers and marketplace providers by introducing a scalable seller recommender system that utilizes an expert knowledge graph. The resolution of such inefficiency offers numerous advantages: quality improvement of listings, facilitation of inventory expansion and management for sellers, an increase in the number of new buyers to the marketplace, and a boost in Gross Merchandise Bought (GMB) and purchase frequency by existing buyers.

2. Background

Seller recommender systems have not been extensively developed due to the nature of its complexity, with scalability and relevance being primary concerns. The advent of Large Language Models has facilitated approaches to these issues, particularly by employing LLMs to create a common-sense world model. Such a model can address a limitation of eBay category Taxonomy.

Ebay’s inventory is organized into a predefined category hierarchy, which forms the basis of all recommendation systems. This structure imposes limitations, especially when items from different categories share a common field of interest. For instance, a Michael Jordan sports trading card and a signed Chicago Bulls t-shirt belong to different categories but share an interest in the Chicago Bulls. By leveraging a common-sense model, connections that would otherwise remain unidentified can be established.

eBay is known for its unique inventory, but not all inventory is listed or exists on eBay. Leveraging LLMs and an expert knowledge graph enables the expansion of recommender systems beyond internal data, a pace not possible without external data sources until recently.

Traditional recommendation systems utilize historical internal data of user browsing and purchasing activities to predict cohorts with similar interests. An expert knowledge graph can provide an additional layer of connections that extends beyond historical data, identifying trends previously undiscovered due to data limitations. By overcoming these challenges, a high-quality recommender system for sellers can be provided.

Utilizing an expert knowledge graph allows for differentiation in listing recommendations based on the nature of the inventory. This differentiation facilitates multiple opportunities for inventory extension while maintaining high seller satisfaction scores. Listing inventory recommendations can be categorized as follows:

• Complementary Inventory: Recommend products that buyers frequently search for or purchase alongside a seller’s inventory but are not currently offered by the seller. For example, a buyer purchasing sports trading cards may also be interested in protective covers or display cases, which could be recommended as complementary items to enhance the buyer’s purchase experience.
• In-Category Inventory: Recommend products that are in high demand within the same category as the seller’s main inventory. Leveraging an expert knowledge model can identify connections, such as recommending a Michael Jordan baseball card to collectors of basketball trading cards, thereby expanding the seller’s inventory beyond their typical offerings.
• Subject-Related Inventory: Recommend products that are part of the same subject-related inventory. For instance, a seller of Michael Jordan sports trading cards could extend their inventory to include Michael Jordan-related memorabilia, Chicago Bulls-related items, or basketball-related products.
• Out-of-Category Inventory: An expert knowledge graph can introduce guardrails to ensure recommendations are of high quality. For example, a seller specializing in sports trading cards would not be recommended to list car-related accessories. By differentiating types of inventory, item listing recommendations can be tailored based on seller preferences, offering multiple options that cover a range of possibilities. Complementary item recommendations represent vertical inventory expansion, in-category and subject-related recommendations signify horizontal inventory expansion, and out-of-category recommendations act as cross-industry expansion guardrails due to the marketplace’s nature.

3. Methodology

3.1 Model Overview

Knowledge graphs represent an effective approach frequently employed in recommendation systems. Typically constructed based on user-item interactions such as purchase behaviors or click and view actions, these knowledge graphs enhance our understanding of users and items through incorporating semantic similarities and associations between them. However, they are susceptible to sparsity issues and the cold-start problem due to limited explicit user-item interactions. Another approach involves path-based recommendations, which utilize auxiliary meta-paths. These meta-paths represent pre-defined higher-order relational compositions between different types of entities in knowledge graphs, serving as contextual information between users and items. However, this type of knowledge graph heavily relies on the accuracy of entity relationships, such as category taxonomy and item aspect associations. Capturing complementary and substitute relations between item categories also poses a significant challenge.

With the advent of Large Language Models (LLMs), we now possess the capability to delve deeper into the intrinsic associations within items. This unlocks deeper contextual information underlying people’s purchase behaviors. By leveraging LLMs, we are able to discern the underlying factors contributing to the popularity of specific items in a seller’s inventory and establish correlations with other related items to provide expert recommendations for expanding the inventory for a seller. Take the category sport trading cards as an example, if the 1988 Fleer Michael Jordan card emerges as the best seller in a seller’s inventory, LLM can intuit that customers may also be intrigued by other cards showcasing the US Dream Team or commemorating Jordan’s dominance on the international stage. We can then capture this information as context in our recommendation. Without this expert knowledge, it is hard for a traditional recommendation system to capture the player’s career milestones and effectively incorporate them into the recommendation output.

As illustrated in the Figure 1, to leverage LLM expert knowledge, we created two types of knowledge graphs based on the seller’s inventory: leaf category knowledge graph and the item aspect knowledge graph. The leaf category knowledge graph is designed to unearth complementary relationships, while the item aspect knowledge graph is aimed at unveiling alternative items of buyer interest. Aligned with the user-item graph derived from the user purchase behaviors, we have three distinct types of graphs to augment the contextual understanding between the user and item relations. Subsequently, we sampled meta-paths within each knowledge graph and employed path encoding techniques to learn the vector representations of meta-path-based context. Attention mechanism has been integrated into the model to enhance the understanding of how sellers can engage with items across different contexts.

3.2 LLM expert knowledge graph construction

We utilized LLM to construct two expert knowledge graphs: one representing the in-category item aspect relations and the other capturing the complementary relationships within leaf categories.

In-category item aspect graph

To extract substitutive relationships among items, we harnessed LLM to produce item relations. The expert knowledge generated by LLM is often unstructured. To represent this output in a graph, it must be converted into a structured data format. The construction process can be viewed as the transformation of expert knowledge into structured data. Initially, we identify the entities within the graph. While items can be treated as entities, the diversity of item information poses challenges. Analyzing the relationships among items can be computationally intensive, especially when dealing with billions of items in the eBay inventory. Alternatively, we can aggregate item commonalities based on their aspects to alleviate these complexities.

Given a seller’s inventory, we initially identify the top-selling items and extract their associated item aspects types, along with the corresponding aspect values. The item aspect expert knowledge graph is then constructed using chain-of-thought approach, as illustrated in the Figure 2. In the first step, we construct prompt to utilize LLM to pick top aspects that can influence users purchase decisions:

"When people purchase in the {Sport Trading Cards}, what are the top {2} key aspects people will consider?

Pick from the list {Player, Season, Team, Set, Rookie, Grade}.

Output format: One aspect one line, without any explanations."

For instance, in the category Sport Trading Cards, LLM will prioritize aspects such as player, set, and seasons as central factors in their decision-making process.

Secondly, for each selected aspect ai , we construct a prompt to retrieve top K most relevant aspect values in each aspect dimension.

“Provide recommendations for the top 3 associated {set} buyer will be interested if they show purchase intent in {1986-87 Fleer Basketball}.

Output format: One {set} one line, without any explanations."

In line with the Sport Trading Cards illustration, should the best-selling cards originate from the Set: 1986-87 Fleer Basketball, LLM will infer that customers of this store are also likely to be interested in other significant milestones of Michael Jordan, like the Set: 1984-85 Star Company Basketball, thus this edge between two sets are generated.

Based on the output of the LLM, we can get the pseudo of the entities and their edges in the item aspect knowledge graph. These pseudo entities will then be mapped with the in-house item aspect data using the text matching techniques based on text embeddings. As a result, the item aspect knowledge graph for this category can be constructed and the inner relation within the aspect values are decoded in the graph.

Complementary leaf category graph

Besides the substitute relations inside one category, LLM is also capable of understanding the cross-category complementary relationship, as shown in figure 3. Similar to the process of constructing the item aspect graph, we construct the prompt to get the top M category recommendation.

“Recommend top {3} complementary categories people will be interested in if they purchase in the category {sport trading cards}.

Output format: One {category} one line, without any explanations."

In the example of Sport Trading Cards, LLM discerns that buyers interested in purchasing cards may also express interest in acquiring card protectors or other sports card memorabilia. It's important to recognize that unlike the item aspect graph, the leaf category relationship is unidirectional, signifying that category B may not necessarily complement category A if A complements B. Consequently, we map the LLM output with the eBay leaf categories as entities to construct the leaf category graph.

3.3 Encoding for meta-path based expert knowledge context

In the following sections, we will demonstrate how to incorporate expert knowledge context in the seller inventory
recommendations shown in Figure 4.

We firstly learn the vector representations for buyers and items by using DeepWalk to extract information from the user-item bipartite graph. This graph is constructed using customers' co-purchase sequences. With DeepWalk, it is able to optimize the co-occurrence probability among items. Then based on the item representations, we are able to obtain the seller representations by using the Max-Pooling of representative items in his inventory. Then we sampled and get a few meta-path instances for each leaf-category pair and aspect pair to participate in recommendation model training.

To incorporate the meta-path based context, we learn the vector representation of the path instances. Specifically, the information underlying the aspect, item, leaf category is conveyed through text, i.e. the aspect value, item title and the corresponding leaf category name. To measure the text similarity, we utilized pre-train eBert embeddings to encode the entity including leaf category, item and aspect along the path instance.

3.4 Incorporating Expert Knowledge Graph in the seller recommender system

Self Attention for meta-path based context embedding

After getting the embedding of the paths instances sampled from the LLM based knowledge graph and purchase behaviors based user-item graphs, we obtained the meta-based context embedding through the self attention mechanism. The self attention mechanism allows the model to weigh the importance of different path instances in the same context against each other, capturing dependencies between them. Compared to a simple pooling method to aggregate the path embedding into context embedding, self attention mechanism is better at capturing complex information as the context.

For example, a customer buy a 1992 Upper Deck Magic Johnson card along with 1992 Skybox USA Basketball #545 Charles Barkley may because of multiple reasons: 1) Two cards are from different players, but both featuring Dream Team during the 1992 Olympics in Barcelona. 2) These two cards both demonstrate the variety of parallels available to collectors. With self attention, this information can be well captured and mixed with importance in the context.

In detail, the meta-path based context embedding can be obtained through:

Where Query Q, key K and value V are all from path embedding associated with path φ, and W is the weight. dk is the dimensionality. Concat(.) is the concatenation operation.

Context interaction via cross-attention mechanism

Given the context embedding including leaf category meta-path, user-item meta-path and the item aspect meta-path, we are able to incorporate this context embedding with the seller embedding hs and the item embedding hithrough a cross-attention mechanism. With cross-attention, it allows the model to attend to relevant meta-path content while updating the seller and item embedding. Similarly to the self-attention mentioned above, the cross-attention can also be obtained with this equation. The query Q corresponds to the seller embedding or the item embedding, contingent upon whether it pertains to seller-context or item-context cross attention, respectively. Key K, value V are the path context embedding obtained from self-attention. We annotated the updated seller embeddings as ̃hs and updated item embeddings as ̃h

3.5 The complete model

Ultimately, based on the item inventory of a seller, we applied graphs generated by LLM and encoded this expert knowledge as context embeddings to update seller and item representation. We concatenated them into a unified representation of the interaction h_{s,i}. We then applied an MLP layer on this vector representation to get the final score between the seller S and the item I. The MLP contains a two-hidden-layer neural network with ReLU as the activation function and sigmoid function as the output layer.

During the training phase, we learned the parameters of the model using negative sampling. The loss function is calculated as:

4. Conclusion

The implementation of the proposed seller recommender system has the potential to significantly reduce market inefficiencies caused by information asymmetry between sellers and marketplace providers. By utilizing LLMs and expert knowledge graphs, the system offers personalized and relevant inventory expansion recommendations that cater to the specific needs and preferences of sellers. The differentiation in listing recommendations allows for vertical, horizontal, and cross-industry inventory expansion, with guardrails in place to maintain high-quality suggestions. The system’s scalability and adaptability make it a valuable tool for a wide range of seller segments, particularly those with limited resources for market research. Ultimately, the recommender system not only benefits sellers by enhancing their inventory management and expansion but also contributes to the overall marketplace ecosystem by attracting new buyers and increasing the frequency of purchases.