按年被提到
20232023 · 2 次2026
AI 安全、伦理与社会影响›大模型与人类行为·arXiv:2305.06474 · cs.IR

Do LLMs Understand User Preferences? Evaluating LLMs On User Rating Prediction

Wang-Cheng Kang, Jianmo Ni, Nikhil Mehta, Maheswaran Sathiamoorthy, Lichan Hong, Ed Chi, Derek Zhiyuan Cheng · 2023
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2023-05-12 · 最近 2023-06-21
2不同的人
2活跃月份
2讨论串
133被引(OpenAlex)
arXiv 摘要(候选层不写导读)

Large Language Models (LLMs) have demonstrated exceptional capabilities in generalizing to new tasks in a zero-shot or few-shot manner. However, the extent to which LLMs can comprehend user preferences based on their previous behavior remains an emerging and still unclear research question. Traditionally, Collaborative Filtering (CF) has been the most effective method for these tasks, predominantly relying on the extensive volume of rating data. In contrast, LLMs typically demand considerably less data while maintaining an exhaustive world knowledge about each item, such as movies or products. In this paper, we conduct a thorough examination of both CF and LLMs within the classic task of user rating prediction, which involves predicting a user's rating for a candidate item based on their past ratings. We investigate various LLMs in different sizes, ranging from 250M to 540B parameters and evaluate their performance in zero-shot, few-shot, and fine-tuning scenarios. We conduct comprehensive analysis to compare between LLMs and strong CF methods, and find that zero-shot LLMs lag behind traditional recommender models that have the access to user interaction data, indicating the importance of user interaction data. However, through fine-tuning, LLMs achieve comparable or even better performance with only a small fraction of the training data, demonstrating their potential through data efficiency.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

没有评论直接链到它;下面是它被提交的讨论串里回复最多的评论(不计入统计)
讨论串评论yonz · 2023-05

“I've often pondered the idea of semantic recommendation but it is a dynamic problem with an aversion for repetition. I am looking forward to digging into this paper. A big question for me is if you can successfully avoid repeating different coverage while maximizing affinity.”

Do LLMs Understand User Preferences? Evaluating LLMs on User Rating Prediction原评论 ↗
被提交为 story · 2 次