按年被提到
20122012 · 1 次2026
系统、数据库与网络›数据库与向量检索·arXiv:1210.7350 · cs.IR

Fast Data in the Era of Big Data: Twitter's Real-Time Related Query Suggestion Architecture

Gilad Mishne, Jeff Dalton, Zhenghua Li, Aneesh Sharma, Jimmy Lin · 2012
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2012-10-30 · 最近 2015-02-06
2不同的人
2活跃月份
2讨论串
14被引(OpenAlex)
arXiv 摘要(候选层不写导读)

We present the architecture behind Twitter's real-time related query suggestion and spelling correction service. Although these tasks have received much attention in the web search literature, the Twitter context introduces a real-time "twist": after significant breaking news events, we aim to provide relevant results within minutes. This paper provides a case study illustrating the challenges of real-time data processing in the era of "big data". We tell the story of how our system was built twice: our first implementation was built on a typical Hadoop-based analytics stack, but was later replaced because it did not meet the latency requirements necessary to generate meaningful real-time results. The second implementation, which is the system deployed in production, is a custom in-memory processing engine specifically designed for the task. This experience taught us that the current typical usage of Hadoop as a "big data" platform, while great for experimentation, is not well suited to low-latency processing, and points the way to future work on data analytics platforms that can handle "big" as well as "fast" data.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

没有评论直接链到它;下面是它被提交的讨论串里回复最多的评论(不计入统计)
讨论串评论bslatkin · 2012-10

“Why didn't they just run Hadoop on a sample of the logs instead of the full dataset? It seems like they were just looking for popular related queries anyways. Any ideas?”

[pdf] Twitter’s Real-Time Related Query Suggestion Architecture原评论 ↗
被提交为 story · 2 次