按年被提到
20172017 · 2 次2026
社会、经济与科学研究›科研评价与算法公平·arXiv:1702.00502 · cs.DL

Single versus Double Blind Reviewing at WSDM 2017

Andrew Tomkins, Min Zhang, William D. Heavlin · 2017
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2017-02-05 · 最近 2017-12-01
2不同的人
2活跃月份
2讨论串
10被引(OpenAlex)
arXiv 摘要(候选层不写导读)

In this paper we study the implications for conference program committees of using single-blind reviewing, in which committee members are aware of the names and affiliations of paper authors, versus double-blind reviewing, in which this information is not visible to committee members. WSDM 2017, the 10th ACM International ACM Conference on Web Search and Data Mining, performed a controlled experiment in which each paper was reviewed by four committee members. Two of these four reviewers were chosen from a pool of committee members who had access to author information; the other two were chosen from a disjoint pool who did not have access to this information. This information asymmetry persisted through the process of bidding for papers, reviewing papers, and entering scores. Reviewers in the single-blind condition typically bid for 22% fewer papers, and preferentially bid for papers from top institutions. Once papers were allocated to reviewers, single-blind reviewers were significantly more likely than their double-blind counterparts to recommend for acceptance papers from famous authors and top institutions. The estimated odds multipliers are 1.63 for famous authors and 1.58 and 2.10 for top universities and companies respectively, so the result is tangible. For female authors, the associated odds multiplier of 0.78 is not statistically significant in our study. However, a meta-analysis places this value in line with that of other experiments, and in the context of this larger aggregate the gender effect is also statistically significant.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论bo1024 · 2017-12

“This is very cool, but I'm having trouble following the method behind the results. Say concretely, the result that single-blind reviewers give higher reviews to papers from prestigious sources. Can anyone help me understand? https://arxiv.org/pdf/1702.00502.pdf They give each paper a "blind" score bqps from the blind reviewers only. Then they fit the ve…”

Understanding Bias in Peer Review原评论 ↗
被提交为 story · 1 次