“TL;DR quote: > The bottom line is that the difficulty of the decision is the most important factor in determining whether a player makes a mistake. In other words, examining the complexity of the board position is a much better predictor of whether a player is likely to blunder than his or her skill level or the amount of time left in the game. That fir…”
Assessing Human Error Against a Benchmark of Perfection
An increasing number of domains are providing us with detailed trace data on human decisions in settings where we can evaluate the quality of these decisions via an algorithm. Motivated by this development, an emerging line of work has begun to consider whether we can characterize and predict the kinds of decisions where people are likely to make errors. To investigate what a general framework for human error prediction might look like, we focus on a model system with a rich history in the behavioral sciences: the decisions made by chess players as they select moves in a game. We carry out our analysis at a large scale, employing datasets with several million recorded games, and using chess tablebases to acquire a form of ground truth for a subset of chess positions that have been completely solved by computers but remain challenging even for the best players in the world. We organize our analysis around three categories of features that we argue are present in most settings where the analysis of human error is applicable: the skill of the decision-maker, the time available to make the decision, and the inherent difficulty of the decision. We identify rich structure in all three of these categories of features, and find strong evidence that in our domain, features describing the inherent difficulty of an instance are significantly more powerful than features based on skill or time.
候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“It's poorly written press coverage of https://arxiv.org/abs/1606.04956 tl;dr We find strong evidence that in our domain, features describing the inherent difficulty of an instance are significantly more powerful than features based on skill or time.”