按年被提到
20192022 · 2 次2026
大模型与语言模型›检索增强与知识库·arXiv:1903.10676 · cs.CL

SciBERT: A Pretrained Language Model for Scientific Text

Iz Beltagy, Kyle Lo, Arman Cohan · 2019
开始阅读 ↗PDFalphaXiv 讨论
被提及 · 2 人首次 2022-06-06 · 最近 2022-11-15
2不同的人
2活跃月份
2讨论串
50被引(OpenAlex)
arXiv 摘要(候选层不写导读)

Obtaining large-scale annotated data for NLP tasks in the scientific domain is challenging and expensive. We release SciBERT, a pretrained language model based on BERT (Devlin et al., 2018) to address the lack of high-quality, large-scale labeled scientific data. SciBERT leverages unsupervised pretraining on a large multi-domain corpus of scientific publications to improve performance on downstream scientific NLP tasks. We evaluate on a suite of tasks including sequence tagging, sentence classification and dependency parsing, with datasets from a variety of scientific domains. We demonstrate statistically significant improvements over BERT and achieve new state-of-the-art results on several of these tasks. The code and pretrained models are available at https://github.com/allenai/scibert/.

候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。

HN 书评

推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及
评论andreyk · 2022-06

“How is this BS? "How AI technology can tame the scientific literature" https://www.nature.com/articles/d41586-018-06617-5 "SCIBERT: A Pretrained Language Model for Scientific Text" https://arxiv.org/pdf/1903.10676.pdf "Domain-specific language model pretraining for biomedical natural language processing" https://dl.acm.org/doi/pdf/10.1145/3458754 Thes…”

AI Is Ushering in a New Scientific Revolution原评论 ↗
评论smeeth · 2022-11

“I use scientific language models professionally. I skimmed the paper and was immediately disappointed. - They benchmarked against general models like GPT-3 but not well-established specific models that have been trained for specific tasks like SPECTER[0] or SciBert[1]. Specter outperformed GPT-3 on tasks like citation prediction two years ago. Nobody ser…”

Galactica: an AI trained on humanity's scientific knowledge (by Meta)原评论 ↗