“We benchmarked Gemini 2.5 on 100 open source object detection datasets in our paper: https://arxiv.org/abs/2505.20612 (see table 2) Notably, performance on out of distribution data like those in RF100VL is super degraded It worked really well zero-shot (comparatively to the foundation model field) achieving 13.3 average mAP, but counterintuitively perfo…”
Roboflow100-VL: A Multi-Domain Object Detection Benchmark for Vision-Language Models
Vision-language models (VLMs) trained on internet-scale data achieve remarkable zero-shot detection performance on common objects like car, truck, and pedestrian. However, state-of-the-art models still struggle to generalize to out-of-distribution classes, tasks and imaging modalities not typically found in their pre-training. Rather than simply re-training VLMs on more visual data, we argue that one should align VLMs to new concepts with annotation instructions containing a few visual examples and rich textual descriptions. To this end, we introduce Roboflow100-VL, a large-scale collection of 100 multi-modal object detection datasets with diverse concepts not commonly found in VLM pre-training. We evaluate state-of-the-art models on our benchmark in zero-shot, few-shot, semi-supervised, and fully-supervised settings, allowing for comparison across data regimes. Notably, we find that VLMs like GroundingDINO and Qwen2.5-VL achieve less than 2% zero-shot accuracy on challenging medical imaging datasets within Roboflow100-VL, demonstrating the need for few-shot concept alignment. Lastly, we discuss our recent CVPR 2025 Foundational FSOD competition and share insights from the community. Notably, the winning team significantly outperforms our baseline by 17 mAP! Our code and dataset are available at https://github.com/roboflow/rf100-vl and https://universe.roboflow.com/rf100-vl/.
候选层:至少 2 个不同的人、在 2 个不同月份提到过,还没到主书架的 3 人 3 个月。没有做立场标注,所以只写“被提及”。
HN 书评
推荐在前,同类按时间 · 每篇最多保留 20 条,统计用全部提及“> See the url in my comment (search for the term rfdetr-2xlarge). 2XL does indeed go up to 800x800 and has PML1.0 license instead of apache 2.0. All of the models, including the Apache 2.0 ones, can be configured to go higher than 800x800. The difference between the ones with the PML license and the Apache 2.0 ones is the backbone, not the resolution. I…”