Grounding DINO 零样本目标检测实战：文本提示与多类别筛选

Grounding DINO 零样本目标检测实战文本提示与多类别筛选这篇教程根据我复现 Grounding DINO 零样本目标检测流程时整理重点演示如何编译模型、加载权重、设置文本提示并对单图和多图进行开放词汇检测。Grounding DINO 适合用自然语言直接找目标不需要提前把类别写死在分类头里。本文更适合做开放词汇目标检测的入门模板。本文会重点跑通以下流程编译 Grounding DINO 并安装依赖准备示例图片和模型权重使用文本提示执行零样本检测调整提示词和阈值筛选目标把同一模型继续用于本地样例图测试如果你正在系统学习目标检测、实例分割、OCR、多目标跟踪或视觉大模型建议收藏本文配套 notebook、示例图片和运行环境说明后续会继续整理。如果环境配置卡住可以在评论区说明具体报错。 文章目录Grounding DINO 零样本目标检测实战文本提示与多类别筛选⚙️ 环境准备⬇️ 下载权重️ 准备示例图片 加载 Grounding DINO 模型 单图零样本检测 多目标提示词测试 本地数据集验证 结果汇总 小结 同系列教程汇总⚙️ 环境准备先检查运行环境并安装依赖。建议优先使用带 NVIDIA GPU 的环境避免推理和训练阶段显存不足。!nvidia-smiimportos HOMEos.getcwd()print(HOME)%cd{HOME}!git clone https://github.com/IDEA-Research/GroundingDINO.git# Fix for CUDA build error introduced in PyTorch 2.1 (updated by default in Google Colab)# Older PyTorch versions still work without this patch, see https://github.com/IDEA-Research/GroundingDINO/issues/402%cd{HOME}/GroundingDINO/groundingdino/models/GroundingDINO/csrc/MsDeformAttn !sed-is/value.type()/value.scalar_type()/gms_deform_attn_cuda.cu !sed-is/value.scalar_type().is_cuda()/value.is_cuda()/gms_deform_attn_cuda.cu%cd{HOME}/GroundingDINO !pip install-q-e.importos CONFIG_PATHos.path.join(HOME,GroundingDINO/groundingdino/config/GroundingDINO_SwinT_OGC.py)print(CONFIG_PATH,; exist:,os.path.isfile(CONFIG_PATH))⬇️ 下载权重先把模型权重准备好后面的推理和训练才能顺利运行。%cd{HOME}!mkdir{HOME}/weights%cd{HOME}/weights !wget-q https://github.com/IDEA-Research/GroundingDINO/releases/download/v0.1.0-alpha/groundingdino_swint_ogc.pthimportos WEIGHTS_NAMEgroundingdino_swint_ogc.pthWEIGHTS_PATHos.path.join(HOME,weights,WEIGHTS_NAME)print(WEIGHTS_PATH,; exist:,os.path.isfile(WEIGHTS_PATH))️ 准备示例图片这一段会准备图片或视频示例。实际复现时可直接换成你从数据集后台下载的资源。%cd{HOME}!mkdir{HOME}/data%cd{HOME}/data# 请从数据集后台下载示例图片并放到 {HOME}/data 目录。# 文件名保持为 dog.jpeg、dog-2.jpeg、dog-3.jpeg、dog-4.jpeg。 加载 Grounding DINO 模型先把模型加载起来再通过文本提示做开放词汇检测。%cd{HOME}/GroundingDINOfromgroundingdino.util.inferenceimportload_model,load_image,predict,annotate modelload_model(CONFIG_PATH,WEIGHTS_PATH) 单图零样本检测先在单张图片上验证文本提示词和阈值设置。importosimportsupervisionassv IMAGE_NAMEdog-3.jpegIMAGE_PATHos.path.join(HOME,data,IMAGE_NAME)TEXT_PROMPTchairBOX_TRESHOLD0.35TEXT_TRESHOLD0.25image_source,imageload_image(IMAGE_PATH)boxes,logits,phrasespredict(modelmodel,imageimage,captionTEXT_PROMPT,box_thresholdBOX_TRESHOLD,text_thresholdTEXT_TRESHOLD)annotated_frameannotate(image_sourceimage_source,boxesboxes,logitslogits,phrasesphrases)%matplotlib inline sv.plot_image(annotated_frame,(16,16))importosimportsupervisionassv IMAGE_NAMEdog-3.jpegIMAGE_PATHos.path.join(HOME,data,IMAGE_NAME)TEXT_PROMPTchair with man sitting on itBOX_TRESHOLD0.35TEXT_TRESHOLD0.25image_source,imageload_image(IMAGE_PATH)boxes,logits,phrasespredict(modelmodel,imageimage,captionTEXT_PROMPT,box_thresholdBOX_TRESHOLD,text_thresholdTEXT_TRESHOLD)annotated_frameannotate(image_sourceimage_source,boxesboxes,logitslogits,phrasesphrases)%matplotlib inline sv.plot_image(annotated_frame,(16,16))importosimportsupervisionassv IMAGE_NAMEdog-3.jpegIMAGE_PATHos.path.join(HOME,data,IMAGE_NAME)TEXT_PROMPTchair, dog, table, shoe, light bulb, coffee, hat, glasses, car, tail, umbrellaBOX_TRESHOLD0.35TEXT_TRESHOLD0.25image_source,imageload_image(IMAGE_PATH)boxes,logits,phrasespredict(modelmodel,imageimage,captionTEXT_PROMPT,box_thresholdBOX_TRESHOLD,text_thresholdTEXT_TRESHOLD)annotated_frameannotate(image_sourceimage_source,boxesboxes,logitslogits,phrasesphrases)%matplotlib inline sv.plot_image(annotated_frame,(16,16))importosimportsupervisionassv IMAGE_NAMEdog-2.jpegIMAGE_PATHos.path.join(HOME,data,IMAGE_NAME)TEXT_PROMPTglassBOX_TRESHOLD0.35TEXT_TRESHOLD0.25image_source,imageload_image(IMAGE_PATH)boxes,logits,phrasespredict(modelmodel,imageimage,captionTEXT_PROMPT,box_thresholdBOX_TRESHOLD,text_thresholdTEXT_TRESHOLD)annotated_frameannotate(image_sourceimage_source,boxesboxes,logitslogits,phrasesphrases)%matplotlib inline sv.plot_image(annotated_frame,(16,16))importosimportsupervisionassv IMAGE_NAMEdog-2.jpegIMAGE_PATHos.path.join(HOME,data,IMAGE_NAME)TEXT_PROMPTglass most to the rightBOX_TRESHOLD0.35TEXT_TRESHOLD0.25image_source,imageload_image(IMAGE_PATH)boxes,logits,phrasespredict(modelmodel,imageimage,captionTEXT_PROMPT,box_thresholdBOX_TRESHOLD,text_thresholdTEXT_TRESHOLD)annotated_frameannotate(image_sourceimage_source,boxesboxes,logitslogits,phrasesphrases)%matplotlib inline sv.plot_image(annotated_frame,(16,16))importosimportsupervisionassv IMAGE_NAMEdog-2.jpegIMAGE_PATHos.path.join(HOME,data,IMAGE_NAME)TEXT_PROMPTstrawBOX_TRESHOLD0.35TEXT_TRESHOLD0.25image_source,imageload_image(IMAGE_PATH)boxes,logits,phrasespredict(modelmodel,imageimage,captionTEXT_PROMPT,box_thresholdBOX_TRESHOLD,text_thresholdTEXT_TRESHOLD)annotated_frameannotate(image_sourceimage_source,boxesboxes,logitslogits,phrasesphrases)%matplotlib inline sv.plot_image(annotated_frame,(16,16))importosimportsupervisionassv IMAGE_NAMEdog-4.jpegIMAGE_PATHos.path.join(HOME,data,IMAGE_NAME)TEXT_PROMPTmens shadowBOX_TRESHOLD0.35TEXT_TRESHOLD0.25image_source,imageload_image(IMAGE_PATH)boxes,logits,phrasespredict(modelmodel,imageimage,captionTEXT_PROMPT,box_thresholdBOX_TRESHOLD,text_thresholdTEXT_TRESHOLD)annotated_frameannotate(image_sourceimage_source,boxesboxes,logitslogits,phrasesphrases)%matplotlib inline sv.plot_image(annotated_frame,(16,16)) 多目标提示词测试通过更长的提示词组合看看模型能否锁定更细粒度的目标。 本地数据集验证把同样的检测逻辑继续放到本地数据集样本上检查泛化表现。%cd{HOME}fromtypesimportSimpleNamespace# 从数据集后台下载 COCO 格式数据集后修改 DATASET_DIR 指向解压目录。DATASET_DIR/content/dataset# 修改为数据集后台导出的数据集目录datasetSimpleNamespace(locationDATASET_DIR)fromrandomimportrandrangefrom.core.datasetimportDatasetdefpick_random_image(dataset:Dataset,subdirrectory:strvalid)-str:image_directory_pathf{dataset.location}/{subdirrectory}image_namesos.listdir(image_directory_path)image_indexrandrange(len(image_names))image_nameimage_names[image_index]image_pathos.path.join(image_directory_path,image_name)returnimage_pathfromimportrf()projectrf.workspace(work-safe-project).project(safety-vest---v4)datasetproject.version(3).download(coco)TEXT_PROMPT, .join(project.classes.keys())TEXT_PROMPTimage_pathpick_random_image(datasetdataset)importosimportsupervisionassv BOX_TRESHOLD0.35TEXT_TRESHOLD0.25image_source,imageload_image(image_path)boxes,logits,phrasespredict(modelmodel,imageimage,captionTEXT_PROMPT,box_thresholdBOX_TRESHOLD,text_thresholdTEXT_TRESHOLD)annotated_frameannotate(image_sourceimage_source,boxesboxes,logitslogits,phrasesphrases)%matplotlib inline sv.plot_image(annotated_frame,(16,16)) 结果汇总最后把几组提示词的效果放在一起总结文本提示对结果的影响。TEXT_PROMPTreflective safety vest, helmet, head, nonreflective safety vestimportosimportsupervisionassv BOX_TRESHOLD0.35TEXT_TRESHOLD0.25image_source,imageload_image(image_path)boxes,logits,phrasespredict(modelmodel,imageimage,captionTEXT_PROMPT,box_thresholdBOX_TRESHOLD,text_thresholdTEXT_TRESHOLD)annotated_frameannotate(image_sourceimage_source,boxesboxes,logitslogits,phrasesphrases)%matplotlib inline sv.plot_image(annotated_frame,(16,16)) 小结Grounding DINO 的核心是提示词设计和阈值筛选。实际项目里先用短提示词定位主要目标再逐步叠加描述词效果通常更稳定。这一类 notebook 建议按“先环境、再数据、再单样例、最后批量推理”的顺序复现。遇到报错时优先检查 GPU、依赖版本、数据集目录和模型权重路径。后续我会继续按源项目顺序整理同系列中的目标检测、实例分割、OCR、多目标跟踪和视觉大模型教程。 同系列教程汇总Google Gemini 3.5 Flash 零样本目标检测教程从提示词到可视化结果GLM-OCR 文档识别实战教程从验证码、公式到车牌 OCRRF-DETR ByteTrack 多目标跟踪实战教程从命令行到 Python 视频轨迹可视化SAM 3 图像分割实战教程文本、框和点提示的多种分割方式Grounding DINO 零样本目标检测实战文本提示与多类别筛选-本文