OpenBMB/Trident

OpenBMB/Trident

Stars: 1

Forks: 0

Open issues: 0

Created: 2026-10-08T11:52:19Z

Pushed: 2026-10-09T03:01:26Z

Default branch: main

Fork: no

Archived: no

README: Trident

Mitigating Modality Preference in Mixed-Modality Retrieval

Official repository for "Chaos in the Text: Revealing the Modality Preference in Mixed-Modality Retrievers"

> 🚧 This repository is under construction. Code, data, and model checkpoints will be released soon. Star / watch the repo to get notified.

Overview

Dense retrievers perform well on text and image corpora separately, but real-world corpora often mix text-only, image-only, and fused text-image documents. We find that retrieval performance is highly sensitive to modality composition:

• Chaos in the Text. Irrelevant text hurts more than an equal number of irrelevant images, and retrievers often rank irrelevant text above relevant images.
• Modality preference. Text representations receive systematically higher similarity scores than image representations.

To mitigate this bias, we propose Trident, which builds text, image, and fused text-image views of each document as co-equal positives, and jointly optimizes relevance discrimination and positive-view balance via Multi-Positive View InfoNCE. Trident improves mixed-modality retrieval on both CLIP-based and VLM-based architectures, reduces sensitivity to modality composition and text distractors, and improves average single-modality retrieval performance.

Release Plan

• [ ] Paper on arXiv
• [ ] Training code
• [ ] Evaluation code and mixed-modality benchmark construction scripts
• [ ] Model checkpoints
• [ ] Usage examples

Repository Structure

Trident/
├── README.md
└── (to be added)

Citation

If you find this work useful, please cite:

coming soon

Contact

For questions, please open an issue or contact the corresponding authors: Yukun Yan and Zhenghao Liu.