EAR query reranker checkpoints

Official scorer checkpoints for Expand, Rerank, and Retrieve: Query Reranking for Open-Domain Question Answering (Findings of ACL 2023).

EAR is a query Expansion And Reranking method. It samples diverse query expansions and trains a reranker to select expansions that improve passage retrieval.

Contents

The repository preserves the original experiment directory names:

Path	Dataset	Variant	Files
best_nq/vanilla	Natural Questions	EAR-RI	3
best_nq/wtop1	Natural Questions	EAR-RD	3
best_trivia/vanilla	TriviaQA	EAR-RI	3
best_trivia/wtop1	TriviaQA	EAR-RD	4

Each directory contains answer, sentence, and title scorer checkpoints. best_trivia/wtop1 additionally preserves the historical scorer-answer-best.bin file from the original release.

These are raw PyTorch checkpoint/state-dict files used by the EAR codebase. They are not standalone AutoModel.from_pretrained repositories.

Usage

Use these checkpoints with the official code:

git clone https://github.com/voidism/EAR
cd EAR

For EAR-RI:

bash one_pass_eval_ri.sh nq /path/to/best_nq/vanilla
bash one_pass_eval_ri.sh trivia /path/to/best_trivia/vanilla

For EAR-RD:

bash one_pass_eval_rd.sh nq /path/to/best_nq/wtop1
bash one_pass_eval_rd.sh trivia /path/to/best_trivia/wtop1

The original release used Python 3.7.13, PyTorch 1.10.1, Transformers 4.24.0, Tokenizers 0.11.1, Pyserini, and Weights & Biases. The rerankers were trained from microsoft/deberta-v3-base.

SOURCE_MANIFEST.sha256 contains a SHA-256 checksum for every checkpoint. The source archive was models_ear.tar.gz with SHA-256 d756d111404fe8859fd094e313f1e2b95c489691228dfb05044c6471fe31c819.

No training or evaluation dataset is included in this model repository.

Links

• Paper: https://aclanthology.org/2023.findings-acl.768/
• Code: https://github.com/voidism/EAR
• arXiv: https://arxiv.org/abs/2305.17080

Citation

@inproceedings{chuang-etal-2023-expand,
  title = {Expand, Rerank, and Retrieve: Query Reranking for Open-Domain Question Answering},
  author = {Chuang, Yung-Sung and Fang, Wei and Li, Shang-Wen and Yih, Wen-tau and Glass, James},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2023},
  year = {2023},
  pages = {12131--12147},
  doi = {10.18653/v1/2023.findings-acl.768},
  url = {https://aclanthology.org/2023.findings-acl.768/}
}