Awesome papers about intent understanding
Introduction
The intent clarify component is to understand the question, and guide the retrieval to obtain better documents. Usually, there could be different types of understanding, e.g., whether to retrieve and query formulations.
1. Retrieval Detection
These kind of methods try to determine whether to retrieve content for response generation or directly generate the response.
| Date | Title | Authors | Orgnization | Abs | Dataset |
|---|---|---|---|---|---|
| 2024/05/04 | When to Retrieve: Teaching LLMs to Utilize Information Retrieval Effectively [code:  ] |
Tiziano Labruna, et al. | University of Bozen-Bolzano | <summary>This paper presents ADAPT-LLM …</summary>This paper presents ADAPT-LLM by fine-tuning a base LLM on an open-domain QA dataset. It first take base LLM to zero-shot evaluation to determin its accuracy in QA. For questions with incorrect answers, it train the LLM to generate a spectial token |
NQ, SQuAD, PopQA |
| 2023/10/08 | Self-Knowledge Guided Retrieval Augmentation for Large Language Models | Yile Wang, Peng Li, Maosong Sun, Yang Liu | Tsinghua University | <summary>This paper presents SKR …</summary>This work introduces Self-Knowledge guided Retrieval augmentationSKR to flexibly call the retriever. Three steps: 1) collection self-knowledge of LLM by asking a number of questions, and divide the question into two categories D+ and D- according to the answer correctness, 2) eliciting self-knowledge of LLMs by either direct prompt or training a classifier based on D+ and D-, 3) using self-knowledge for adaptive retrieval augmentation based on prediction of 2). |
TemporalQA, CommonsenseQA, TabularQA, StrategyQA, TruthfulQA |
2. Query Reformulation
| Date | Title | Authors | Orgnization | Abs | Dataset |
|---|---|---|---|---|---|
| 2024/03/31 | RQ-RAG: Learning to Refine Queries for Retrieval Augmented Generation | Chi-Min Chan, Chunpu Xu, Ruibin Yuan, et. al. | Hong Kong University of Science and Technology, Hong Kong Polytechnic University, MIT | <summary>This paper presents RQ-RAG …</summary>This work proposes RQ-RAG (Refine Query RAG) to enhance the generator (LLaMA2) to explicitly rewrite, decompose, and disambiguate, before final answer generation. In this way, the RAG process interleaves between retrieval (guided by refined query) and generation. |
Arc-Challenge, PopQA, OpenbookQA, HotpotQA, 2WikiMHQA, Musique |
3. Query Expansion
3.1 Generative-Relevance Feedback
| Date | Title | Authors | Orgnization | Abs | Dataset |
|---|---|---|---|---|---|
| 2024/11/12 | Exploring the Best Practices of Query Expansion with Large Language Models [code:  ] |
Le Zhang, Yihong Wu, Qian Yang, et. al. | University of Montreal | <summary>This paper presents MUGI …</summary>This work proposes MUGI (Multi-Text Generation Integration), which leverages LLM to generate multiple pseudo-references. Findings: 1) increasing the number of references from LLM benefits IR systems; 2) A balance between the query and pseudo-documents, and an effective integration strategy is important; 3) contextual information from LLM is essential. |
TREC DL19 and DL20, BEIR |
| 2024/10/21 | GaQR: An Efficient Generation-augmented Question Rewriter | Oliver Young, Yixing Fan, Ruqing Zhang, et al. | ICT, CAS | <summary>This paper presents GaQR …</summary>The work proposes introduce an efficient GaQR to reformulate a question into several queries using Chain of Thought (CoT) and make it more efficient through knowledge distillation. |
MS MARCO, Miracl, BEIR |
| 2024/08/24 | Meta Knowledge for Retrieval Augmented Large Language Models | Laurent Mombaerts, Terry Ding, Florian Felice, Jonathan Taws, Adi Banerjee, Tarik Borogovac | Amazon Web Services | <summary>This paper presents MK Summary …</summary>The work proposes a novel data-centric RAG workflow for LLMs, relying on generating metadata and synthetic Questions and Answers (QA) for each document, as well as introducing the new concept of Meta Knowledge Summary (MK Summary) for metadata-based clusters of documents. It transforms the traditional retrieve-then-read system into a more advanced prepare-then-rewrite-then-retrieve-then-read framework, to achieve higher domain expert-level understanding of the knowledge base. |
arXiv |
| 2023/10/19 | Large Language Models Know Your Contextual Search Intent: A Prompting Framework for Conversational Search [code:  ] |
Kelong Mao, Zhicheng Dou, Fengran Mo, Jiewen Hou, Haonan Chen, Hongjin Qian | Renmin University of China | <summary>This paper presents LLM4CS …</summary>The work proposes a simple yet effective prompting framework, called LLM4CS, to leverage LLMs as a text-based search intent interpreter to help conversational search.It explores three prompting methods to generate multiple query rewrites and hypothetical responses, and then proposes to aggregate them into an integrated representation. |
CAsT-19&20&21 |
| 2023/10/11 | Query2doc: Query Expansion with Large Language Models, [code:  ] |
Liang Wang, Nan Yang, Furu Wei | Microsoft Research | <summary>This paper presents Query2doc …</summary>This work proposes a simple yet effective query expansion approach, denoted as Query2doc, to improve both sparse and dense retrieval systems. The proposed method first generates pseudo-documents by few-shot prompting large language models (LLMs), and then expands the query with generated pseudo-documents. |
MS-MARCO passage, TREC-DL 19&20 |
| 2023/6/16 | GRM: Generative Relevance Modeling Using Relevance-Aware Sample Estimation for Document Retrieval | Iain Mackie, Ivan Sekulic, Shubham Chatterjee, Jeffrey Dalton, Fabio Crestani. | University of Glasgow,Università della Svizzera italiana | <summary>This paper presents GRM …</summary>This work proposes Generative Relevance Modeling (GRM) that uses Relevance-Aware Sample Estimation (RASE) for more accurate weighting of expansion terms. Specifically, it identifies similar real documents for each generated document and uses a neural re-ranker to estimate their relevance. |
CODEC, Robust04 |
| 2022/12/20 | Precise Zero-Shot Dense Retrieval without Relevance Labels, [code:  |
Luyu Gao, Xueguang Ma, Jimmy Lin, Jamie Callan. | Language Technologies Institute, Carnegie Mellon University,David R. Cheriton School of Computer Science, University of Waterloo | <summary>This paper presents HyDE …</summary>This work proposes to pivot through Hypothetical Document Embeddings(HyDE), which first zero-shot instructs an instruction-following language model to generate a hypothetical document and then grounds the generated document to the actual corpus with an unsupervised contrastively learned encoder. |
TREC-DL 19&20, BEIR |
3.2 Pseudo-Relevant Feedback
| Date | Title | Authors | Orgnization | Abs | Dataset |
|---|---|---|---|---|---|
| 2023/8/2 | Large Language Models are Strong Zero-Shot Retriever | Tao Shen, Guodong Long, Xiubo Geng, Chongyang Tao, Tianyi Zhou, Daxin Jiang | AAII, Microsoft,University of Maryland | <summary>This paper presents LameR …</summary>This work proposes the Language language model as Retriever (LameR) to augment a query with its potential answers by prompting LLMs with a composition of the query and the query’s in-domain candidates and proposes to leverage a non-parametric lexicon-based method (e.g., BM25) as the retrieval module to capture query-document overlap in a literal fashion. |
MS-MARCO passage, TREC-DL 19&20, BEIR |
3.3 Methods Combination
| Date | Title | Authors | Orgnization | Abs | Dataset |
|---|---|---|---|---|---|
| 2023/12/12 | Synergistic Interplay between Search and Large Language Models for Information Retrieval [code:  ] |
Tao Shen, Guodong Long, Xiubo Geng, Chongyang Tao, Tianyi Zhou, Daxin Jiang | Peking University, Microsoft,AAII | <summary>This paper presents InteR …</summary>This work proposes InteR, a novel framework that facilitates information refinement through synergy between RMs and LLMs,which allows RMs to expand knowledge in queries using LLM-generated knowledge collections and enables LLMs to enhance prompt formulation using retrieved documents. |
TREC-DL 19&20, BEIR |
| 2023/5/12 | Generative and Pseudo-Relevant Feedback for Sparse, Dense and Learned Sparse Retrieval | Iain Mackie, Shubham Chatterjee, Jeffrey Dalton. | University of Glasgow | <summary>This paper presents …</summary>This work proposes combining generative and pseudo-relevance feedback ranking signals to achieve the benefits of both feedback classes. |
Robust04, TREC-DL 19&20, CODEC |
| 2023/5/5 | Query Expansion by Prompting Large Language Models | LRolf Jagerman, Honglei Zhuang, Zhen Qin, Xuanhui Wang, Michael Bendersky. | Google Research | <summary>This paper presents …</summary>This work proposes an approach to query expansion that leverages the generative abilities of Large Language Models (LLMs) and studies a variety of different prompts, including zero-shot, few-shot and Chain-of-Thought (CoT) finding that CoT prompts are especially useful for query expansion. |
MS-MARCO passage, BEIR |