ConversationalIR

Workshops and conferences

Please make a pull request if you find a venue to be missing.

2017

• Search-oriented conversational AI @ ICTIR 2017
• Conversational Approaches to IR @ SIGIR 2017
• Talking with Conversational Agents in Collaborative Action @ CSCW 2017
• 1st Conversational AI: Today's Practice and Tomorrow's Potential @ NeurIPS 2017

2018

• Search-oriented conversational AI @ EMNLP 2018
• 1st Workshop of Knowledge-aware and Conversational Recommender Systems @ RecSys 2018
• Conversational Approaches to IR @ SIGIR 2018
• 2nd Conversational AI: Today's Practice and Tomorrow's Potential @ NeurIPS 2018

2019

• NLP for Conversational AI @ ACL 2019
• Search-Oriented Conversational AI @ WWW 2019
• Search-Oriented Conversational AI @ IJCAI 2019
• Conversational Interaction Systems @ SIGIR 2019
• Conversational Agents: Acting on the Wave of Research and Development @ CHI 2019
• 1st International Conference on Conversational User Interfaces 2019
• Dagstuhl 2019 Seminar Conversational Search
• 3rd Conversational AI: Today's Practice and Tomorrow's Potential @ NeurIPS 2019
• user2agent 2019: 1st Workshop on User-Aware Conversational Agents @ IUI 2019

2020

• Conversational Systems for E-Commerce Recommendations and Search @ WSDM 2020
• 3rd International Workshop on Conversational Approaches to Information Retrieval (CAIR) @ CHIIR 2020
• Search-Oriented Conversational AI @ EMNLP 2020
• user2agent 2020: 2nd Workshop on User-Aware Conversational Agents @ IUI 2020
• 2nd International Conference on Conversational User Interfaces 2020
• KDD Converse 2020: Workshop on Conversational Systems Towards Mainstream Adoption
• CUI@CSCW2020: Collaborating through Conversational User Interfaces
• ACM Multimedia 2020 workshop Multimodal Conversational AI

2021

• Workshop on Mixed-Initiative ConveRsatiOnal Systems @ ECIR 2021
• CUI@IUI2021 workshop: Theoretical and Methodological Challenges in Intelligent Conversational User Interface Interactions
• Conversational User Interfaces Conference (CUI 2021)
• Future Conversations 2021 workshop @ CHIIR 2021
• Let's Talk About CUIs: Putting Conversational User Interface Design Into Practice @ CHI 2021
• SCAI QReCC 21 Conversational Question Answering Challenge as part of the 6th SCAI workshop
• 3rd NLP for Conversational AI Workshop @ EMNLP 2021

2022

• When Creative AI Meets Conversational AI Workshop @ COLING 2022
• 7th Workshop on Search-Oriented Conversational Artificial Intelligence @ SIGIR 2022
• 2nd Workshop on Mixed-Initiative ConveRsatiOnal Systems (MICROS) @ CIKM 2022
• 4th Workshop on NLP for Conversational AI @ ACL 2022

Research themes

This categorization was created in a bottom-up manner based on the 300+ papers accepted by the workshops/conferences listed above (up to and including the MICROS 2021 workshop). Each paper was included in only one category or sub-category. This is of course purely subjective; ask 10 different researchers to come up with categories and you get 10 different results ...

The bracketed numbers starting with → indicate the total number of papers in this branch of the tree; e.g. there are a total of 48 papers that fall into some node inside the Domains category. The bracketed numbers after a sub-category without → indicate the number assigned to this particular node by me. For instance, chitchat (4) means that 4 papers fell in this category while Alexa Prize (3) means that three - different - papers fell into this more specific sub-category.

The mindmap was created with markmap, a neat markdown to mindmap tool. The svg of the mindmap is also available: open it in your favourite browser to experience an unpixelated mindmap. If you want to reuse the categories, alter/update/edit them, take the markdown file as starting point and then head over to markmap!

Interestingly, if you check out the report published about the Dagstuhl Conversational Search seminar in late 2019 (where many researchers came together to define/work on a roadmap) you will find that the themes chosen there are not well covered in the above research lines. Dagstuhl themes:

• Defining conversational search
• Evaluating conversational search
• Modeling in conversational search
• Argumentation and explanation
• Scenarios that invite conversational search
• Conversational search for learning technologies
• Common conversational community prototype

Relevant datasets/benchmarks 🗂 and leaderboards 🚴

• 🚴 XOR-TyDi QA
  • "XOR-TyDi QA brings together for the first time information-seeking questions, open-retrieval QA, and multilingual QA to create a multilingual open-retrieval QA dataset that enables cross-lingual answer retrieval. It consists of questions written by information-seeking native speakers in 7 typologically diverse languages and answer annotations that are retrieved from multilingual document collections."
• 🚴 Leaderboard for multi-turn response selection
  • "Multi-turn response selection in retrieval-based chatbots is a task which aims to select the best-matched response from a set of candidates, given the context of a conversation. This task is attracting more and more attention in academia and industry. However, no one has maintained a leaderboard and a collection of popular papers and datasets yet. The main objective of this repository is to provide the reader with a quick overview of benchmark datasets and the state-of-the-art studies on this task, which serves as a stepping stone for further research."
• 🚴 Papers with code leaderboard for conversational response selection
• 🗂 MANtIS: a Multi-Domain Information Seeking Dialogues Dataset.
  • "Unlike previous information-seeking dialogue datasets that focus on only one domain, MANtIS has more than 80K diverse conversations from 14 different Stack Exchannge sites, such as physics, travel and worldbuilding. Additionaly, all dialogues have a url, providing grounding to the conversations. It can be used for the following tasks: conversation response ranking/generation and user intent prediction. We provide manually annotated user intent labels for more than 1300 dialogues, resulting in a total of 6701 labeled utterances."
• 🗂 CANARD
  • "CANARD is a dataset for question-in-context rewriting that consists of questions each given in a dialog context together with a context-independent rewriting of the question."
  • "CANARD is constructed by crowdsourcing question rewritings using Amazon Mechanical Turk. We apply several automatic and manual quality controls to ensure the quality of the data collection process. The dataset consists of 40,527 questions with different context lengths."
• 🗂 QReCC
  • "We introduce QReCC (Question Rewriting in Conversational Context), an end-to-end open-domain question answering dataset comprising of 14K conversations with 81K question-answer pairs. The goal of this dataset is to provide a challenging benchmark for end-to-end conversational question answering that includes the individual subtasks of question rewriting, passage retrieval and reading comprehension."
• 🗂 CAsT-19: A Dataset for Conversational Information Seeking
  • "The corpus is 38,426,252 passages from the TREC Complex Answer Retrieval (CAR) and Microsoft MAchine Reading COmprehension (MARCO) datasets. Eighty information seeking dialogues (30 train, 50 test) are on average 9 to 10 questions long. A dialogue may explore a topic broadly or drill down into subtopics." (source)
• 🗂 FIRE 2020 task: Retrieval From Conversational Dialogues (RCD-2020)
  • "Task 1: Given an excerpt of a dialogue act, output the span of text indicating a potential piece of info