FIRE 2023

Forum for Information Retrieval Evaluation

Goa Business School, Goa University, Panjim

15th-18th December

Unleashing the Power of Large Language Models: A Hands-On Tutorial
by Payel Santra, Madhusudan Ghosh, Shrimon Mukherjee, Debasis Ganguly, Partha Basuchowdhuri, and Sudip Kumar Naskar

Large Language Models (LLMs) have exhibited exceptional proficiency in the realm of natural language processing and other domains. The effectiveness of LLMs has led to a steep rise in various research fields including both academia and industry. In this tutorial, we present the audience with an introduction to LLMs and the associated challenges. We briefly discuss certain recent publicly accessible LLMs, specifically a robust chatbot named ChatGPT, and the significant impact that the technological advancement of LLMs has had on the entire AI community. The tutorial is structured in the following manner. First, we provide a brief preface that outlines the fundamental principles of Natural Language Processing (NLP). Following that, we explore the area of distributional representation learning for NLP. Later on, we delve into the essential component of transformer-based pretrained language models. We gradually outline the concept of prompt learning and how it replaced the pretraining and finetuning process. Then we discuss the concept of in-context learning (ICL) in LLMs, which entails making predictions based on augmented contexts with a limited number of examples and also the challenges of using LLMs. Afterward, a hands-on coding and demonstration session is carried out in a practical manner.

Explainability of Text Processing and Retrieval Methods
by Sourav Saha, Avishek Anand, Procheta Sen, and Mandar Mitra

This tutorial presents explainability of text processing and retrieval methods, an emerging area focused on fostering responsible and trustworthy deployment of machine learning systems in the context of information retrieval. As the field has rapidly evolved in the past 4-5 years, numerous approaches have been proposed that focus on different access modes, stakeholders, and model development stages. This tutorial aims to introduce IR-centric notions, classification, and evaluation styles in explainable information retrieval (ExIR) while focusing on IR-specific tasks such as ranking, text classification, and learning-to-rank systems. We will delve into method families and their adaptations to IR, extensively covering post-hoc methods, axiomatic and probing approaches, and recent advances in interpretability-by-design approaches. We will also discuss ExIR applications for different stakeholders, such as researchers, practitioners, and end-users, in contexts like web search, patent and legal search, and high-stakes decision-making tasks. To facilitate practical understanding, we will provide a hands-on session on applying text processing and ExIR methods, reducing the entry barrier for students, researchers, and practitioners alike. Earlier version of this tutorial has been presented in SIGIR 2023.

Recent Advancements in Unbiased Learning to Rank
by Shashank Gupta, Philipp Hager, and Harrie Oosterhuis

The previous tutorials on Unbiased Learning to Rank (ULTR) focused on introducing the fundamentals of the area to beginners and practitioners. While relevant at the time, the field of ULTR has matured significantly, and fundamental advancements have been made since then. Recently, Learning to Rank (LTR) has also seen significant growth from the application side, including fair LTR. The focus of the previous tutorials was primarily on the fundamentals of ULTR with a limited emphasis on practical applications. To bridge this gap between theory and practice, we plan to discuss recent applications of ULTR in this tutorial. We will discuss the latest interaction biases from the ULTR literature, the latest research in bias correction techniques, and novel applications of ULTR including fair LTR, and finally, conclude with some open questions and future work.

AI and Data-driven Approaches for Agriculture
by Manjira Sinha, Tirthankar Dasgupta, and Adway Mitra

According to data from Govt. of India, agriculture is a high-priority sector of the Indian economy, with 58\% of all families dependent on it directly or indirectly, for their livelihoods. With increasing world population, climate change and limited arable land, the current practices in agriculture are becoming unsustainable for both people and planet as they incur huge environmental cost and wastage. These in turn leave the small producers at or below the poverty level.
Therefore, there is an urgent need of paradigm shift in this domain, and adoption of digital and AI based technologies can aid in the shift. AI and Data-driven technologies have the potential to enhance productivity and efficiency, thus generating higher farm income, waste reduction and sustainability.
In this tutorial we will briefly describe about how we can apply state of the art ML/DL techniques on data from different modalities and different perspective to gain useful insights on enhanced agricultural output.