Team Member:

• Kush Mangukiya (24M0769)
• Kaustubh Shejole(24M2109)

Learning Human Expertise in RLHF using Extended-BTL Model

This is the Readme file of the project done by Mr. Kaustubh and Mr. Kush under the guidance of Prof. Arpit Agarwal for the course project of Human-Centered AI at CSE IIT Bombay.

The project is about learning the variable competence of workers performing pairwise comparisons in RLHF settings.

The code folder contains latex code and two .ipynb notebooks, The first one i.e. Polya_gamma_original_em_model.ipynb contains the original code for Polya-Gamma based optimization with taking contrainsts on rewards and competence in consideration.

The another notebook contains the version of EM Model that can work with GPU, it also contains the code of simulations and experimentation on synthethic dataset. The experiments can be run by sequentially running the Notebooks, we have added sufficient markdown and comments that can help in execution.

We emphasize to continue this work by analysing the competence values, including real datasets and aligning models based on it as evident from the Limitations and Future Work Section of the report.

Thank You.