MentorMatcher

01

Overview

MentorMatcher helps students identify relevant faculty mentors through search, ranking, and conversational discovery. The system began as a RAG-powered FAISS and LangChain prototype and evolved into a service with hybrid retrieval, a dedicated ranking stage, and production-oriented infrastructure.

02

Problem

Faculty profiles contain overlapping terminology, sparse descriptions, and domain-specific language. Retrieval alone can surface plausible candidates but does not always order them well enough for a user deciding whom to contact.

03

Approach

The later architecture combines lexical and vector retrieval, PostgreSQL-backed application data, and a reranking stage behind a FastAPI service. It is packaged with Docker and deployed on AWS EC2.

I prototyped an LLM-as-judge for ranking and evaluated its outputs. The rankings were unstable, so I replaced that stage with a fine-tuned RoBERTa cross-encoder whose narrower objective made ranking behavior more consistent.

04

My contribution

I built the initial search and chat prototype, designed and implemented the later retrieval and ranking pipeline, ran the LLM-judge evaluation, trained the replacement cross-encoder, and shipped the service stack. The first system was used live at a Texas Children’s Hospital research symposium.

05

Evaluation & lessons

The pivotal result was negative: the LLM judge did not provide stable enough rankings. Treating that as an engineering signal, rather than forcing an LLM into the architecture, produced a simpler and more defensible ranking component. The project reinforced that evaluation design and model selection are part of the system architecture.