Human Interview for LLM Evaluation

Mercor connects elite creative and technical talent with leading AI research labs. · ...

+Evaluate LLM-generated responses for effectiveness in answering user queries. Conduct fact-checking using trusted public sources and external tools. · ...

We're looking for an LLM Evaluation, Benchmarking & Experimentation Engineer to rigorously test our proprietary LLM API and build the infrastructure for systematic model improvement. · ...

Evaluate LLM-generated responses on their ability to effectively answer user queries. Conduct fact-checking using trusted public sources and external tools. Generate high-quality human evaluation data by annotating response strengths, areas for improvement, and factual inaccuraci ...

+I'm seeking a technical mentor to help deepen my understanding of LLM evaluation and benchmarking, with particular attention to high-stakes applications (e.g., mental health), while developing a generalizable framework for reasoning about model performance across domains. · ...

We are seeking experienced software engineers who are comfortable working with high-quality public GitHub repositories (500+ stars). This role combines hands-on engineering work with AI model evaluation. · ...

To audit train and improve Large Language Models LLMs specialized in finance ensuring accuracy and mitigating risks. · Key Tasks Evaluating LLM outputs for accuracy logicality and reliability in financial tasks e.g trading reporting Developing Ground Truth data for fine-tuning mo ...

We need someone who can systematically test AI workflows — not just use them, but evaluate output quality, diagnose failure modes (hallucinations, grounding issues, tool-call failures), and help us build rubrics and regression datasets to track quality over time. · ...

We are seeking a talented creative writer who possesses a strong statistical background to develop benchmarks for large language model output evaluation.Develop benchmarks for LLM output evaluation · ...

I am looking for someone with very strong English language and analytical skills to help design evaluation rubrics for assessing AI-generated summaries. · The work involves creating clear, structured marking criteria that can be used to evaluate how well a large language model (L ...

We're building an internal system that helps B2B teams write non-generic outreach by using structured information pulled from public sources (company websites competitor sites LinkedIn posts YouTube video transcripts etc.). The system should generate actionable outreach suggestio ...

I'm hiring a full-time (40 hours/week) assistant to help me execute and scale finance-focused RLHF / LLM evaluation contracts (model grading, rubric design, golden responses QA and reviewer feedback) · This work blends finance (valuation accounting markets) with AI evaluation (co ...

We are seeking a skilled LLM Engineer to design fine-tune deploy Large Language Model–based solutions. · Design develop deploy applications using Large Language Models (LLMs) such as GPT Claude LLaMA or similar. · ...

We are building a serious AI product focused on transforming real-world business conversations into structured intelligence insights and automation. · AI pipelines that analyze recorded conversations speech text structured insights · LLM-based systems for summarization classifi ...

We are looking for a part-time Research Advisor or Consultant to help guide our methodology and provide high-level input as we scale our evaluation framework and data programs. · ...

We are building a GenAI-driven recommendation engine that generates structured recommendations by passing user context + prompts to LLMs and evaluating the output. · ...

Title: Senior GenAI Engineer (LLMs, RAG) · Location: Toronto, ON (3 Days Hybrid) · Job Overview: · We are looking for a Senior GenAI Engineer to design and build LLM-powered AI solutions supporting business users. This role involves developing RAG pipelines, embeddings, prompt en ...

Data Scientist – II (4+ Years Experience) · Mandatory Requirements (Non-Negotiable): · * Strong background in NLP, LLMs, prompt engineering, and deep learning · * Strong proficiency in Python · * Experience with LangChain, PyTorch, and Pandas · * Hands-on experience with LLMs (an ...

+We are seeking a PhD-level expert in Psychology or related field, strong knowledge of consciousness and theory of Mind, who also possesses understanding of Large Language Models (LLMs). · +Evaluating theories related to consciousness and contributing insights on how LLMs can be ...

We need an experienced PyTorch ML engineer to modify our LLM routing framework to add response-aware routing capabilities. · The system currently makes routing decisions before any LLM generates a response; it only evaluates the incoming prompt to predict difficulty. · ...