Senior Software Engineer – LLM Evaluation

Project Overview: We are supporting a client building an AI Shopping Assistant that responds to natural-language user queries by generating product collections and refinement suggestions. · This role focuses on evaluating and improving LLM outputs by reviewing interaction traces, ...

+Evaluate LLM-generated responses for effectiveness in answering user queries. Conduct fact-checking using trusted public sources and external tools. · ...

We're looking for an LLM Evaluation, Benchmarking & Experimentation Engineer to rigorously test our proprietary LLM API and build the infrastructure for systematic model improvement. · ...

Evaluate LLM-generated responses on their ability to effectively answer user queries. Conduct fact-checking using trusted public sources and external tools. Generate high-quality human evaluation data by annotating response strengths, areas for improvement, and factual inaccuraci ...

Mercor connects elite creative and technical talent with leading AI research labs. · ...

+I'm seeking a technical mentor to help deepen my understanding of LLM evaluation and benchmarking, with particular attention to high-stakes applications (e.g., mental health), while developing a generalizable framework for reasoning about model performance across domains. · ...

Participants will undergo a three-step procedure including informed consent review and a one-on-one text-based chat interview. · ...

To audit train and improve Large Language Models LLMs specialized in finance ensuring accuracy and mitigating risks. · Key Tasks Evaluating LLM outputs for accuracy logicality and reliability in financial tasks e.g trading reporting Developing Ground Truth data for fine-tuning mo ...

We need someone who can systematically test AI workflows — not just use them, but evaluate output quality, diagnose failure modes (hallucinations, grounding issues, tool-call failures), and help us build rubrics and regression datasets to track quality over time. · ...

We are seeking a talented creative writer who possesses a strong statistical background to develop benchmarks for large language model output evaluation.Develop benchmarks for LLM output evaluation · ...

We're building an internal system that helps B2B teams write non-generic outreach by using structured information pulled from public sources (company websites competitor sites LinkedIn posts YouTube video transcripts etc.). The system should generate actionable outreach suggestio ...

I'm hiring a full-time (40 hours/week) assistant to help me execute and scale finance-focused RLHF / LLM evaluation contracts (model grading, rubric design, golden responses QA and reviewer feedback) · This work blends finance (valuation accounting markets) with AI evaluation (co ...

We are seeking a skilled LLM Engineer to design fine-tune deploy Large Language Model–based solutions. · Design develop deploy applications using Large Language Models (LLMs) such as GPT Claude LLaMA or similar. · ...

We are looking for a part-time Research Advisor or Consultant to help guide our methodology and provide high-level input as we scale our evaluation framework and data programs. · ...

We're building an LLM-powered assistant (agent) that can answer Korean healthcare-domain questions accurately and safely. · ...

We are building a serious AI product focused on transforming real-world business conversations into structured intelligence insights and automation. · AI pipelines that analyze recorded conversations speech text structured insights · LLM-based systems for summarization classifi ...

We are looking for a hands-on LLM data/evaluation practitioner to create accurate and credible marketing content. · We need a technical writer who truly understands how LLM training and evaluation projects work in practice. · This is not a generic prompt engineer role. We need so ...

We are building a GenAI-driven recommendation engine that generates structured recommendations by passing user context + prompts to LLMs and evaluating the output. · ...

I'm looking for an experienced ML/LLM Engineer to do a focused Architecture Review. · ...

+We are seeking a PhD-level expert in Psychology or related field, strong knowledge of consciousness and theory of Mind, who also possesses understanding of Large Language Models (LLMs). · +Evaluating theories related to consciousness and contributing insights on how LLMs can be ...