Lead / Senior Product Manager Analytics, Evals & Conversational BI (Agentic Studio) - Toronto - Netomi

    Netomi
    Netomi Toronto

    2 weeks ago

    Description

    About the Company


    Netomi is the leading agentic AI platform for enterprise customer experience. We work with the largest global brands like Delta Airlines, MetLife, MGM, United, and others to enable agentic automation at scale across the entire customer journey. Our no-code platform delivers the fastest time to market, lowest total cost of ownership, and simple, scalable management of AI agents for any CX use case. Backed by WndrCo, Y Combinator, and Index Ventures, we help enterprises drive efficiency, lower costs, and deliver higher quality customer experiences.

    Want to be part of the AI revolution and transform how the world's largest global brands do business? Join us

    About the Role


    Netomi is hiring a Lead/Senior Product Manager to own the Analytics, Reporting, and Conversational BI product surfaces in Agentic Studio—the measurement and trust layer that helps enterprises build, operate, and continuously improve agentic customer experiences across chat, mobile, email, messaging, and voice/IVR. In mission‑critical environments (airlines, retail, banking, gaming), analytics is not a "dashboard"; it's the control plane for reliability, governance, and outcomes. You will lead the productization of end-to-end analytics including real-time operational monitoring, historical reporting, LLM quality/evaluation (autoevals), observability for tool calling and workflows, and Conversational BI (natural language exploration over a governed metrics model). This role requires deep domain fluency in data science, data preparation, analytics systems, and the realities of LLM-driven systems (probabilistic outputs, evaluation, safety/guardrails, and runtime tracing)—combined with strong product craft, design sense, and delivery execution. This is an opportunity to lead with data at the frontier: partnering with internal Data Science and Engineering teams while collaborating with frontier model and cloud ecosystem partners to deliver best-in-class analytics and governance for enterprise agentic AI.

    Responsibilities

    • Define and execute the Analytics/Evals/Governance roadmap with clear sequencing (MVP → v1 → v2) and measurable adoption targets.
    • Partner deeply with Data Science to productize evaluation methodology (what we score, how we calibrate, how we prevent "gaming," how we track drift).
    • Partner with Engineering and Observability teams to standardize telemetry (tool spans, workflow transitions, model calls, cost/latency metrics) and make it usable in product.
    • Drive a cohesive Agentic Studio UX across Build / Operate / Improve workflows: dashboards, drill-downs, investigation flows, alerts, and remediation actions.
    • Establish objective success metrics and instrument them end-to-end (data correctness, timeliness, reliability, and customer impact).
    • Work with Delivery/CS and enterprise partners to ensure analytics is usable for real operational processes: incident response, change management, governance reviews, and quarterly business reviews.

    Requirements

    • Bachelor's degree in Computer Science, Data Science, Engineering, Statistics, or a related quantitative/technical field (Master's a plus).
    • 6–10+ years of experience across Product Management and/or Data Science/Analytics leadership, including shipping analytics products to enterprise users.
    • Demonstrated depth in: data modeling, telemetry/event schemas, ETL/ELT concepts, and analytics correctness.
    • Building operational + business reporting (real-time monitoring and historical analytics).
    • Experimentation and measurement design (A/B testing concepts, uplift measurement, cohorting).
    • Strong hands‑on experience applied to AI/LLM or data science platforms, including at least several of:
    • LLM evaluation methods (automated scoring, multi‑turn evaluation, calibration).
    • Observability for probabilistic systems (quality + latency + cost).
    • Tool calling / workflow execution analytics and failure taxonomy.
    • Governance measurement (policy outcomes, escalation triggers, compliance reporting).
    • Strong product craft: UX and information architecture instincts, ability to design operator workflows, and capability packaging that scales across customers.

    Preferred Qualification

    • Experience building analytics for multi‑tenant SaaS and regulated customers (auditability, RBAC, retention controls, privacy‑by‑design).
    • Familiarity with cloud analytics stacks and enterprise BI integration patterns.
    • Experience partnering with major cloud and frontier AI vendors (model providers, observability stacks, data platforms) to ship joint capabilities.
    • Experience with contact center / CX metrics (containment, resolution, AHT, CSAT, handoff performance) and multi‑channel operations (voice + digital).

    Netomi is an equal opportunity employer committed to diversity in the workplace. We evaluate qualified applicants without regard to race, color, religion, sex, sexual orientation, disability, veteran status, and other protected characteristics.


    #J-18808-Ljbffr

  • Lead / Senior Product Manager Analytics, Evals & Conversational BI (Agentic Studio)

    4 weeks ago

    Only for registered members Toronto, ON Remote job

    We are hiring a Lead/Senior Product Manager to own the Analytics, Reporting, and Conversational BI product surfaces in Agentic Studio—the measurement and trust layer that helps enterprises build, operate, and continuously improve agentic customer experiences across chat, mobile, ...

  • Lead / Senior Product Manager Analytics, Evals & Conversational BI (Agentic Studio)

    19 hours ago

    Only for registered members Toronto , Canada

    Transform how the world's largest global brands do business.Join Netomi as a Lead/Senior Product Manager for Analytics,Evals and Conversational BI in Agentic Studio.Partner with internal Data Science and Engineering teams to deliver best-in-class analytics and governance for ente ...

  • Lead / Senior Product Manager Analytics, Evals & Conversational BI (Agentic Studio)

    1 month ago

    Only for registered members Toronto , Canada

    We are hiring a Lead/Senior Product Manager to own the Analytics, Reporting, and Conversational BI product surfaces in Agentic Studio—the measurement and trust layer that helps enterprises build, operate, and continuously improve agentic customer experiences across chat, mobile, ...

  • Lead / Senior Product Manager Analytics, Evals & Conversational BI (Agentic Studio)

    4 weeks ago

    Only for registered members Toronto, Ontario Remote job

    We are hiring a Lead/Senior Product Manager to own the Analytics, Reporting, and Conversational BI product surfaces in Agentic Studio. · Define and execute the Analytics/Evals/Governance roadmap with clear sequencing (MVP → v1 → v2) and measurable adoption targets. · Partner deep ...

  • Lead / Senior Product Manager Analytics, Evals & Conversational BI (Agentic Studio)

    1 month ago

    Only for registered members Toronto Full time

    We work with the largest global brands like Delta Airlines, MetLife, MGM, United. · You will lead productization of end-to-end analytics including operational monitoring historical reporting LLM quality/evaluation observability for tool calling workflows Conversational BI natural ...

  • Lead / Senior Product Manager Analytics, Evals & Conversational BI (Agentic Studio)

    1 month ago

    Only for registered members Toronto

    We are hiring a Lead/Senior Product Manager to own the Analytics, Reporting, and Conversational BI product surfaces in Agentic Studio—the measurement and trust layer that helps enterprises build, operate, and continuously improve agentic customer experiences across chat,mobile,em ...

  • Gen AI Developer

    4 weeks ago

    Only for registered members Toronto, ON

    AI Developer specialized in Agentic AI, OpenAI development, and RAG (Retrieval-Augmented Generation) · 4–8+ years in software engineering / ML engineering; 1–3+ years in LLM/agent/RAG systems. · Strong in Python (or TypeScript), async programming, API design, distributed systems. ...

  • Conversational AI Solutions Consultant

    17 hours ago

    Only for registered members Greater Toronto Area Remote job

    We are looking for a single high agency owner to lead our customer deployments. You will act as the technical and strategic quarterback for our customers translating their business goals into scalable AI solutions partnering with engineering to bring them to life and ensuring the ...

  • AI Solutions Engineer

    2 days ago

    Only for registered members Greater Toronto Area Remote job

    We are looking for a single, high-agency owner to lead our customer deployments. · ...

  • Staff Machine Learning Engineer

    1 month ago

    Only for registered members Toronto, Ontario

    Set the technical direction for ML, LLMs, and agents at Boam AI. · ...

  • Staff Machine Learning Engineer

    3 weeks ago

    Only for registered members Toronto, Ontario

    We are looking for a Staff Machine Learning Engineer to set the technical direction for ML, LLMs, and agents at Boam AI. You will architect advanced ML systems, define standards for training and deployment, and own the reliability and quality of ML across the company. · ...

  • AI Training

    1 month ago

    Only for registered members Toronto, Ontario

    We're looking for AI Trainer – Lawyers to help train and evaluate cutting-edge AI models using real legal expertise. If you have the necessary experience, we'll send you a quick 10- to 15-minute test to assess your skills and suitability for AI tasks. If successful, you'll be inv ...

  • Conversational AI Solutions Consultant

    1 day ago

    Only for registered members Toronto

    We are looking for a single high agency owner to lead our customer deployments.You will act as the technical and strategic quarterback for our customers translating their business goals into scalable AI solutions partnering with engineering to bring them to life and ensuring they ...

  • AI Solutions Engineer

    3 days ago

    Only for registered members Toronto

    We are looking for a single high-agency owner to lead our customer deployments. · You will act as the technical quarterback for our customers—translating their business goals into technical requirements building the solution alongside engineering and ensuring they go live fast. · ...

  • Staff Machine Learning Engineer

    1 month ago

    Only for registered members Toronto

    We are looking for a Staff Machine Learning Engineer to set the technical direction for ML, LLMs and agents at Boam AI. · ...

  • Staff Machine Learning Engineer

    4 weeks ago

    Only for registered members Toronto

    We are looking for a Staff Machine Learning Engineer to set the technical direction for ML, LLMs and agents at Boam AI. · As a Staff Machine Learning Engineer, you will set the technical direction for how ML, LLMs and agents power our platform. · ...

  • Staff Machine Learning Engineer

    1 month ago

    Only for registered members Toronto Full time

    We are looking for a Staff Machine Learning Engineer to set the technical direction for ML, LLMs and agents at Boam AI. · ...

  • AI Training

    1 month ago

    Only for registered members Toronto

    We're looking for AI Trainer – Lawyers to help train and evaluate cutting-edge AI models using real legal expertise. · Completing AI training tasks such as analyzing, editing, and writing annotations, grounded in legal reasoning and professional practice Judging the performance o ...

  • Software Developer Backend

    2 weeks ago

    Only for registered members Toronto

    About Dialogue is the · We provide exceptional online health and wellness programs (primary care, mental health, iCBT, EAP, and wellness) to organizations that want to improve the wellness of their employees and families. · ...

  • Software Developer Backend

    3 weeks ago

    Only for registered members Toronto

    We're looking for Software Developers (Backend) to join the new ServiceOS - Workflow Team. · Design, architect, and build workflows to deliver high-velocity results; · ...

  • Internship - MScAC

    1 month ago

    Only for registered members Toronto InternshipSHIP

    Boson AI is an early-stage startup of 30 scientists building large language tools for interaction and entertainment. · We are seeking interns from the University of Toronto enrolled in the MScAC program to join us in our Toronto office. · Work on modeling and training LLMs · Unde ...

Jobs
>
Toronto