AskCR, CR’s AI-powered advisor that answers your questions based on CR’s trusted research and product data, is evolving to deliver increasingly relevant and reliable information to consumers. We’re seeking a creative consultant to help build next-generation observability, analysis and post-precessing systems in AskCR.
This consultant opportunity will focus on designing and implementing real-time guardrails, hallucination detection, and post-processing analysis pipelines to ensure AskCR delivers safe, reliable, and high-quality information to consumers. This opportunity will span 2-3 months and is remote, with the option for hybrid work for candidates located near CR’s offices in New York.
We are Looking for Someone Who Can
- Design and implement an observability system with both real-time guardrails and asynchronous analysis
- Create innovative approaches to hallucination detection, fact-checking, topic modeling, intent mapping and more
- Create dashboards and evaluation frameworks to continuously measure and improve reliability
How You’ll Make an Impact
- Architect a real-time output guardrails system that intercepts potential issues before they reach users, including content moderation, hallucination detection, and guidance
- Develop asynchronous analysis pipelines that process conversation logs to identify trends, quality issues, and opportunities for improvement
- Work to make sure responses remain safe, helpful, and aligned with our values
- Develop trace analysis tools to pinpoint and address the root causes of quality issues
What We’re Looking For
- Experience with AI observability tools and performance monitoring systems
- Experience in Databricks
- Knowledge of LLM guardrails and safety mechanisms
- Familiarity with post-processing techniques like response ranking, factual verification, and linguistic refinement
- Strong software engineering skills with experience building reliable production AI systems
- Passion for consumer advocacy and commitment to AI that genuinely helps people
Why Join Us?
This is an opportunity to push the boundaries of AI observability while supporting CR’s mission of creating a fair, just marketplace. Your work will directly impact the quality and trustworthiness of information provided to millions of consumers.
Interested? Let’s talk! Reach out by emailing us at innovationlab@cr.consumer.org with your background, portfolio, and thoughts on building observability systems that protect users and continuously improve AI assistants.
