QA Lead

What you'll do Strategic QA Leadership: Own the end-to-end testing and evaluation strategy for multi-agent architectures and high-volume data pipelines, defining direction even when product requirements are fluid or ambiguous. Business-to-Evaluation Translation: Act as the primary bridge between Product, Data Science, and Engineering. Translate complex business objectives into concrete, data-driven evaluation criteria and operational rubrics. Brand Scoring Architecture: Oversee the methodology for scoring brands against complex criteria, ensuring that automated systems accurately transform massive datasets into precise, reliable business metrics (e.g., Share of Model, Net Sentiment). Framework & Tool Building: Drive the creation of internal tools, data-driven testing pipelines, and validation frameworks capable of handling non-deterministic AI outputs and large-scale information ingestion. System Integrity & Governance: Establish risk-adaptive guardrails and checkpoints to ensure data precision, compliance (PII), and logical reasoning across complex, multi-step agentic workflows. Lead & Orchestrate Evaluation Frameworks: Define, architect, and oversee validation processes for application layers, complex API workflows, and the data transformation engine. Define Brand Scoring Criteria: Establish the mathematical and logical rules that operate over massive data volumes to audit whether brand evaluations match underlying raw metrics reliably. Direct "Golden" Test Set Curation: Lead the strategy for compiling, synthesizing, and maintaining massive baseline validation datasets to test edge cases, user intent, and model drift. Drive Adversarial & Ambiguity Testing: Design "red-teaming" scenarios and boundary testing to evaluate how gracefully the system handles prompt variations, conversational drift, and highly ambiguous data inputs. Tooling Innovation: Collaborate with engineering to build proprietary testing tools or automation scripts that streamline the ingestion and analysis of high-volume data for QA purposes. Defect Profiling & Root-Cause Strategy: Move beyond simple bug logging; analyze trends in statistical behavioral defects and guide cross-functional teams toward systemic root-cause resolution. Platform Observability & Metrics: Monitor cloud logging and data observability dashboards to track execution latency, data drift, and accuracy trends, transforming insights into platform optimizations. What we're looking for Experience: 8+ years of proven experience in QA Engineering, Systems Analysis, or Software Testing, with at least 2+ years in a Lead or Strategic role managing complex, data-heavy digital platforms. Leadership in Ambiguity: Proven "cancha" (battle-tested experience) leading QA initiatives in fast-paced environments where requirements are not fully defined, showing a high capacity to resolve ambiguity and establish order. Data & Statistical Mindset: Strong data-driven background. Exceptional capability to shift away from purely binary (pass/fail) testing toward evaluating data trends, composite scoring, and statistical outcomes. High-Volume Data Literacy: Comfortable working with big data contexts—understanding how information flows from raw ingestion (data lakes) through transformation layers to user-facing dashboards. Core QA Mastery: Deep expertise in traditional and modern software testing methodologies, Agile environments, risk management, and project tracking ecosystems (such as JIRA). Scripting & Data Querying: Practical experience with Python, JavaScript, or SQL to query databases, manipulate data structures, and trigger/build automated evaluation pipelines. Conceptual AI Fluency: Deep operational awareness of LLM behaviors, agentic workflows, and typical AI challenges (hallucinations, context limits, instruction adherence). Preferred Skills and Qualifications AI Tooling & Observability: Experience with LLM evaluation frameworks (e.g., Ragas, LangSmith, TruLens) or prompt engineering playgrounds. Advanced Data Infrastructure: Familiarity with cloud data warehouses and analytics platforms (e.g., BigQuery, Databricks, Google Cloud Platform components). Custom Tool Development: Experience building internal scripts or lightweight tools specifically designed to automate QA data validation. If the selected candidate holds a degree in Engineering or a related profession, they must present their professional license. To verify your degree requires this license, please visit www.copnia.gov.co and https://www.consejoprofesional.org.co/ . About Huge Huge is a design and technology company. We create products and experiences that grow the world’s most ambitious brands. We do this by designing experiences for people, not users, and uncovering new sources of growth by leveraging our creative talent, our proprietary platform LIVE and unlocking the advantages brought to us by emerging technologies. We believe all experiences should be intelligent, shoppable and unique to every brand. Huge’s nearly 1,000 thinkers, tinkerers, makers and creators, have been problem-solving across North America, Europe, and Latin America for over 25 years. Interested? You’ll find more information at www.hugeinc.com . Huge is committed to creating an inclusive employee experience for all. Regardless of race, gender, religion, sexual orientation, age, disability, or if you’re parenting the next generation of innovators, we firmly believe th