Amazon's AGI Data Services organization is looking for a Senior Manager of Product Management - Technical for its Data Quality business unit. The purpose of the role is to establish and drive the evolution of the set of technology used to accurately measure and improve data quality of our human-in-the-loop processes. Come be part of an initiative that is mission-critical to building and improving the just-launched Amazon Nova foundational models!

Today, AGI Data Services produces tens of millions of units of human-created data that is used to train and fine-tune large language models. These data must be evaluated based on whether they were generated following the conventions defined by the workflow, on dimensions such as complexity, correctness, helpfulness and harmfulness. Filtering out high quality data from low quality is of paramount importance, as it ensures that we maximize our chances of successfully improvement performance of the model under test.

To achieve this objective, the Data Quality organization is responsible for auditing individual units of data, as well as building an automated system using a large language model to predict and filter high quality units of data from low. The software that we design to solve these problems must be highly available and scalable given the number of quality auditors and teams that rely on these capabilities on a daily basis. The stakeholders for these products are varied in their skill sets and responsibilities throughout the organization and company, ranging from highly-technical Applied Scientists to the Device Associates who need highly-personalized feedback to improve their daily quality.