Lead Data Engineer

About Us: Prometric is a leading provider of technology-enabled testing and assessment solutions to many of the world’s most recognized licensing and certification organizations, academic institutions, and government agencies. We support more than 7 million test takers annually at our testing locations in 180 countries around the world. With over three decades of experience working with clients of all sizes across a multitude of industry sectors, our mission is to design and deliver the highest quality and most innovative testing solutions anytime, anywhere.

Job Title:                     Lead Data Engineer                           

Reports To:                 Director, Software Development

Department:              Technology & Product           

Location:                     Remote 

Prometric is a global leader in test development, test delivery, and candidate services for academic, professional, government, and corporate clients. We are looking for a highly skilled and motivated technology lead to join our team to work to on innovative technologies in the Data Engineering space.

Main Duties & Responsibilities:

• Design, develop, and support scalable data processing pipelines using Apache Spark and Java/Scala.
• Lead a talented team and make a significant impact on our data engineering capabilities.
• Implement and manage workflow orchestration with AirFlow for efficient data processing.
• Proficiently use SQL for querying and data manipulation tasks.
• Collaborate with cross-functional teams to gather requirements and ensure alignment with data engineering solutions.

Essential Criteria:

• Bachelor’s degree in computer science or another relevant discipline, and a minimum of five years relevant experience in Data Engineering.
• Solid experience with Apache Spark for large-scale data processing.
• Proficiency in Java or Scala programming languages.
• Strong knowledge of AirFlow for workflow orchestration.
• Proficient in SQL for data querying and manipulation.

Desirable Criteria:

• Familiarity with Avro and Schema Registry for efficient data serialization.
• Experience with containerization technologies such as Docker and orchestration tools like Kubernetes (K8S).
• Hands-on experience with cloud-based data engineering services, preferably Azure Data Factory or AWS Glue.
• Understanding of release management processes.
• Familiarity with version control systems such as Git and experience with release pipelines using Azure DevOps or similar tools.
• Ability to effectively work with teams across different geographical locations, including Ireland, the United States (Eastern Time Zone), and India.

If you are a passionate and experienced Tech Lead with a strong background in data engineering, we encourage you to apply. Join us to help shape the future of our data solutions at Prometric.

Prometric is an Equal Employment/Affirmative Action employer. We do not discriminate in hiring based on sex, gender identity, sexual orientation, race, color, religious creed, national origin, physical or mental disability, protected Veteran status, or any other characteristic protected by federal, state, and local law.