Chief Data Transformation Specialist

The goal of this role is to design and implement a robust data transformation pipeline that ensures the accuracy and consistency of our data across all platforms.

Key Responsibilities:

• Develop and execute PySpark and SQL scripts to validate data pipelines, transformations, and integrations.
• Design and run tests for data validation, storage, and retrieval using Azure services like Data Lake, Synapse, and Data Factory, adhering to industry standards.
• Continuously enhance automated tests as new features are developed, ensuring data accuracy, completeness, and consistency across the platform.
• Formulate and maintain test strategies—including smoke, performance, functional, and regression testing—to ensure data processing and ETL jobs meet requirements.
• Collaborate with development teams to assess changes in data workflows and update test cases to preserve data integrity.
• Share knowledge and best practices by collaborating with business analysts and technology teams to document testing processes and findings.
• Communicate testing progress effectively with stakeholders, highlighting issues or blockers, and ensuring alignment with business objectives.
• Maintain a comprehensive understanding of the Azure Data Lake platform's data landscape to ensure thorough testing coverage.

For this role, we require:

• Bachelor's degree in computer science or related discipline.
• 3-6 years of hands-on experience in Data Engineering particularly in Data Lake environments on Azure's cloud platform.
• Proficient in Azure Data Factory, Azure Synapse Analytics, and Databricks for big data processing and scaled data quality checks.
• Proficiency in SQL, capable of writing and optimizing both simple and complex queries for data validation and testing purposes.
• Proficient in PySpark, with experience in data manipulation and transformation, and a demonstrated ability to write and execute test scripts for data processing and validation.
• Hands-on experience with Functional & system integration testing in big data environments, ensuring seamless data flow and accuracy across multiple systems.
• Knowledge and ability to design and execute test cases in a behaviour-driven development environment.
• Fluency in Agile methodologies, with active participation in Scrum ceremonies and a strong understanding of Agile principles.
• Familiarity with tools like Jira, including experience with X-Ray or Jira Zephyr for defect management and test case management.
• Proven experience working on high-traffic and large-scale software products, ensuring data quality, reliability, and performance under demanding conditions.
• Identify and resolve issues in complex Big Data environments, ensuring data accuracy and system efficiency.
• Initiate and manage tasks, delivering high-quality results within deadlines in a dynamic setting.
• Confidently challenge existing assumptions and processes to enhance data engineering and QA practices.
• Maintain organized testing schedules and documentation, ensuring thorough coverage of all scenarios.
• Exhibit strong interpersonal skills, working effectively in cross-functional, multicultural teams with minimal supervision.
• Comfortable in Agile and Scrum environments, proficient with tools like Jira, and adaptable to changing priorities.
• Manage multiple projects simultaneously, demonstrating strong planning and organizational skills to ensure timely task completion.

We offer a collaborative and innovative work environment that fosters growth and learning. Our team members have access to cutting-edge technologies and contribute to the success of the company while enhancing their own skills and expertise.

Our company is certified with ISO 9001:2015 (QMS), ISO 27001:2022 (ISMS), ISO 20000-1:2018 (ITSM), and ISO 14001:2015 (EMS) standards, reflecting our commitment to quality, security, and sustainability.