Chief Data Architect

Transformative Data Engineer Role

• We are undergoing a significant data transformation to ensure accurate and consistent data is available precisely when and where it's needed.
• As a key member of our team, you will be responsible for designing and running tests for data validation, storage, and retrieval using Azure services like Data Lake, Synapse, and Data Factory.
• You will also be tasked with continuously enhancing automated tests as new features are developed, ensuring that data accuracy, completeness, and consistency are maintained across the platform.
• Additionally, you will formulate and maintain test strategies – including smoke, performance, functional, and regression testing – to ensure that data processing and ETL jobs meet requirements.
• Collaboration with development teams will be essential in assessing changes in data workflows and updating test cases to preserve data integrity.
• Sharing knowledge and best practices by collaborating with business analysts and technology teams will be crucial in documenting testing processes and findings.
• Effective communication of testing progress to stakeholders, highlighting issues or blockers, and ensuring alignment with business objectives will be vital.
• A comprehensive understanding of the Azure Data Lake platform's data landscape will be necessary to ensure thorough testing coverage.

Key Responsibilities:

• Design and run tests for data validation, storage, and retrieval using Azure services.
• Enhance automated tests as new features are developed.
• Maintain test strategies for smoke, performance, functional, and regression testing.
• Collaborate with development teams to assess changes in data workflows.
• Document testing processes and findings.

Requirements:

• Bachelor's degree in computer science or related discipline.
• 3-6 years of hands-on experience in Data Engineering particularly in Data Lake environments on Azure's cloud platform.
• Proficient in Azure Data Factory, Azure Synapse Analytics and Databricks for big data processing and scaled data quality checks.
• Proficiency in SQL, capable of writing and optimizing both simple and complex queries for data validation and testing purposes.
• Proficient in PySpark, with experience in data manipulation and transformation, and a demonstrated ability to write and execute test scripts for data processing and validation.
• Hands-on experience with Functional & system integration testing in big data environments, ensuring seamless data flow and accuracy across multiple systems.
• Knowledge and ability to design and execute test cases in a behaviour-driven development environment.
• Fluency in Agile methodologies, with active participation in Scrum ceremonies and a strong understanding of Agile principles.
• Familiarity with tools like Jira, including experience with X-Ray or Jira Zephyr for defect management and test case management.
• Proven experience working on high-traffic and large-scale software products, ensuring data quality, reliability, and performance under demanding conditions.
• Identify and resolve issues in complex Big Data environments, ensuring data accuracy and system efficiency.
• Initiate and manage tasks, delivering high-quality results within deadlines in a dynamic setting.
• Confidently challenge existing assumptions and processes to enhance data engineering and QA practices.
• Maintain organized testing schedules and documentation, ensuring thorough coverage of all scenarios.
• Exhibit strong interpersonal skills, working effectively in cross-functional, multicultural teams with minimal supervision.
• Comfortable in Agile and Scrum environments, proficient with tools like Jira, and adaptable to changing priorities.
• Manage multiple projects simultaneously, demonstrating strong planning and organizational skills to ensure timely task completion.

Why Work with Us?

Our organization values collaboration, innovation, and continuous improvement. We offer a dynamic work environment with opportunities for growth and professional development.

What We Offer:

• A competitive compensation package.
• Opportunities for career advancement and professional growth.
• A collaborative and innovative work environment.
• Professional development and training opportunities.

How to Apply:

If you are a motivated and experienced data engineer looking to take your career to the next level, please submit your application. We look forward to hearing from you