Head Data Engineer

Job Purpose: Head Data Engineer will design and maintain data pipelines of real-time and batch, creates data strategy, data architecture, best practices and who can lead the organization in making strategical improvements in the data engineering solutions. This role requires to have a knowledge of core system, retail banking systems, digital journeys, marketing systems, risk and compliance systems which are very data intensive systems across UAE, Egypt, Pakistan etc. where bank operates retail business. Our core function spans multiple platforms like Enterprise Data Lake, Azure Data Lake, Business Intelligence, Advanced Analytics and Realtime Streaming platforms. The work involves developing new requirements in both on-premises and cloud data lake's estimating the efforts, designing an effective solution, data architecture and management, change release management, resource management, handling the deliverables across the business groups in the Bank. Also, handling critical applications that influence bank's management decision making process, regulatory reporting (which can influence financial/non-financial penalties) and operational reporting. Key Responsibilities: Develops and maintains scalable data pipelines of batch and real-time to support continuing increases in data demands of volume and complexity. Leading data engineering team and ensuring best practices and quality on the deliveries. Designs and Develops Data domains across banking products and laying foundation for Data Mesh architecture. Demonstrate deep knowledge and experience in databases, Datamart's, Data Modelling, Data Governance, Data Security, BI tools, ETL Solutions. Data Orchestration and Data-Ops practices enabler for the team Collaborates with analyst, data scientists and business teams to improve data models that feed business intelligence tools, increasing data accessibility, and fostering data-driven decision making across the organization. Implements processes and systems to monitor data quality, ensuring production data is always accurate and available for key stakeholders and business processes that depend on it. Writes unit/integration tests, contributes to engineering wiki, and documents work. Experience in Automating Deployment, scheduling, monitoring, and management of datasets and pipelines Understand business requirements and identify key performance indicators (KPIs) in Data Engineering. Classification: Internal Use Design data models to transform raw data into meaningful insights. Ability to Debug Data issues in data platforms and Provide L2, L3 Production support as per SLA. Recommend improvements to enhance existing ETL and data integration processes to enable performance and overall scalability. Evaluate company's present data infrastructure/governance/architecture, identify gaps, and develop plans to integrate the current systems with the desired future model. Identify, analyze various risks, and implement risk management controls, RCSA and contingency plans on Data Platform to mitigate future risk. Reviewing, executing, and documenting Audit activities and its corrective actions on Data Platform. Ensuring Data Platform security and compliance with data privacy regulations and company policies. Implementing access controls, data masking, and encryption where necessary to safeguard sensitive information. Understands industry trends and the practical application of existing, new, and emerging technologies. Forecasting future data storage and processing requirements and planning for scalability. Identifying potential bottlenecks and devising strategies to accommodate growing data volumes. Developing and maintaining disaster recovery plans and business continuity strategies to address potential risks and minimize downtime. Communicate and effectively articulate the DWH/DL solution/roadmaps to business and technical delivery teams. Coordinate with change management approval board to get all desired approvals and ensure production deployments on timely manner. Collaborate with Incident management team to track and report on incidents, their resolutions, and any necessary follow-up actions. Interacting with Data Platform software and hardware vendors to coordinate support, updates, and service-level agreements RequirementsPrincipal Data engineer with a solid technical background and experience in data-intensive systems (> 15 years) Strong experience in Big Data, Hadoop Ecosystem, Spark Streaming, Kafka, Python, SQL, Hive, NIFI, Airflow Proficient with Azure Cloud services such as Azure Data Factory (ADF), Databricks, ADLS, Azure Synapse, Logic Apps, Azure Functions. Or similar data stack knowledge within Google/AWS cloud services Proficiency in relational SQL, Graph and NoSQL databases. Proficiency in Elastic Search and Couchbase databases In-depth skills in developing and maintaining ETL/ELT data pipelines. Experience in data modelling techniques such as Kimball star schema, 3NF, vault modelling etc. Experience in workflow management tools such as Airflow, Oozie and CI/CD tools Data streaming solution in Kafka or Confluent Kafka Hands on experience in Google Big Query, Google Analytics & Clickstream Data Model Reporting knowledge in Power BI, Tableau, Qlik etc. Sound in Data Management Fundamentals and Data Architect, Modelling, GovernanceManagement Skills: People management responsibilities encompass the tasks and activities involved in leading, guiding, and developing a team of individuals to achieve organizational goals effectively. Setting clear performance expectations, conducting regular performance reviews, providing feedback, and offering guidance on areas of improvement. Follow predefined processes and workflows to carry out specific tasks and activities accurately and in a timely manner.  Identify opportunities for process improvement and efficiency enhancement and provide feedback or suggestions to optimize workflows.  Analyze the needs, expectations, and concerns of each stakeholder. Assess their level of influence, interest, and potential power to affect the project's success. Developing a comprehensive project plan that outlines the project scope, objectives, deliverables, timelines, resource requirements, and budget. Additional Skills Desired: Bachelors/Master's degree in computer science or equivalent Excellent verbal and written communication skills and good stake holder management skills Strong Analytical thinking and ability to handle stakeholders. Confident in decision making and the ability to explain processes or choices as needed. Ability to multitask on multiple projects in various phases. Agile and waterfall Project Methodology Knowledge Program Management & Deliverable