Senior Cloud Data Architect Position

Job Title: Data Architect

As a seasoned professional, you will lead the architectural strategy and execution for our real-time and batch data platforms. This involves ensuring scalability, reliability, and performance across a modern cloud-native ecosystem.

Key Responsibilities:

• Define and evolve the data architecture roadmap to ensure alignment with business goals and scalability of data platforms.
• Architect real-time streaming and batch data pipelines using Kafka, Apache Flink, and Spark Structured Streaming.
• Lead the design of lakehouse architectures leveraging technologies such as Apache Spark (PySpark), Delta Lake, Iceberg, and Parquet.
• Establish governance, security, and compliance frameworks across data platforms and pipelines.
• Set standards for data orchestration using tools like Apache Airflow and Azure Data Factory.
• Guide the containerization strategy with Docker, Kubernetes, and Helm for scalable deployment of data applications.
• Architect and optimize analytics engines such as Trino or Presto for high-performance query execution.
• Define and implement observability frameworks to monitor SLAs, lineage, data quality, and pipeline health.
• Collaborate with DevOps on infrastructure-as-code (IaC) using Terraform and CI/CD automation through Azure DevOps.
• Establish and advocate best practices for coding standards, modular development, and testing in Python and SQL.
• Collaborate cross-functionally with product owners, data scientists, platform teams, and engineering leads.
• Lead architecture reviews, POCs, and technology evaluations to guide technology adoption and innovation.
• Mentor and guide senior and mid-level data engineers across solutioning, technical decisions, and design patterns.

Required Skills & Experience:

• Bachelor's or Master's degree in Computer Science, Information Systems, or a related discipline.
• 12+ years of experience in data engineering and architecture, with at least 3+ years in a lead/architect role.
• Must have the ability to take things to completion overcoming challenges like unavailability of stakeholders or information.
• Expert-level knowledge in Python, SQL, and scalable distributed systems.
• Deep expertise in streaming architectures with Kafka, Flink, and Spark Streaming.
• Strong experience designing and implementing batch data workflows using Apache Spark (PySpark).
• In-depth knowledge of cloud data platforms, particularly Azure.
• Extensive experience with data orchestration tools like Airflow and Azure Data Factory.
• Proven ability to design and implement data lakehouse architectures with formats like Parquet, Delta Lake, and Iceberg.
• Proficiency in containerization (Docker), orchestration (Kubernetes), and deployment via Helm.
• Solid experience implementing observability, lineage, and data quality frameworks, such as Great Expectations.
• Strong background in infrastructure as code (Terraform) and CI/CD integrations.
• Deep understanding of data governance, metadata management, and data privacy standards.
• Excellent communication skills with the ability to influence stakeholders and present architectural strategies to leadership.
• Experience working in agile development environments and fostering team-wide engineering excellence.
• Experience with large-scale API architecture and scalable data access layers is a plus.
• Familiarity with AI/ML-integrated data systems or tools is a bonus.

Benefits:

Our company offers an amazing environment where people work together with cutting-edge technologies while contributing success to the company and themselves.

Others:

Confiz has a vibrant culture of learning and collaboration, making the workplace fun.