Data Engineering Expert
3 days ago
We are seeking an experienced Data Engineer to contribute to our data engineering team.
About the RoleIn this position, you will design, build, and maintain scalable data solutions to support critical business needs. As a key member of our team, you will work closely with cross-functional teams to understand data requirements and deliver solutions that meet business objectives.
This role involves participation in an on-call rotation to ensure the reliability and performance of production systems:
- Rotation Schedule: Weekly rotation beginning Tuesday at 9:00 PM PST through Monday at 9:00 AM PST.
- Responsibilities During On-Call:
- Monitor system health and respond to alerts promptly.
- Troubleshoot and resolve incidents to minimize downtime.
- Escalate issues as needed and document resolutions for future reference.
The successful candidate will be responsible for:
- Data Platform Development: Design, develop, and maintain data pipelines and workflows on distributed data platforms such as BigQuery, Hadoop/EMR/DataProc, or Teradata.
- Cloud Integration: Build and optimize cloud-based solutions using AWS or GCP to process and store large-scale datasets.
- Workflow Orchestration: Design and manage workflows and data pipelines using Apache Airflow to ensure scalability, reliability, and maintainability.
- Containerization and Orchestration: Deploy and manage containerized applications using Kubernetes for efficient scalability and resource management.
- Event Streaming: Work with Kafka to implement reliable and scalable event streaming systems for real-time data processing.
- Programming and Automation: Write clean, efficient, and maintainable code in Python and SQL to automate data processing, transformation, and analytics tasks.
- Database Management: Design and optimize relational and non-relational databases to support high-performance querying and analytics.
- System Monitoring & Troubleshooting: Participate in the on-call rotation to monitor systems, address incidents, and ensure the reliability of production environments.
- Collaboration: Work closely with cross-functional teams to understand data requirements and deliver solutions that meet business objectives.
To succeed in this role, you must have:
- Primary Technologies: Big Query or other distributed data platform, for example, Big Data (Hadoop/EMR/DataProc), SnowFlake, Teradata, or Netezza, AWS, GCP, Kubernetes, Kafka, Python, SQL.
- Bachelor's degree in computer science, Engineering, or a related field (or equivalent work experience).
- 3+ years of experience in data engineering or related roles.
- Hands-on experience with distributed data platforms such as BigQuery, Hadoop/EMR/DataProc, Snowflake, or Teradata.
- Proficiency in Apache Airflow for building and orchestrating workflows and data pipelines.
- Proficiency in Python and SQL for data processing and analysis.
- Experience with cloud platforms like AWS or GCP, including building scalable solutions.
- Familiarity with Kubernetes for container orchestration.
- Knowledge of Kafka for event streaming and real-time data pipelines.
- Strong problem-solving skills and ability to troubleshoot complex systems.
- Excellent communication and collaboration skills to work effectively in a team environment.
While not required, experience with CI/CD pipelines for automated deployments, knowledge of data governance, security, and compliance best practices, and familiarity with DevOps practices and tools are highly desirable.
-
Principal, Data Engineer
4 weeks ago
Hyderabad City Taluka, Pakistan Warner Bros. Discovery, Inc. Full timeWelcome to Warner Bros. Discovery… the stuff dreams are made of.Who We Are…When we say, "the stuff dreams are made of," we're not just referring to the world of wizards, dragons and superheroes, or even to the wonders of Planet Earth. Behind WBD's vast portfolio of iconic content and beloved brands, are the storytellers bringing our characters to life,...
-
Solutions Engineer, Data Engineering
4 weeks ago
Hyderabad City Taluka, Pakistan Astronomer Full timeAstronomer designed Astro, an industry-leading, orchestration-first DataOps platform for data teams. Powered by Airflow, Astro accelerates building reliable data products that unlock insights, unleash AI value, and drive data-driven applications.We're a globally-distributed and rapidly growing venture-backed team of learners, innovators and collaborators....
-
Presales Engineer, Data Engineering
4 weeks ago
Hyderabad City Taluka, Pakistan Astronomer Full timeAstronomer designed Astro, an industry-leading, orchestration-first DataOps platform for data teams. Powered by Airflow, Astro accelerates building reliable data products that unlock insights, unleash AI value, and drive data-driven applications.We're a globally-distributed and rapidly growing venture-backed team of learners, innovators, and collaborators....
-
Principal Data Science and Engineering Manager
2 weeks ago
Hyderabad City Taluka, Pakistan Warner Bros. Discovery, Inc. Full timeQualifications and RequirementsTo be successful in this role, you will need:A bachelor's degree in computer science, engineering, or a related field14+ years of experience in data engineering and architectureExpert knowledge of data integration, data pipeline, and data deliveryExperience with cloud-based environments like AWSExcellent communication and...
-
Solutions Engineer
2 weeks ago
Hyderabad City Taluka, Pakistan Astronomer Full timeRole SummaryWe are seeking an accomplished Solutions Engineer to join our Global Services Delivery Team. As a key member of this team, you will be responsible for delivering expert Airflow solutions to our customers worldwide.Key ResponsibilitiesMaintain and enhance your technical expertise in Airflow and Astronomer's managed platform.Collaborate with...
-
Data Engineer
6 days ago
Hyderabad City Taluka, Pakistan beBee Careers Full timeAbout the RoleThe Data and Analytics organisation owns and delivers the Data and AI platform to equip markets with improved decision-making capabilities. The team consists of six key areas: Platform & Architecture, Data Science, Data Engineering, Data Visualisation, Data Governance, and Strategy & Operations.
-
Senior Data Platform Expert
1 day ago
Hyderabad City Taluka, Pakistan beBee Careers Full timeAbout the RoleWe are seeking an experienced Senior Data Platform Expert to join our team. In this role, you will work with global accounts to accelerate their data journey.Responsibilities:Presents data platform solutions to executives and technical contributors at prospects and customers.Works hands-on with prospects and customers to demonstrate and...
-
Data Engineering Solutions Specialist
2 weeks ago
Hyderabad City Taluka, Pakistan Astronomer Full timeAbout UsAstronomer is a pioneering organization in the data operations space, renowned for its innovative orchestration-first DataOps platform. Our mission is to empower data teams to build reliable data products that unlock insights and drive decision-making.Job DescriptionWe are seeking an accomplished Solutions Engineer to join our Global Services...
-
Data Engineering Manager
7 days ago
Hyderabad City Taluka, Pakistan beBee Careers Full timeAbout the RoleWe are seeking an experienced Data Engineering Manager to lead our analytics engineering team. This is a pivotal role in driving the data and analytics strategy across various streaming and studio data domains, including commerce, product, and content analytics.The successful candidate will have 14+ years of experience in data engineering, data...
-
Data Engineering Professional
3 days ago
Hyderabad City Taluka, Pakistan beBee Careers Full timeJob Description:We are seeking a highly skilled Data Engineering Professional to design, develop, and maintain large-scale data systems. This individual will collaborate with cross-functional teams to create efficient data pipelines, ensuring seamless integration with various tools and technologies.Main Responsibilities:Develop and implement robust data...
