
Big Data Developer
3 weeks ago
Direct message the job poster from Datamatics Technologies
Talent Acquisition Lead | Global Technical Recruitment | Technical Recruitment | EMEA & MENAJob Summary:
We are looking for a highly skilled Big Data Engineer to spearhead the development of a scalable Big Data Platform. The ideal candidate will have extensive experience with Kafka, Flink, Spark, Python, and Talend, along with expertise in Apache Ranger, Prometheus, and strong SQL skills. This role involves designing and implementing data pipelines, ensuring security, monitoring system performance, and collaborating with cross-functional teams to deliver high-performance data solutions.
Key Responsibilities:
- Design, develop, and optimize a Big Data Platform leveraging Kafka, Flink, Spark, and Talend.
- Architect and implement real-time and batch data processing pipelines.
- Ensure secure data access and governance using Apache Ranger.
- Monitor and optimize system performance with Prometheus.
- Develop and maintain ETL workflows and data ingestion pipelines.
- Write efficient and scalable SQL queries for data transformation and analysis.
- Collaborate with Data Scientists, Analysts, and DevOps teams to align data infrastructure with business goals.
- Implement data security, compliance, and access control policies.
- Troubleshoot and resolve issues related to data processing, storage, and system performance.
Required Skills & Qualifications:
- 5+ years of experience in Big Data Engineering.
- Hands-on expertise in Kafka, Flink, Spark, Python, and Talend.
- Strong proficiency in SQL, with experience in query optimization and performance tuning.
- Familiarity with Apache Ranger for data security and access control.
- Familiarity with Prometheus for monitoring and performance optimization.
- Deep understanding of distributed computing, data partitioning, and pipeline orchestration.
- Experience with cloud-based big data solutions (AWS, Azure, or GCP) is a plus.
- Strong problem-solving and analytical skills.
- Excellent communication and leadership abilities.
Preferred Qualifications:
- Experience with Docker, Kubernetes, and containerized big data deployments.
- Knowledge of workflow orchestration tools like Airflow.
- Hands-on experience with NoSQL databases (e.g., Cassandra, HBase).
Mid-Senior level
Employment typeFull-time
Job functionEngineering and Information Technology
IndustriesIT Services and IT Consulting
#J-18808-Ljbffr-
Big Data Developer Position
4 days ago
Islamabad, Islamabad, Pakistan Datamatics Technologies Full timeJob Summary:We are seeking an experienced Big Data Engineer to lead the development of a scalable Big Data Platform. The ideal candidate will have extensive experience with Kafka, Flink, Spark, Python, and Talend, along with expertise in Apache Ranger, Prometheus, and strong SQL skills.This role involves designing and implementing data pipelines, ensuring...
-
Big Data Professional
4 days ago
Islamabad, Islamabad, Pakistan NorthBay Solutions Full timeWe are NorthBay Solutions, a leading provider of innovative data solutions. We are currently seeking a highly skilled Big Data Professional to join our team.About the Role:We are looking for an individual with extensive knowledge of big data technologies, including AWS Glue, Lambda, Step Functions, and others, to design, develop, and maintain our data...
-
Big Data Automation Specialist
4 days ago
Islamabad, Islamabad, Pakistan Confiz Full timeJob DescriptionThe purpose of this role is to have a highly skilled QA Automation Expert who can participate in day-to-day QA activities such as sprint calls and providing updates on assigned tickets. The ideal candidate will improve their knowledge in the relevant domain and create and identify the regression testing suite to eliminate manual effort from...
-
Cloud Big Data Architect
3 days ago
Islamabad, Islamabad, Pakistan Fusemachines Full timeJob SummaryAbout FusemachinesFusemachines is a leading provider of AI solutions and products to various industries. Our company was founded by Sameer Maskey, Ph.D., an Adjunct Associate Professor at Columbia University, who aims to democratize AI and leverage its global impact. With a robust presence in four countries and a team of over 400 employees, we...
-
Big Data Infrastructure Architect
3 days ago
Islamabad, Islamabad, Pakistan Fusemachines Full time**About the Role**Fusemachines is a global AI company with a presence in four countries, committed to democratizing AI and harnessing its power to transform businesses worldwide. We are currently seeking a highly skilled Sr. Data Engineer to join our team and lead the development of big data infrastructure using Microsoft Azure in the Media domain.This...
-
Big Data Testing and Automation Expert
4 days ago
Islamabad, Islamabad, Pakistan Confiz Full timeRole OverviewThis role is focused on leveraging big data technologies to improve our QA process. We are seeking a highly skilled QA Automation Expert to join our team and contribute to the success of our company.Responsibilities:Developing and automating processes for gathering expected results from data sources and comparing data results from...
-
Senior Big Data Engineer Role
4 days ago
Islamabad, Islamabad, Pakistan Datamatics Technologies Full timeAbout the Job:Datamatics Technologies is looking for a highly skilled Big Data Engineer to join our team. As a key member of our engineering team, you will be responsible for designing and implementing a scalable Big Data Platform.Responsibilities:Design, develop, and optimize Big Data Platform using Kafka, Flink, Spark, and Talend.Architect and implement...
-
Big Data Solutions Expert
3 days ago
Islamabad, Islamabad, Pakistan Fusemachines Full timeAbout the RoleData Engineering LeadWe are seeking a highly skilled Senior Data Engineer to lead, design, build, and maintain the infrastructure required for data integration, storage, processing, and analytics (BI, visualization and Advanced Analytics) using Microsoft Azure in the Media domain. The ideal candidate will possess hands-on experience with...
-
Big Data Engineer
7 days ago
Islamabad, Islamabad, Pakistan VisionX Technologies, Inc. Full timeKey QualificationsTo succeed as a Senior Data Engineer at VisionX Technologies, Inc., you should possess:A minimum of a bachelor's degree in Computer Science, Information Technology, or a related field.A deep passion for data and proven ability to create high-quality data products.At least 8+ years of experience in implementing Big Data, Data Lakehouse, and...
-
Big Data QA Automation
3 weeks ago
Islamabad, Islamabad, Pakistan Confiz Full timeThe purpose of this job is to have a QA Automation Expert who can participate in day-to-day QA activities such as sprint calls and providing updates on assigned tickets. You will improve your knowledge in the relevant domain and, with your abilities and skillset, create and identify the regression testing suite to eliminate manual effort from the existing QA...
-
Cloud Architect for Big Data Solutions
3 days ago
Islamabad, Islamabad, Pakistan Fusemachines Full time**Job Description**Fusemachines is a leading AI company that specializes in delivering state-of-the-art AI products and solutions to various industries. As a Sr. Data Engineer, you will be responsible for designing, building, and maintaining the infrastructure required for data integration, storage, processing, and analytics using Microsoft Azure in the...
-
Islamabad, Islamabad, Pakistan Confiz Full timeAbout This OpportunityWe are looking for a talented QA Automation Expert to join our team. In this role, you will be responsible for participating in day-to-day QA activities and providing updates on assigned tickets. You will also improve your knowledge in the relevant domain and create and identify the regression testing suite to eliminate manual effort...
-
Senior QA Engineer for Big Data Projects
4 days ago
Islamabad, Islamabad, Pakistan Confiz Full timeAbout the RoleWe are seeking an experienced QA Automation Expert to join our team. As a key member of our QA team, you will be responsible for participating in day-to-day QA activities and providing updates on assigned tickets. You will also improve your knowledge in the relevant domain and create and identify the regression testing suite to eliminate manual...
-
Islamabad, Islamabad, Pakistan Confiz Full timeConfiz is a leading technology company that has been at the forefront of innovation, delivering cutting-edge solutions to businesses across various industries. We are currently seeking an experienced Machine Learning Engineer with expertise in big data processing to join our team.The ideal candidate will have strong programming skills in Python or R,...
-
Cloud and Big Data Expert
1 week ago
Islamabad, Islamabad, Pakistan VisionX Technologies, Inc. Full timeVisionX Technologies, Inc. OverviewVisionX is a global provider of innovation solutions, partnering with Fortune 1000 companies to drive business growth. Our team leverages agile methodologies, technology accelerators, and Intellectual Property creation to deliver high-quality products.We have been recognized as one of the Top 10 Most Innovative Companies of...
-
Big Data Product Owner
2 weeks ago
Islamabad, Islamabad, Pakistan Confiz Full timeWe are undertaking a Data Transformation and as data becomes fundamental to what we are doing, the ability to have the right data in the right place at the right time, that is accurate and consistent is essential.This role will play a really important central role within this evolving landscape, working alongside the Technology Team and the Product Owner to...
-
Data Engineering Lead
4 days ago
Islamabad, Islamabad, Pakistan Datamatics Technologies Full timeAbout this Opportunity:We are looking for an experienced Big Data Engineer to join our team. As a key member of our engineering team, you will be responsible for designing and implementing a scalable Big Data Platform.This role involves working closely with cross-functional teams to deliver high-performance data solutions. The successful candidate will have...
-
Big Data Solutions Developer
6 days ago
Islamabad, Islamabad, Pakistan DigyCorp Full timeJob FunctionThe successful candidate will be responsible for implementing data pipelines to feed real-time and batch data to AI/ML models, supporting deployment and monitoring of machine learning models using Azure Machine Learning and MLOps best practices, integrating data from various on-premises and cloud sources, creating and optimizing Extract,...
-
Cloud Based Data Solutions Specialist
4 days ago
Islamabad, Islamabad, Pakistan Datamatics Technologies Full timeThe Role:We are seeking an experienced Big Data Engineer to design and implement a scalable Big Data Platform. The ideal candidate will have hands-on experience with Kafka, Flink, Spark, Python, and Talend, as well as expertise in Apache Ranger and Prometheus.This role involves working closely with cross-functional teams to deliver high-performance data...
-
Sr. Data Engineer
3 weeks ago
Islamabad, Islamabad, Pakistan Fusemachines Full timeGet AI-powered advice on this job and more exclusive features.About FusemachinesFusemachines is a 10+ year old AI company, dedicated to delivering state-of-the-art AI products and solutions to a diverse range of industries. Founded by Sameer Maskey, Ph.D., an Adjunct Associate Professor at Columbia University, our company is on a steadfast mission to...