
Data Engineer
2 weeks ago
Systems Limited is looking for a Senior Data Engineer with expertise in PySpark, Airflow, and Databricks to join our growing team.
Responsibilities:
- Design, build, and maintain scalable data pipelines using PySpark for ETL/ELT processing, ensuring data is ingested, transformed, and loaded efficiently across multiple platforms.
- Use Apache Airflow to automate, schedule, and monitor complex data workflows, ensuring reliable and timely data processing.
- Leverage Databricks for building and deploying production-grade data pipelines, running data processing jobs, and collaborating on data projects with data scientists and analysts.
- Develop and optimize data pipelines and storage solutions on cloud platforms such as AWS, Azure, or Google Cloud. Experience with S3, Redshift, Azure Data Lake, BigQuery, or similar services is a plus.
- Monitor and optimize the performance of data pipelines, troubleshoot bottlenecks, and apply best practices for efficient data processing and storage.
- Work closely with data scientists, analysts, and business teams to understand data requirements and ensure data infrastructure supports business objectives.
- Implement data validation, quality checks, and error handling mechanisms to ensure high-quality and accurate data for analytics and decision-making.
- Lead junior data engineers, providing technical guidance and mentorship. Promote best practices in data engineering, testing, and automation.
- Maintain clear and comprehensive documentation of data pipelines, architecture, and processes. Provide regular updates on the progress of data engineering tasks.
Requirements:
- Bachelor's or Master's degree in Computer Science, Engineering, Data Science, or a related field.
- 8+ years of experience in Data Engineering, with a proven track record of designing, implementing, and optimizing data pipelines.
- Expertise in PySpark for large-scale data processing and transformation.
- Extensive experience with Apache Airflow for managing and orchestrating complex data workflows.
- Strong hands-on experience with Databricks, including building notebooks, clusters, and end-to-end pipelines.
- Proficiency in SQL and working knowledge of relational databases (e.g., PostgreSQL, MySQL, Redshift, etc.).
- Cloud platform experience (AWS, Azure, GCP), with proficiency in cloud-based data storage and computing services like S3, BigQuery, Azure Data Lake, or Redshift.
- Knowledge of data warehousing concepts and experience with modern data warehouses (e.g., Snowflake, Redshift, BigQuery).
- Strong experience with ETL/ELT processes and data pipeline orchestration.
- Familiarity with version control systems such as Git.
- Experience working in Agile development teams, following Scrum or Kanban methodologies.
Location: Lahore, Karachi, Islamabad, Multan, and Faisalabad
Seniority levelMid-Senior level
Employment typeFull-time
Job functionInformation Technology, Engineering, and Project Management
IndustriesIT Services and IT Consulting, Oil and Gas, and Software Development
#J-18808-Ljbffr-
Enterprise Data Engineer
7 days ago
Lahore, Punjab, Pakistan Data Pilot Full timeJob SummaryWe are seeking an experienced Associate Data Architect to join our team at Data Pilot. As a key member of our data architecture team, you will be responsible for designing and implementing robust data architectures, embedding data governance controls, and contributing to the development of next-generation data platforms.You will work closely with...
-
Scalable Data Architect
7 days ago
Lahore, Punjab, Pakistan Data Pilot Full timeAbout the RoleAs an Associate Data Architect at Data Pilot, you will play a critical role in shaping the future of our data architecture landscape. You will be responsible for designing and implementing scalable, efficient, and well-governed data architectures that drive advanced data engineering, analytics, and AI initiatives.You will work closely with...
-
Data Architecture Expert
7 days ago
Lahore, Punjab, Pakistan Data Pilot Full timeJob DescriptionData Pilot is a pioneering data product development and consulting services company, dedicated to empowering businesses with expert engineering and business acumen. As an Associate Data Architect, you will play a pivotal role in shaping scalable, efficient, and well-governed data architectures that drive advanced data engineering, analytics,...
-
Associate Data Architect
2 weeks ago
Lahore, Punjab, Pakistan Data Pilot Full timeAssociate Data Architect - Analytics & AICompany Overview:At Data Pilot, we're more than just a data product development and consulting services company – we're your strategic partner on the journey from data to decisions. Our mission is simple: to help businesses unlock the full potential of their data assets and maximize business value through our expert...
-
Associate Data Architect
1 week ago
Lahore, Punjab, Pakistan Data Pilot Full timeAt Data Pilot, we're more than just a data product development and consulting services company – we're your strategic partner on the journey from data to decisions. Our mission is simple: to help businesses unlock the full potential of their data assets and maximize business value through our expert engineering and business acumen.In the dynamic landscape...
-
Data Engineer
14 hours ago
Lahore, Punjab, Pakistan Strategic Systems International Full timeJOB TITLE: Data EngineerDEPARTMENT: Software DevelopmentREPORTS To: Team Lead / Project ManagerPURPOSE:The Data Engineer will be responsible for supporting data engineering and cloud-related tasks, ensuring efficient data processing and troubleshooting in production environments. This role is ideal for individuals with a strong problem-solving mindset, a...
-
Data Engineer
2 weeks ago
Lahore, Punjab, Pakistan Avanceon Limited Full timeJoin to apply for the Data Engineer role at Avanceon Middle East & South Asia.About Us?Avanceon is a multinational corporation which specializes in automation, control systems integration, energy management solutions, and support services. With offices in Dubai, Qatar, Saudi Arabia, and Pakistan, it has established a significant market presence. Over its...
-
Data Engineer
4 days ago
Lahore, Punjab, Pakistan Tkxel Full timeTkxel is a leading software development company located in Reston, Virginia. We are committed to developing innovative software solutions for leading enterprises in the world, helping them grow their businesses using the latest technology solutions.Job DescriptionWe are seeking a skilled and experienced Data Engineer with expertise in PySpark and Azure Data...
-
Data Engineer
4 days ago
Lahore, Punjab, Pakistan Avanceon Middle East & South Asia Full timeCompany OverviewAvanceon Middle East & South Asia is a leading provider of technology solutions, committed to delivering exceptional services to our clients. Our team of experts works tirelessly to stay ahead of the curve in the ever-evolving world of data storage and processing.Job DescriptionAs a Senior Big Data Engineer at Avanceon Middle East & South...
-
Lahore, Punjab, Pakistan Data Magnetics Full timeJob OverviewData Magnetics Technologies is a Software Solution services company offering transformative solutions across social and support services, finance, and technology sectors.We require an enthusiastic Junior Software Engineer to join our Lahore office, focusing on developing innovative web/desktop applications using .NET Web/Desktop Applications and...
-
Data Engineer
7 days ago
Lahore, Punjab, Pakistan Digifloat Full timeJob descriptionDigifloat is looking for a passionate, motivated, eager to learn data enthusiast.Responsibilities:Turn raw data into trustworthy analytical datasets, that decrease time to market towards insights and reduce the cost of data consumption.Manage these datasets (and the related data pipelines) as a product with all the responsibilities that come...
-
Data Engineer
7 days ago
Lahore, Punjab, Pakistan Devsinc Full timeDesign, implement, and maintain data pipelines to ensure seamless flow of data from various sources to storage systemsWork with large, complex datasets to ensure high performance, availability, and scalability of data systemsCollaborate with data scientists, analysts, and other engineering teams to deliver insights and business value from structured and...
-
Data Engineering Expert
12 hours ago
Lahore, Punjab, Pakistan Strategic Systems International Full timeJob Title: Data Engineering ExpertAbout Us:We are Strategic Systems International, a leading company in software development.Description:A Data Engineering Expert will be responsible for supporting data engineering and cloud-related tasks, ensuring efficient data processing and troubleshooting in production environments. This role is ideal for individuals...
-
Data Engineer Specialist
3 days ago
Lahore, Punjab, Pakistan Tkxel Full timeJob Overview:Tkxel, a leading software development company, seeks a highly skilled Data Engineer to join our team. As a key member of our data engineering group, you will be responsible for designing and developing scalable data pipelines using PySpark and Azure Data Factory.Key Responsibilities:Design and develop efficient data workflows that meet business...
-
Senior Data Engineer
2 weeks ago
Lahore, Punjab, Pakistan Tkxel Full timeTkxel is a leading software development company located in Reston, Virginia. We are committed to developing innovative software solutions for leading enterprises in the world, helping them grow their businesses using the latest technology solutions.Job DescriptionWe are seeking an experienced Data Engineer to design, develop, and optimize our client data...
-
Data Engineering Expert
6 days ago
Lahore, Punjab, Pakistan Prime System Solutions Full timeJob DescriptionAt Prime System Solutions, we are seeking a highly skilled Data Engineer to join our team. As a Data Engineer, you will be responsible for designing, developing, and maintaining scalable and reliable data pipelines using Azure Data Factory, Data Lake, and other Microsoft tools.The ideal candidate will have 5+ years of experience as a Data...
-
Data Engineering Leader
2 days ago
Lahore, Punjab, Pakistan kisaangroup Full timeAbout UsKisaan Group is a catalyst for change, empowering rural communities to drive Pakistan's economy forward.We address critical challenges in our rural economy and unlock opportunities for sustainable growth through data-driven, research-backed projects.Our focus is on creating a lasting impact at the grassroots level, building resilient communities and...
-
Data Engineer
2 weeks ago
Lahore, Punjab, Pakistan Burq, Inc. Full timeAbout BurqBurq started with an ambitious mission: how can we turn the complex process of offering delivery into a simple turnkey solution.We started with building the largest network of delivery networks, partnering with some of the biggest delivery companies. We then made it extremely easy for businesses to plug into our network and start offering delivery...
-
Senior Data Engineer
2 days ago
Lahore, Punjab, Pakistan kisaangroup Full timeExpert Talent Sourcer | Building Strong Candidate Relationships | Data-Driven Recruitment Strategies | Emotional IntelligenceWaseela is a catalyst for change, empowering rural communities to drive Pakistan's economy forward. Through data-driven, research-backed projects, we address critical challenges in our rural economy and unlock opportunities for...
-
Big Data Engineer
1 week ago
Lahore, Punjab, Pakistan Arbisoft Full timeSoftware Engineering - Arbisoft, Lahore, Islamabad, KarachiWe are looking for a skilled Big Data Engineer with 5+ years of experience in building scalable data solutions. The ideal candidate will have strong coding skills (preferably in Python or Java) and experience in managing the end-to-end development lifecycle—from development to deployment in...