
Senior Data Engineer
2 weeks ago
We are seeking an experienced Data Engineer to design, develop, and optimize our client data infrastructure. This role requires deep expertise in cloud technologies (primarily Azure with AWS as a plus) and data engineering best practices, with additional experience in Apache Spark and Databricks for large-scale data processing. The Data Engineer will work closely with data scientists, analysts, and other stakeholders to create scalable and efficient data systems that support advanced analytics and business intelligence. Additionally, this role involves mentoring junior engineers and driving technical innovation within the data engineering team.
Key Responsibilities:
- Collaborate with Solution Architects: Work with Big Data Solution Architects to design, prototype, implement, and optimize data ingestion pipelines, ensuring effective data sharing across business systems.
- ETL/ELT Pipeline Development: Build and optimize ETL/ELT pipelines and analytics solutions using a combination of cloud-based technologies, with an emphasis on Apache Spark and Databricks for large-scale data processing.
- Data Processing with Spark: Leverage Apache Spark for distributed data processing, data transformation, and analytics at scale. Experience with Databricks for optimized Spark execution is highly desirable.
- Production-Ready Solutions: Ensure data architecture, code, and processes meet operational, security, and compliance standards, making solutions production-ready in cloud environments.
- Project Support & Delivery: Actively participate in project and production delivery meetings, providing technical expertise to resolve issues quickly and ensure successful project execution.
- Database Management: Manage both SQL (e.g., PostgreSQL, MySQL) and NoSQL (e.g., DynamoDB, MongoDB) databases, ensuring data is efficiently stored, retrieved, and queried.
- Real-Time Data Processing: Implement and maintain real-time data streaming solutions using tools such as Apache Kafka, AWS Kinesis, or other technologies for low-latency data processing.
- Cloud Monitoring & Automation: Use monitoring and automation tools (e.g., AWS CloudWatch, Azure Monitor) to ensure efficient use of cloud resources and optimize data pipelines.
- Data Governance & Security: Implement best practices for data governance, security, and compliance, including data encryption, access controls, and audit trails to meet regulatory standards.
- Collaboration with Stakeholders: Work closely with data scientists, analysts, and business teams to align data infrastructure with strategic business objectives and goals.
- Documentation: Maintain clear and detailed documentation of data models, pipeline processes, and system architectures to support collaboration and troubleshooting.
Required Skills & Qualifications:
- 5+ years of experience as a Data Engineer, with strong expertise in cloud-based data warehousing, ETL pipelines, and large-scale data processing.
- Proficiency with cloud technologies, with experience in platforms like Azure or AWS.
- Hands-on experience with Apache Spark for distributed data processing and transformation. Experience with Databricks is highly desirable.
- Strong SQL skills and experience with relational databases (e.g., PostgreSQL, MySQL) as well as NoSQL databases (e.g., MongoDB, DynamoDB).
- Proficient in Python for data processing, automation tasks, and building data workflows.
- Experience with PySpark for large-scale data engineering, particularly in Spark clusters or Databricks.
- Experience in designing and optimizing data warehouse architectures, ensuring optimal query performance in large-scale environments.
- A strong understanding of data governance, security, and compliance best practices, including encryption, access control, and data privacy.
Preferred Qualifications:
- Bachelor's degree in Computer Science, Engineering, or a related field.
- Certifications in Data Engineering from cloud providers (e.g., AWS Certified Big Data - Specialty, Microsoft Certified: Azure Data Engineer Associate) are a plus.
- Experience with advanced data engineering tools and platforms such as Databricks, Apache Spark, or similar distributed computing technologies.
-
Associate Data Architect
2 weeks ago
Lahore, Punjab, Pakistan Data Pilot Full timeAssociate Data Architect - Analytics & AICompany Overview:At Data Pilot, we're more than just a data product development and consulting services company – we're your strategic partner on the journey from data to decisions. Our mission is simple: to help businesses unlock the full potential of their data assets and maximize business value through our expert...
-
Associate Data Architect
1 week ago
Lahore, Punjab, Pakistan Data Pilot Full timeAt Data Pilot, we're more than just a data product development and consulting services company – we're your strategic partner on the journey from data to decisions. Our mission is simple: to help businesses unlock the full potential of their data assets and maximize business value through our expert engineering and business acumen.In the dynamic landscape...
-
Senior Data Engineer
3 days ago
Lahore, Punjab, Pakistan kisaangroup Full timeExpert Talent Sourcer | Building Strong Candidate Relationships | Data-Driven Recruitment Strategies | Emotional IntelligenceWaseela is a catalyst for change, empowering rural communities to drive Pakistan's economy forward. Through data-driven, research-backed projects, we address critical challenges in our rural economy and unlock opportunities for...
-
Senior Data Engineering Specialist
5 days ago
Lahore, Punjab, Pakistan NorthBay Solutions LLC Full timeAbout the RoleNorthBay Solutions LLC is seeking a seasoned Senior Big Data Engineer to lead our data engineering efforts.This role involves designing and developing large-scale ETL pipelines in AWS-based environments, as well as collaborating with the team to develop cutting-edge Big Data and Cloud technologies.Key Responsibilities:Design and develop complex...
-
Senior Big Data Engineer
7 days ago
Lahore, Punjab, Pakistan Avanceon Limited Full timeJoin to apply for the Senior Big Data Engineer (Azure) role at Avanceon Middle East & South Asia.As a Big Data engineer, you will be responsible for data-related implementation tasks that include provisioning of data storage services, ingesting, streaming and batch data, transforming data, implementing security requirements, implementing data retention...
-
Data Engineer
2 weeks ago
Lahore, Punjab, Pakistan Systems Limited Full timeSystems Limited is looking for a Senior Data Engineer with expertise in PySpark, Airflow, and Databricks to join our growing team.Responsibilities:Design, build, and maintain scalable data pipelines using PySpark for ETL/ELT processing, ensuring data is ingested, transformed, and loaded efficiently across multiple platforms.Use Apache Airflow to automate,...
-
Senior Cloud Data Engineer
23 minutes ago
Lahore, Punjab, Pakistan Tkxel LLC Full timeAbout UsTkxel LLC is a renowned software development company located in Reston, Virginia. We specialize in delivering cutting-edge software solutions that cater to the diverse needs of leading enterprises worldwide.Job SummaryWe are seeking an accomplished Senior Data Engineer to join our talented team. As a critical component of our data infrastructure...
-
Data Engineer
2 weeks ago
Lahore, Punjab, Pakistan Avanceon Limited Full timeJoin to apply for the Data Engineer role at Avanceon Middle East & South Asia.About Us?Avanceon is a multinational corporation which specializes in automation, control systems integration, energy management solutions, and support services. With offices in Dubai, Qatar, Saudi Arabia, and Pakistan, it has established a significant market presence. Over its...
-
Senior Big Data Engineer
5 days ago
Lahore, Punjab, Pakistan NorthBay Solutions LLC Full timeWe are hiring for Senior Big Data Engineer with solid experience in data engineering and a passion for Big Data. The ideal candidate will have experience of over 4 to 6+ years specifically in Data Engineering.You will have the chance to join a global, multi-cultural team of Data Engineers, participate in a large variety of exciting and challenging projects,...
-
Data Engineer
5 days ago
Lahore, Punjab, Pakistan Avanceon Middle East & South Asia Full timeCompany OverviewAvanceon Middle East & South Asia is a leading provider of technology solutions, committed to delivering exceptional services to our clients. Our team of experts works tirelessly to stay ahead of the curve in the ever-evolving world of data storage and processing.Job DescriptionAs a Senior Big Data Engineer at Avanceon Middle East & South...
-
Data Engineer
1 week ago
Lahore, Punjab, Pakistan Devsinc Full timeDesign, implement, and maintain data pipelines to ensure seamless flow of data from various sources to storage systemsWork with large, complex datasets to ensure high performance, availability, and scalability of data systemsCollaborate with data scientists, analysts, and other engineering teams to deliver insights and business value from structured and...
-
Senior Data Engineer
7 days ago
Lahore, Punjab, Pakistan Arbisoft Full timeWe are seeking a motivated Data Engineer to join our dynamic team. The ideal candidate has hands-on experience with Python, SQL, and AWS data services. This role involves designing, maintaining, and optimizing data pipelines, supporting ETL processes, and ensuring the integrity of our data infrastructure. It's an excellent opportunity for someone passionate...
-
Senior Data Engineering Position
4 days ago
Lahore, Punjab, Pakistan Tkxel Full timeData Engineer Role:We are seeking a seasoned Data Engineer with expertise in PySpark and Azure Data Factory to drive the development of our data infrastructure. As a key contributor, you will design, develop, and maintain scalable data pipelines to support business growth and innovation.Responsibilities:Develop efficient data workflows using PySpark and...
-
Data Engineering Team Lead
5 days ago
Lahore, Punjab, Pakistan NorthBay Solutions LLC Full timeJob DetailsNorthBay Solutions LLC is seeking a highly skilled Senior Big Data Engineer to join our team.The ideal candidate will have experience in data engineering and a passion for Big Data, with a strong understanding of the ETL cycle and distributed processing technologies.Key Responsibilities:Design and develop complex ETL jobs and pipelines in...
-
Data Engineering Leader
2 days ago
Lahore, Punjab, Pakistan kisaangroup Full timeAbout UsKisaan Group is a catalyst for change, empowering rural communities to drive Pakistan's economy forward.We address critical challenges in our rural economy and unlock opportunities for sustainable growth through data-driven, research-backed projects.Our focus is on creating a lasting impact at the grassroots level, building resilient communities and...
-
Senior Software Engineer
2 days ago
Lahore, Punjab, Pakistan kisaangroup Full timeAbout Our CompanyKisaan Group aims to create a lasting impact at the grassroots level, building resilient communities and fostering a prosperous Pakistan through data-driven initiatives.We empower rural communities in Pakistan by addressing critical challenges in our rural economy and unlocking opportunities for sustainable growth.We focus on developing...
-
Senior Data Architect
10 hours ago
Lahore, Punjab, Pakistan LeapSofts Full timeOur CompanyWelcome to LeapSofts, a tech company specializing in cutting-edge AI, web, and software development solutions. Our team is dedicated to delivering tailor-made strategies for businesses to thrive in the digital landscape.What you'll doAs a Data Architect in Marketing Technology, you will play a pivotal role in designing and implementing scalable...
-
Senior Data Engineering Professional
6 days ago
Lahore, Punjab, Pakistan Confiz Full timeWe are seeking a highly skilled data engineer to expand and optimize our big data pipelines using Azure services.About You:You have a strong understanding of data engineering concepts, including data warehousing, publishing, and visualization.You have experience in designing, prototyping, implementing, and optimizing data ingestion pipelines.You have...
-
Big Data Engineer Lead
5 days ago
Lahore, Punjab, Pakistan NorthBay Solutions LLC Full timeJob DescriptionWe are seeking a highly skilled Senior Big Data Engineer to join our team at NorthBay Solutions LLC.The ideal candidate will have experience in data engineering and a passion for Big Data, with a strong understanding of the ETL cycle and distributed processing technologies.Key Responsibilities:Design and develop Big Data solutions using...
-
Data Engineering Leadership Position
2 days ago
Lahore, Punjab, Pakistan FINCA Impact Finance Full timeSenior Data Engineer Job DescriptionWe are seeking a highly skilled Senior Data Engineer to join our team at FINCA Impact Finance, Pakistan.About the RoleThe successful candidate will be responsible for designing, developing, and implementing database systems, tools, and applications using the Microsoft SQL Server platform. This includes development and...