
Expertise in Big Data Processing
1 week ago
Job Summary
">We are seeking an experienced Senior Data Engineer to join our team. As a key member of our organization, you will be responsible for designing, implementing, and optimizing large-scale data pipelines using PySpark, Apache Airflow, and Databricks.
Key responsibilities include:
">- ">
- Designing and maintaining scalable data pipelines using PySpark for ETL/ELT processing;">
- Using Apache Airflow to automate, schedule, and monitor complex data workflows;">
- Leveraging Databricks for building and deploying production-grade data pipelines;">
- Developing and optimizing data pipelines and storage solutions on cloud platforms like AWS, Azure, or Google Cloud;">
- Monitoring and optimizing the performance of data pipelines;">
Requirements
">To succeed in this role, you must have:
">- ">
- A Bachelor's or Master's degree in Computer Science, Engineering, Data Science, or a related field;">
- 8+ years of experience in Data Engineering with a proven track record of designing, implementing, and optimizing data pipelines;">
- Expertise in PySpark for large-scale data processing and transformation;">
- Extensive experience with Apache Airflow for managing and orchestrating complex data workflows;">
- Strong hands-on experience with Databricks, including building notebooks, clusters, and end-to-end pipelines;">
- Proficiency in SQL and working knowledge of relational databases;">
- Cloud platform experience, including proficiency in cloud-based data storage and computing services;">
- Knowledge of data warehousing concepts and experience with modern data warehouses;">
- Strong experience with ETL/ELT processes and data pipeline orchestration;">
- Familiarity with version control systems like Git;">
- Experience working in Agile development teams following Scrum or Kanban methodologies;">
-
Full Stack Developer
19 hours ago
Lahore, Punjab, Pakistan Big Data Inc Full timeOverview:Big Data Inc is a cutting-edge technology company that offers innovative solutions for big data processing and analysis. We are currently seeking an experienced Full Stack Java Developer to join our team in Lahore, Pakistan.About the Role:The incumbent will be responsible for designing, testing, and implementing large-scale enterprise applications...
-
Big Data Specialist
3 days ago
Lahore, Punjab, Pakistan Crypthon Technologies Full timeWe're looking for a detail-oriented IT Infrastructure Developer to join our team. As a key member, you will be responsible for designing and implementing scalable data solutions.Key Responsibilities:You'll work closely with cross-functional teams to define data requirements and implement efficient pipelines.Bulk up your skillset by working with cutting-edge...
-
Big Data Engineer
3 weeks ago
Lahore, Punjab, Pakistan Arbisoft Full timeSoftware Engineering - Arbisoft, Lahore, Islamabad, KarachiWe are looking for a skilled Big Data Engineer with 5+ years of experience in building scalable data solutions. The ideal candidate will have strong coding skills (preferably in Python or Java) and experience in managing the end-to-end development lifecycle—from development to deployment in...
-
Big Data Architect
3 days ago
Lahore, Punjab, Pakistan Devsinc Full timeJob Title: Senior Data SpecialistDevsinc is seeking a highly skilled Senior Data Specialist to design, implement, and maintain efficient data pipelines. This position requires expertise in data modeling, ETL frameworks, big data tools, and programming languages.Key ResponsibilitiesData Pipeline Development: Design and implement scalable and secure data...
-
Technical Product Owner
1 week ago
Lahore, Punjab, Pakistan HR POD Careers Full timeAbout the RoleWe are looking for a talented Technical Product Owner - Big Data to lead the development of our big data solutions. As a product owner, you will be responsible for defining product vision, roadmap, and backlogs, ensuring alignment with business objectives.Your expertise in agile environments, lean methodologies, and technical product ownership...
-
Big Data Implementation Specialist
6 days ago
Lahore, Punjab, Pakistan Avanceon Middle East & South Asia Full timeAvanceon Middle East & South Asia is seeking an experienced Big Data Implementation Specialist to join our team. In this role, you will be responsible for designing and implementing big data solutions using cloud-based technologies.Key Responsibilities:Implement big data solutions using cloud-based technologies such as Azure Data FactoryManage data ingestion...
-
Cloud Architect for Big Data Solutions
3 days ago
Lahore, Punjab, Pakistan NorthBay Solutions LLC Full timeNorthBay Solutions LLC is hiring a skilled Cloud Architect for Big Data Solutions. As a Senior Big Data Engineer, you will be responsible for designing and developing cloud-based Big Data solutions using AWS services such as EMR, Glue, Lambda, and Athena.About the Role:Design and develop cloud-based Big Data solutions using AWS services.Develop and manage...
-
Big Data QA Automation
1 week ago
Lahore, Punjab, Pakistan HR POD Careers Full timeRequirements:4+ years of technical QA experience in Big Data technologies.2-3 years of experience with test automation frameworks.Proficient in Python, Pytest, Robot, and SQL for Big Data testing.Experience with Apache Spark and Big Data cloud platforms.Strong knowledge of Data Testing Strategies (data, process, outcome validation, code coverage).Experience...
-
Big Data Infrastructure Specialist
3 days ago
Lahore, Punjab, Pakistan Devsinc Full timeAs a Big Data Infrastructure Specialist, you will play a vital role in designing, developing, and maintaining our big data infrastructure. You will work closely with our cross-functional teams to ensure that our data systems are scalable, reliable, and efficient.Main Responsibilities:Designing and Developing Big Data Infrastructure for High Performance and...
-
Big Data Engineering Expert
3 days ago
Lahore, Punjab, Pakistan NorthBay Solutions LLC Full timeWe are seeking an experienced Big Data Engineering Expert to join our team at NorthBay Solutions LLC. As a Senior Big Data Engineer, you will have the opportunity to design and develop cutting-edge Big Data technologies, build critical distributed systems from scratch, and lead complex ETL jobs and pipelines.About the Role:Design and develop using Big Data...
-
Big Data Technical Lead
3 days ago
Lahore, Punjab, Pakistan Avanceon Middle East & South Asia Full time**About the Role:**We are seeking a highly skilled Senior Big Data Engineer to join our team at Avanceon Middle East & South Asia. As a Senior Big Data Engineer, you will be responsible for developing and implementing big data solutions using cloud-based technologies.Responsibilities:Big Data Solutions: Develop and implement big data solutions using...
-
Big Data Specialist
6 days ago
Lahore, Punjab, Pakistan Devsinc, LLC Full timeThe Senior Data Engineer will play a vital role in shaping our data strategy. We seek an experienced professional who can design, develop, and maintain large-scale data pipelines and databases, as well as collaborate with cross-functional teams to ensure seamless data integration.Key Responsibilities:Data Pipeline Development: Design, develop, and maintain...
-
Big Data Specialist
6 days ago
Lahore, Punjab, Pakistan Devsinc Full timeAbout the RoleThe successful candidate will have proven experience (3+ years) as a Data Engineer or similar role in data management, processing, and analytics. They will possess experience with data modeling, ETL frameworks, and big data tools like Apache Hadoop, Spark, Kafka, etc.They will also demonstrate proficiency in programming languages such as...
-
Big Data Engineer
1 week ago
Lahore, Punjab, Pakistan Burq Full timeWe value diversity and are an equal opportunity employer. We do not discriminate on the basis of race, religion, color, national origin, gender, sexual orientation, age, marital status, veteran status, or disability status.The Ideal Candidate Will HaveExperience working with data engineering tools and platformsStrong programming skills in Python and...
-
Big Data Engineer and Architect
1 week ago
Lahore, Punjab, Pakistan Data Pilot Full timeEmpowering Businesses with Real-Time Insights through Advanced Data ArchitecturesData Pilot is committed to helping organizations unlock the full potential of their data assets and maximize business value through our expert engineering and business acumen. As an Associate Data Architect, you will play a crucial role in designing and implementing scalable,...
-
Senior Big Data Engineer
3 weeks ago
Lahore, Punjab, Pakistan NorthBay Solutions LLC Full timeWe are hiring for Senior Big Data Engineer with solid experience in data engineering and a passion for Big Data. The ideal candidate will have experience of over 4 to 6+ years specifically in Data Engineering.You will have the chance to join a global, multi-cultural team of Data Engineers, participate in a large variety of exciting and challenging projects,...
-
Senior Big Data Engineer
2 weeks ago
Lahore, Punjab, Pakistan NorthBay Solutions Full timeWe are hiring for Senior Big Data Engineer with solid experience in data engineering and a passion for Big Data. The ideal candidate will have experience of over 4 to 6+ years specifically in Data Engineering.You will have the chance to join a global, multi-cultural team of Data Engineers, and be part of the growth and development of it, participate in a...
-
Big Data Engineer
2 days ago
Lahore, Punjab, Pakistan Gtradecenter Full timeInvoZone is a fast-growing company that offers a range of services to clients worldwide. As a Big Data Engineer, you will play a key role in developing innovative business solutions using Apache Spark.ResponsibilitiesDevelop and implement big data solutions using Apache SparkWork with cross-functional teams to identify business needsDesign and implement data...
-
Big Data Automation Specialist
1 week ago
Lahore, Punjab, Pakistan HR POD Careers Full timeJob Description:We are seeking a highly skilled Data Quality Engineer to join our team at HR POD Careers. As a key member of our big data testing team, you will be responsible for developing and automating processes for gathering expected results from data sources and comparing data results from testing.About the Role:Develop and automate test scripts to...
-
Expand Your Career with Big Byte Insights
19 hours ago
Lahore, Punjab, Pakistan Big Byte Insights Full timeAbout Big Byte InsightsAt Big Byte Insights, we're dedicated to empowering businesses with innovative software solutions that drive growth, efficiency, and customer satisfaction. Our team of experts collaborates closely with clients to understand their unique challenges and develops tailored solutions that exceed their expectations.Our ExpertiseWe specialize...