Big Data Pipeline Developer
2 days ago
We are seeking an experienced data engineer to join our team at Confiz. As a data engineer, you will play a crucial role in expanding and optimizing our big data pipelines. You will collaborate with Big Data Solution Architects to design, implement, and optimize data ingestion pipelines, ensuring seamless data sharing across various business systems.
Responsibilities- Collaborate with Big Data Solution Architects to design, prototype, implement, and optimize data ingestion pipelines so that data is shared effectively across various business systems.
- Build ETL/ELT Ingestion pipelines and analytics solutions using cloud and on-premises technologies.
- Ensure the design, code, and procedural aspects of solution are production ready, in terms of operational, security and compliance standards.
- Participate in day-to-day project and production delivery status meetings, and provide technical support for faster resolution of issues.
- A Bachelor's degree from an accredited college/university or equivalent work experience.
- 5+ years of experience performing data engineering, warehousing, publishing and visualization throughout the full data lifecycle.
- At least two (2) years of experience as an Azure Data Engineer, or in a similar role in a technology or IT consulting environment.
- 3 years of experience in Data profiling, cataloguing and mapping to enable the design and build of technical data flows.
- 3 years of experience defining ELT/ETL architecture and process design.
- 2 years of experience performing end-to-end implementation of data warehousing analytics solutions built on MS or Azure platforms.
- 2 years of experience working with Azure.
- Experience in creating data pipelines using ETL tools like Airflow, ADF.
- Experienced in distributing computing technologies like Apache Spark, Databricks.
- Experienced with stream-processing systems like Spark Streaming, Eventhub.
- Proficient in one of the object-oriented and functional languages: Python, Java, Scala, etc.
- Experience of writing effective and maintainable unit and integration tests for ingestion pipelines.
- Experience of using static analysis and code quality tools.
- Experience in using code version control repositories like Git.
- Experienced in security best practices including encryption of sensitive data, encryption at rest and encryption in flight.
- Experience of building CI/CD pipelines & DevOps orchestration tool like Jenkins.
- Familiarity with Data Lakehouse, Delta Lake, Apache Hudi.
-
Big Data Solutions Engineer
18 hours ago
Lahore, Punjab, Pakistan tkxel Full timeBig Data Solutions Engineer:tkxel is looking for a Big Data Solutions Engineer to join our team. As a key member of our team, you will design, develop, and maintain scalable data pipelines using PySpark and Azure Data Factory. You will also transform and integrate large datasets from multiple sources to support analytical and business intelligence...
-
Python Web Development Specialist
2 days ago
Lahore, Punjab, Pakistan Big Data Analytics Full timeAbout Big Data Analytics CompanyWe specialize in providing innovative data analytics solutions.Job OverviewThis role involves working on high-performance web applications using Python and Django.Your Key ResponsibilitiesDeveloping robust backend systems utilizing Python and Django.Collaborating with cross-functional teams to identify and prioritize project...
-
Big Data Architect
2 days ago
Lahore, Punjab, Pakistan Avanceon Middle East & South Asia Full timeJob OverviewAvanceon Middle East & South Asia seeks a seasoned Big Data Architect to spearhead the design and implementation of cloud-based data solutions. As a key member of our engineering team, you will be responsible for leading large-scale data projects from concept to delivery.
-
Big Data Solutions Developer
18 hours ago
Lahore, Punjab, Pakistan Tkxel Full timeTkxel OverviewTkxel is a software development company located in Reston, Virginia. We specialize in developing cutting-edge software solutions for top-tier enterprises worldwide. Our goal is to leverage the latest technology innovations to help businesses grow and succeed.We require a seasoned Data Engineer with hands-on experience in PySpark and Azure Data...
-
Big Data Professional
2 days ago
Lahore, Punjab, Pakistan Avanceon Middle East & South Asia Full timeWe are looking for a skilled Azure Data Engineer to join our team at Avanceon Middle East & South Asia.The ideal candidate will have hands-on experience with data engineering tools and technologies.Key Responsibilities:Developing and maintaining data pipelinesEnsuring data quality and securityCollaborating with cross-functional teamsEssential...
-
Senior Big Data Engineer
2 days ago
Lahore, Punjab, Pakistan Tkxel LLC Full timeJob OverviewTkxel LLC is a leading software development company located in Virginia, committed to developing innovative software solutions for top enterprises worldwide. We are seeking an experienced Senior Data Engineer to design, develop, and deploy data solutions that power business insights.Responsibilities:Develop and maintain large-scale data pipelines...
-
Big Data Engineer Professional
2 days ago
Lahore, Punjab, Pakistan Avanceon Middle East & South Asia Full timeAbout the RoleThis position is an excellent opportunity for a talented Data Engineer to take their career to the next level at Avanceon Middle East & South Asia. We offer a challenging and rewarding work environment where you can utilize your skills and expertise to drive business growth.Responsibilities:Develop and implement data lakes solutions for...
-
Cloud Data Architect Opportunity
2 days ago
Lahore, Punjab, Pakistan Arbisoft Full timeArbisoft seeks a Big Data Engineer to design and implement big data solutions using Hadoop, Spark, and other big data technologies. The ideal candidate has expertise in data processing, machine learning, and data visualization.Key Responsibilities:Design and develop big data architectures using Hadoop, Spark, and other big data technologies.Implement data...
-
Data Scientist
2 days ago
Lahore, Punjab, Pakistan Avanceon Middle East & South Asia Full timeJob DescriptionWe are seeking a highly skilled Data Engineer to join our team at Avanceon Middle East & South Asia. As a key member of our organization, you will be responsible for designing and developing data pipelines that leverage structured, semi-structured, and unstructured data.Key Responsibilities:Perform analysis of large data stores and derive...
-
Big Data Engineer Role
2 days ago
Lahore, Punjab, Pakistan Arbisoft Full timeArbisoft requires a Cloud Data Architect to design and implement scalable cloud-based data solutions using AWS services. The ideal candidate has experience with cloud infrastructure, data warehousing, and ETL processes.Key Responsibilities:Design and implement cloud-based data architectures using AWS services.Develop and maintain data pipelines, ensuring...
-
Senior Data Scientist Position
2 days ago
Lahore, Punjab, Pakistan Arbisoft Full timeArbisoft is looking for a Senior Data Scientist to develop and implement data-driven solutions using big data technologies. The ideal candidate has expertise in data modeling, database design, and workflow orchestration tools like Apache Airflow.Key Responsibilities:Design and develop complex data models and database architectures.Implement data pipelines...
-
Data Engineering Expert
2 days ago
Lahore, Punjab, Pakistan Arbisoft Full timeArbisoft seeks a skilled Data Engineer to design and maintain scalable data pipelines using Python and SQL. The ideal candidate has hands-on experience with ETL processes, AWS data services, and big data technologies.Key Responsibilities:Create efficient data workflows by integrating AWS services (e.g., S3, Redshift, Glue, Lambda).Collaborate with data...
-
Lahore, Punjab, Pakistan Confiz Full timeConfiz is seeking a skilled data engineer to expand and optimize our big data pipelines. You will work closely with Big Data Solution Architects to design, implement, and optimize data ingestion pipelines, ensuring seamless data sharing across various business systems.Job DescriptionDesign, prototype, implement, and optimize data ingestion pipelines so that...
-
Data Architect with Azure Specialization
2 days ago
Lahore, Punjab, Pakistan Confiz Full timeConfiz is seeking a highly skilled data engineer to join our team and contribute to the expansion and optimization of our big data pipelines. As a key member of our team, you will work closely with Big Data Solution Architects to design, implement, and optimize data ingestion pipelines, ensuring seamless data sharing across various business systems.About the...
-
Cloud Data Engineering Leader
2 days ago
Lahore, Punjab, Pakistan Confiz Full timeWe are looking for a seasoned data engineer to join our team at Confiz, where you will be responsible for expanding and optimizing our big data pipelines. Collaborative problem-solving, sophisticated design, and the creation of quality products are essential in this role.Key ResponsibilitiesWe require a skilled data engineer who can work alongside our Big...
-
Enterprise Data Solutions Architect
2 days ago
Lahore, Punjab, Pakistan Tkxel LLC Full timeRole DescriptionThe Senior Data Engineer will be responsible for designing, developing, and deploying data solutions that power business insights at Tkxel LLC. This role requires a strong background in cloud-based data warehousing, ETL pipelines, and big data processing.Key Responsibilities:Develop and maintain large-scale data pipelines using AWS services...
-
Software Development Professional
2 days ago
Lahore, Punjab, Pakistan Big Byte Insights Full timeAbout Big Byte Insights">Big Byte Insights is a leading company in the software development industry, headquartered in Lahore. Our mission is to provide innovative solutions that meet the evolving needs of our clients.We are seeking an experienced SQA Engineer (Manual) with at least 2 years of experience to join our team.">Key Responsibilities">Conduct web...
-
Data Scientist Position
18 hours ago
Lahore, Punjab, Pakistan Techcreatix Full timeCompany Overview:Techcreatix is a leading organization in the field of technology, dedicated to harnessing the power of machine learning and artificial intelligence to drive innovation and growth.Job Description:We are seeking an experienced Machine Learning Engineer to join our team. The ideal candidate will have a strong background in developing and...
-
Pipeline Engineer Expertise
2 days ago
Lahore, Punjab, Pakistan SPEC Refinery (PVT) LTD Full timeAbout the RoleWe are looking for a seasoned Pipeline Engineer with expertise in developing and finalizing piping deliverables. As a key member of our team, you will design pipe supports, prepare data sheets for valves and special items, create tie-in procedures, and perform pipe stress calculations.
-
Big Data Analyst Manager
2 days ago
Lahore, Punjab, Pakistan Zepto Systems Limited Full timeAbout This Opportunity:This is a fantastic chance to join our team as a Data Scientist and contribute to the development of cutting-edge ML solutions.You will work closely with our data engineering and software engineering teams to ensure seamless integration of ML solutions into production environments.As a member of our team, you will have opportunities to...