Senior Big Data Engineer

17 hours ago

Lahore, Punjab, Pakistan NorthBay Solutions Full time

We are hiring for Senior Big Data Engineer with solid experience in data engineering and a passion for Big Data. The ideal candidate will have experience of over 4 to 6+ years specifically in Data Engineering.

You will have the chance to join a global, multi-cultural team of Data Engineers, and be part of the growth and development of it, participate in a large variety of exciting and challenging projects and work with our exceptional team of Technology professionals.

Technology Stack Used & Required Knowledge:

  • 4 to 6+ years of experience on ETL Data Engineer, Python, Big Data (Hadoop, spark, hive)
  • Experience with distributed processing technologies and frameworks, such as Hadoop, Spark, Kafka, and distributed storage systems (e.g., HDFS, S3)
  • Excellent understanding of ETL cycle.
  • Experience in creating and driving large-scale ETL pipelines in AWS-based environment and experience of services such as EMR, Glue, Lambda, Athena
  • Experience with integration of data from multiple data sources.
  • Understanding of data processing pipelines, data management, and data engineering principles, including experience working with large-scale datasets and distributed computing frameworks (e.g., Spark).
  • Good knowledge and understanding of Big data engineering, ETL, Data Modeling, SQL Scripting, and Data Architecture.
  • Strong software development and programming skills with a focus on data using Python, PySpark and/or Scala for data engineering.
  • Experience and understanding of developing and maintaining complex stored procedures
  • Strong proficiency in ETL schedulers such as StepFunction or Apache Airflow
  • Solid understanding of data warehousing concepts and hands-on experience with relational databases (e.g., PostgreSQL, MySQL) and columnar databases (e.g., Redshift, BigQuery, HBase, ClickHouse)
  • Experience and understanding of various core GCP services such as BigQuery, Cloud Composer, Cloud Storage
  • Knowledge of containerization technologies, CI/CD pipelines and IAC tools like Docker, Jenkins, Terraform for automating software delivery process
  • Experience with the AWS data management tools such as Data lake and Databricks or AWS Snowflake is a plus
  • Understanding of descriptive and exploratory statistics, predictive modeling, evaluation metrics, decision trees, machine learning algorithms is a plus
  • Experience and understanding of AWS data migration services such as SCT, DataSync, Data Extractor will be a plus
  • Should have experience in Jira and Agile development methodologies.
  • Must be open to learning and have the ability to grasp new technologies quickly.
  • Be motivated and passionate about technology.
  • Team player with strong integrity and professionalism who can work well with others in the group.
  • Self-starter with a proven ability to take initiative and strong problem-solving skills.

To keep it short, below are some key responsibilities:

  • Design and develop using Big Data technologies.
  • Design and develop cutting-edge Big Data, and Cloud technologies to build critical, highly complex distributed systems from scratch
  • Design, develop and manage complex ETL jobs and pipelines.
  • Act as a team member, consulting other teammates as your new systems transform the way this entire firm leverages data
  • Deploying data models into production environments. This entails providing the model with data stored in a warehouse or coming directly from sources, configuring data attributes, managing computing resources, setting up monitoring tools, etc.
  • Support and help them learn and improve their skills, as well as innovate and iterate on best practices.
  • Solve complex issues and provide guidance to team members when needed.
  • Make improvement and process recommendations that have an impact on the business.
#J-18808-Ljbffr

  • Senior Big Data Engineer Position

    15 hours ago

    Lahore, Punjab, Pakistan NorthBay Solutions Full time

    About the JobWe are looking for a highly skilled Senior Big Data Engineer to join our team at NorthBay Solutions. The ideal candidate will have extensive experience working with big data technologies and a passion for innovation.In this role, you will be responsible for designing and developing complex big data solutions using a variety of technologies,...

  • Senior Big Data Engineer

    1 week ago

    Lahore, Punjab, Pakistan Avanceon Limited Full time

    Join to apply for the Senior Big Data Engineer (Azure) role at Avanceon Middle East & South Asia.As a Big Data engineer, you will be responsible for data-related implementation tasks that include provisioning of data storage services, ingesting, streaming and batch data, transforming data, implementing security requirements, implementing data retention...

  • Senior Big Data Engineer

    6 days ago

    Lahore, Punjab, Pakistan NorthBay Solutions LLC Full time

    We are hiring for Senior Big Data Engineer with solid experience in data engineering and a passion for Big Data. The ideal candidate will have experience of over 4 to 6+ years specifically in Data Engineering.You will have the chance to join a global, multi-cultural team of Data Engineers, participate in a large variety of exciting and challenging projects,...

  • Big Data Engineer Lead

    5 days ago

    Lahore, Punjab, Pakistan NorthBay Solutions LLC Full time

    Job DescriptionWe are seeking a highly skilled Senior Big Data Engineer to join our team at NorthBay Solutions LLC.The ideal candidate will have experience in data engineering and a passion for Big Data, with a strong understanding of the ETL cycle and distributed processing technologies.Key Responsibilities:Design and develop Big Data solutions using...

  • Big Data Engineer

    2 weeks ago

    Lahore, Punjab, Pakistan Arbisoft Full time

    Software Engineering - Arbisoft, Lahore, Islamabad, KarachiWe are looking for a skilled Big Data Engineer with 5+ years of experience in building scalable data solutions. The ideal candidate will have strong coding skills (preferably in Python or Java) and experience in managing the end-to-end development lifecycle—from development to deployment in...

  • Big Data Technology Expert

    15 hours ago

    Lahore, Punjab, Pakistan NorthBay Solutions Full time

    We are seeking an experienced Senior Big Data Engineer to join our team at NorthBay Solutions. In this role, you will be responsible for designing and developing complex big data solutions using a variety of technologies, including Hadoop, Spark, and NoSQL databases.Your primary responsibilities will include:Designing and developing ETL jobs and pipelines...

  • Cloud Big Data Solutions Architect

    5 days ago

    Lahore, Punjab, Pakistan NorthBay Solutions LLC Full time

    Job SummaryWe are looking for a highly skilled Senior Big Data Engineer to join our team at NorthBay Solutions LLC.The ideal candidate will have experience in data engineering and a passion for Big Data, with a strong understanding of the ETL cycle and distributed processing technologies.Key Responsibilities:Design and develop Big Data solutions using...

  • Senior Big Data Engineer

    6 days ago

    Lahore, Punjab, Pakistan Avanceon Middle East & South Asia Full time

    As a Big Data engineer, you will be responsible for data-related implementation tasks that include provisioning of data storage services, ingesting, streaming and batch data, transforming data, implementing security requirements, implementing data retention policies, identifying performance bottlenecks, and accessing external data. You will develop and...

  • Senior Data Engineering Specialist

    5 days ago

    Lahore, Punjab, Pakistan NorthBay Solutions LLC Full time

    About the RoleNorthBay Solutions LLC is seeking a seasoned Senior Big Data Engineer to lead our data engineering efforts.This role involves designing and developing large-scale ETL pipelines in AWS-based environments, as well as collaborating with the team to develop cutting-edge Big Data and Cloud technologies.Key Responsibilities:Design and develop complex...

  • Senior Big Data Solutions Specialist

    6 days ago

    Lahore, Punjab, Pakistan Avanceon Middle East & South Asia Full time

    Job ResponsibilitiesOur ideal candidate will have a strong background in big data engineering, with hands-on experience in designing and developing scalable big data solutions. They will be responsible for:Designing and developing data pipelines using Hadoop, Spark, and NoSQL databasesManaging large datasets and performing data analyticsProviding insights...

  • Data Engineer

    6 days ago

    Lahore, Punjab, Pakistan Avanceon Middle East & South Asia Full time

    Company OverviewAvanceon Middle East & South Asia is a leading provider of technology solutions, committed to delivering exceptional services to our clients. Our team of experts works tirelessly to stay ahead of the curve in the ever-evolving world of data storage and processing.Job DescriptionAs a Senior Big Data Engineer at Avanceon Middle East & South...

  • Data Engineering Team Lead

    5 days ago

    Lahore, Punjab, Pakistan NorthBay Solutions LLC Full time

    Job DetailsNorthBay Solutions LLC is seeking a highly skilled Senior Big Data Engineer to join our team.The ideal candidate will have experience in data engineering and a passion for Big Data, with a strong understanding of the ETL cycle and distributed processing technologies.Key Responsibilities:Design and develop complex ETL jobs and pipelines in...

  • Big Data Expert

    5 days ago

    Lahore, Punjab, Pakistan Gtradecenter Full time

    At Gtradecenter, we're looking for a highly skilled Big Data Expert to join our development team.About UsWe're a growing company in the IT industry, with over 10 years of experience in software development. Our mission is to automate daily work processes for our clients using innovative business solutions.We take pride in our diverse team, consisting of...

  • Senior Big Data Manager

    5 days ago

    Lahore, Punjab, Pakistan Confiz Full time

    Our Technology StackWe utilize a range of big data tools and technologies, including Azure, Apache NiFi, and Power BI. Our team is well-versed in data lake architecture, data model design, and implementation, as well as enterprise application and data integration design and implementation.What We OfferA dynamic and innovative work environment.Opportunities...

  • Senior Cloud Data Engineer

    20 hours ago

    Lahore, Punjab, Pakistan Tkxel LLC Full time

    About UsTkxel LLC is a renowned software development company located in Reston, Virginia. We specialize in delivering cutting-edge software solutions that cater to the diverse needs of leading enterprises worldwide.Job SummaryWe are seeking an accomplished Senior Data Engineer to join our talented team. As a critical component of our data infrastructure...

  • Data Architect Lead

    15 hours ago

    Lahore, Punjab, Pakistan NorthBay Solutions Full time

    We are seeking a highly experienced Senior Big Data Engineer to join our team at NorthBay Solutions. As a key member of our data engineering group, you will play a vital role in designing and developing cutting-edge big data solutions using Hadoop, Spark, and other emerging technologies.Your primary responsibilities will include:Designing and developing...

  • Senior Data Engineering Professional

    7 days ago

    Lahore, Punjab, Pakistan Confiz Full time

    We are seeking a highly skilled data engineer to expand and optimize our big data pipelines using Azure services.About You:You have a strong understanding of data engineering concepts, including data warehousing, publishing, and visualization.You have experience in designing, prototyping, implementing, and optimizing data ingestion pipelines.You have...

  • Senior Data Engineer

    2 weeks ago

    Lahore, Punjab, Pakistan Tkxel Full time

    Tkxel is a leading software development company located in Reston, Virginia. We are committed to developing innovative software solutions for leading enterprises in the world, helping them grow their businesses using the latest technology solutions.Job DescriptionWe are seeking an experienced Data Engineer to design, develop, and optimize our client data...

  • Senior Business Development Manager

    2 weeks ago

    Lahore, Punjab, Pakistan Big Entities Full time

    We are seeking a results-driven Senior Business Development Manager to drive growth and establish strong client relationships.Key ResponsibilitiesDevelop and execute business development strategies for acquiring new clients.Build and manage email pipelines and LinkedIn outreach to engage prospects.Conduct market research to identify potential clients and...

  • Associate Data Architect

    2 weeks ago

    Lahore, Punjab, Pakistan Data Pilot Full time

    Associate Data Architect - Analytics & AICompany Overview:At Data Pilot, we're more than just a data product development and consulting services company – we're your strategic partner on the journey from data to decisions. Our mission is simple: to help businesses unlock the full potential of their data assets and maximize business value through our expert...