Big Data Solutions Expert

1 week ago

Karachi, Sindh, Pakistan beBeeBig Full time 1,000,000 - 1,500,000

Job Title: Big Data Solutions Expert

Job Overview:

We are seeking a highly skilled and experienced professional to play a pivotal role in building, deploying, and optimizing data infrastructure and pipelines in a scalable cloud-based architecture.

The ideal candidate will have a strong foundation in big data, distributed computing, and cloud-based data solutions, with expertise in designing, developing, and maintaining large-scale data pipelines using modern big data technologies and cloud-native tools.

  • Key Responsibilities:
  • Design, develop, and maintain large-scale data pipelines using modern big data technologies and cloud-native tools.
  • Build scalable and efficient distributed data processing systems using Hadoop, Spark, Hive, and Kafka.
  • Work extensively with cloud platforms (preferably AWS) and services like EMR, Glue, Lambda, Athena, S3.
  • Design and implement data integration solutions pulling from multiple sources into a centralized data warehouse or data lake.
  • Develop pipelines using DBT (Data Build Tool) and manage workflows with Apache Airflow or Step Functions.
  • Write clean, maintainable, and efficient code using Python, PySpark, or Scala for data transformation and processing.
  • Build and manage relational and columnar data stores such as PostgreSQL, MySQL, Redshift, Snowflake, HBase, ClickHouse.
  • Implement CI/CD pipelines using Docker, Jenkins, and other DevOps tools.
  • Collaborate with data scientists, analysts, and other engineering teams to deploy data models into production.
  • Drive data quality, integrity, and consistency across systems.
  • Participate in Agile/Scrum ceremonies and utilize JIRA for task management.
  • Provide mentorship and technical guidance to junior team members.
  • Contribute to continuous improvement by making recommendations to enhance data engineering processes and architecture.
Requirements:
  • 5+ years of hands-on experience as a Data Engineer
  • Deep knowledge of Big Data technologies – Hadoop, Spark, Hive, Kafka
  • Expertise in Python, PySpark and/or Scala
  • Proficient with data modeling, SQL scripting, and working with large-scale datasets
  • Experience with distributed storage like HDFS and cloud storage (e.g., AWS S3)
  • Hands-on with data orchestration tools like Apache Airflow or StepFunction
  • Experience working in AWS environments with services such as EMR, Glue, Lambda, Athena
  • Familiarity with data warehousing concepts and experience with tools like Redshift, Snowflake (preferred)
  • Exposure to tools like Informatica, AbInitio, Apache Iceberg is a plus
  • Knowledge of Docker, Jenkins, and other CI/CD tools
  • Strong problem-solving skills, initiative, and a continuous learning mindset
Preferred Qualifications:
  • Bachelor's or Master's degree in Computer Science, Data Engineering, or related field
  • Experience with open table formats such as Apache Iceberg
  • Hands-on with AbInitio (GDE, Collect > IT) or Informatica tools
  • Knowledge of Agile methodology, working experience in JIRA
Soft Skills:
  • Self-driven, proactive, and a strong team player
  • Excellent communication and interpersonal skills
  • Passion for data and technology innovation
  • Ability to work independently and manage multiple priorities in a fast-paced environment

Seniority level: Mid-Senior level

Employment type: Full-time

Job function: Information Technology

Industries: IT Services and IT Consulting

  • Big Data Engineer

    2 weeks ago

    Karachi, Sindh, Pakistan Syhtek Full time

    The Big Data Engineer is responsible for designing, developing and managing big data platforms to process and analyze large datasets. You will work with cutting-edge technologies to ensure seamless data flow, scalability and high-performance systems. Your work will be crucial in providing actionable insights for decision-making across the business. Join us...

  • Senior Big Data Specialist

    1 week ago

    Karachi, Sindh, Pakistan beBeeDataEngineer Full time 1,000,000 - 1,300,000

    Data Engineer OpportunityWe are seeking a highly skilled Data Engineer to join our team. This is an exciting opportunity for the right individual to utilize their expertise in big data technologies, cloud-native tools, and distributed computing to drive business growth.As a Data Engineer, you will design, develop, and maintain large-scale data pipelines...

  • Big Data Architect

    1 hour ago

    Karachi, Sindh, Pakistan beBeeData Full time 6,000,000 - 8,000,000

    Senior Data Solutions DeveloperAs a Senior Data Solutions Developer, you will be responsible for designing and developing data pipelines that extract, load, and transform data from various sources. This includes using ELT tools and technologies to clean, transform, and integrate data from different systems.This role requires strong analytical thinking and...

  • Senior Data Platform Developer

    1 week ago

    Karachi, Sindh, Pakistan beBeeDataEngineer Full time 2,000,000 - 2,500,000

    Job Title: Big Data EngineerThis role involves the design, development, and management of big data platforms for processing and analyzing large datasets.Key Responsibilities:Data System Design: Architect scalable data processing systems to meet business needs.Data Pipeline Management: Build and maintain efficient data pipelines for real-time processing.Data...

  • Data Engineer

    7 days ago

    Karachi, Sindh, Pakistan NorthBay - Pakistan Full time

    Job Title: Data EngineerJob Title: Data EngineerLocation: Karachi, Lahore , Islamabad (Hybrid)Experience: 5+ YearsJob Type: Full-TimeJob OverviewWe are looking for a highly skilled and experienced Data Engineer with a strong foundation in Big Data, distributed computing, and cloud-based data solutions. This role demands a strong understanding of end-to-end...

  • Sales Specialist

    4 weeks ago

    Karachi, Sindh, Pakistan Big Bash Studio Full time

    Big Bash Studio is on the lookout for aSales Specialist (focused on Bark & Upwork) - Night Shift (Karachi) , someone who is not just good at bidding or closing deals — but also takes full ownership, leads with confidence, and is ready to build and grow a high-performance sales team.Job Requirements:- Take full ownership of the sales cycle — from lead...

  • Technology Solutions Expert

    1 week ago

    Karachi, Sindh, Pakistan beBeeTechnical Full time 800,000 - 1,000,000

    Job Overview:We're seeking a skilled professional to join our team as an IT Specialist. This role involves providing technical support and troubleshooting for IT-related issues, assisting in the maintenance and administration of IT systems, collaborating with team members to develop and implement technology solutions, and monitoring and ensuring the security...

  • Chief Data Infrastructure Specialist

    1 week ago

    Karachi, Sindh, Pakistan beBeeDataEngineering Full time

    Senior Data Engineering PositionWe are seeking a highly skilled Senior Data Engineer to join our team. The successful candidate will be responsible for designing, developing, and maintaining scalable data pipelines and infrastructure.The ideal candidate will have extensive experience in big data tools, cloud platforms, and data integration techniques. They...

  • Enterprise Data Modeling Expert

    1 week ago

    Karachi, Sindh, Pakistan beBeeDataArchitect Full time 1,500,000 - 2,500,000

    We are seeking a seasoned Data Architect to lead our data modeling and ontology development efforts. This individual will play a pivotal role in designing, structuring, and maintaining enterprise-wide data models, ontologies, and semantic layers to support advanced analytics and data-driven decision-making.The ideal candidate will possess deep expertise in...

  • Data Manager

    4 weeks ago

    Karachi, Sindh, Pakistan LeadingEdge HR Solutions Full time

    Join to apply for the AI Data Manager-ETL role at LeadingEdge HR SolutionsContinue with Google Continue with Google2 days ago Be among the first 25 applicantsJoin to apply for the AI Data Manager-ETL role at LeadingEdge HR SolutionsJob Title : AI Data ManagerLocation: Bahrain (with frequent travel to GCC)Experience: 4-6 yearsJob SummaryOur client is seeking...