Large Scale Data Specialist

3 days ago

Islamabad, Islamabad, Pakistan beBeeDataEngineer Full time $80,000 - $100,000

Data Engineering Expertise

Job Summary:

We are seeking an experienced Data Engineer with a strong background in PySpark and Databricks. The successful candidate will play a pivotal role in designing, building, and maintaining large-scale data pipelines using these technologies.

Key Responsibilities:
  • Design and implement efficient large-scale data pipelines using PySpark.
  • Develop and maintain data orchestration pipelines within the Databricks platform.
  • Collaborate closely with stakeholders to clarify requirements and suggest improvements to existing data processes.
  • Migrate and integrate existing data products from one company to another ensuring seamless functionality.
  • Develop and maintain data pipelines for customer event journeys supporting analytics for our members.
  • Optimise data processing and storage for efficiency and scalability.
  • Implement monitoring and logging solutions to ensure data pipelines run smoothly and issues are promptly identified.
  • Ensure all data processing complies with security policies and data protection regulations.
  • Incorporate advanced analytics and machine learning where appropriate leveraging interest in AI.
  • Create and maintain comprehensive documentation for data architectures, processes, and procedures.
Requirements:
  • Qualification: Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field (or equivalent work experience).
  • Experience: 3+ years of experience in data engineering focusing on building large-scale data pipelines.
  • Databricks Expertise: At least 2 years of hands-on experience working with Databricks.
  • Big Data Technologies: Proficiency with big data technologies and frameworks e.g Hadoop Spark.
  • Cloud Platforms: Experience with cloud platforms such as AWS Azure or Google Cloud Platform.
  • Data Modelling: Solid understanding of data modelling ETL processes and data warehousing concepts.
  • Programming Skills: Proficiency in Python and SQL experience with other programming languages is a plus.
  • Stakeholder Engagement: Proven ability to communicate effectively with stakeholders to gather requirements and provide solutions.
  • Ai Interest: Demonstrated interest in AI machine learning or data science.
  • Problem-Solving Skills: Excellent analytical and troubleshooting abilities.
  • Communication: Strong verbal and written communication skills.
  • Agile Methodologies: Familiarity with Agile Scrum development processes.
  • Security Awareness: Understanding of data security practices and compliance requirements.
  • Continuous Learning: A proactive attitude towards continuous learning and staying updated with industry trends.
Benefits:
  • Employees Provident Fund medical and other incentives.
  • Unique working environment where you communicate and work directly with international clients.
Other Details:
  • Seniority level Mid-Senior level.
  • Employment type Full-time.
  • Job function Information Technology.
  • Industries IT Services and IT Consulting.

Referrals increase your chances of interviewing by 2x.

  • Visual Data Governance Specialist

    15 hours ago

    Islamabad, Islamabad, Pakistan beBeeDataGovernance Full time 8,000,000 - 15,000,000

    Job Title: Visual Data Governance SpecialistOverview:Our organization is seeking a skilled Visual Data Governance Specialist to oversee the management of our extensive video and image datasets. This role requires deep technical knowledge of both video and image processing, with expertise in handling big data workflows and optimizing data pipelines for...

  • Enterprise Data Engineer

    2 hours ago

    Islamabad, Islamabad, Pakistan beBeeData Full time $900,000 - $1,500,000

    Data Engineering OpportunityWe are seeking a skilled Data Engineer to design, build, and maintain large-scale data pipelines using PySpark and Databricks.The successful candidate will collaborate closely with stakeholders to clarify requirements and suggest improvements, ensuring our data infrastructure is robust, scalable, and efficient.

  • Data Acquisition Specialist Job Opening

    2 hours ago

    Islamabad, Islamabad, Pakistan beBeeData Full time 5,500,000 - 8,500,000

    Web Data Acquisition SpecialistWe are seeking a skilled and detail-oriented professional to join our data acquisition team. In this role, you will be responsible for designing, developing, and maintaining efficient and scalable data pipelines to collect information from various websites and online platforms.You will work closely with data analysts, product...

  • Video and Image Data Manager

    2 hours ago

    Islamabad, Islamabad, Pakistan beBeeData Full time 3,000,000 - 6,000,000

    Job Title: Video and Image Data Management SpecialistWe are seeking an experienced professional to oversee the management of our extensive video and image datasets. This role requires in-depth technical knowledge of both video and image processing, with expertise in handling large-scale data workflows and optimizing data pipelines for computer vision...

  • Chief Data Visionary

    6 days ago

    Islamabad, Islamabad, Pakistan beBeeDataArchitecture Full time 12,000,000 - 16,000,000

    Senior Data ArchitectWe are seeking a highly skilled Senior Data Architect to join our team. The ideal candidate will have expertise in designing and implementing large-scale data systems, utilizing technologies such as Python and Spark.About the Role:This is an exciting opportunity for a talented data professional to take on a key role in shaping our...

  • Senior Big Data Specialist

    2 hours ago

    Islamabad, Islamabad, Pakistan beBeeDataEngineer Full time $150,000 - $180,000

    Job OverviewWe are seeking a skilled Data Engineer to join our team. As a key member of our data engineering group, you will be responsible for designing, developing, and maintaining large-scale data systems.Your primary focus will be on building scalable, efficient, and reliable data pipelines that meet the needs of our business.You will work closely with...

  • Chief Data Infrastructure Specialist

    2 hours ago

    Islamabad, Islamabad, Pakistan beBeeData Full time 1,500,000 - 2,000,000

    Job Title: Data EngineerWe are seeking a skilled professional to design and develop large-scale data pipelines using modern big data technologies and cloud-native tools.Key Responsibilities:Design, build, and maintain scalable data infrastructure and pipelines in a cloud-based architecture.Develop efficient distributed data processing systems using Hadoop,...

  • Data Scientist/Engineer

    4 weeks ago

    Islamabad, Islamabad, Pakistan SwipBox Full time

    Job Summary: We are looking for a skilled Data Engineer/Data Scientist with strong Python programming and analytical skills, who can work across the data pipeline from ingestion and processing to modelling and reporting. The ideal candidate has experience handling large-scale (big) datasets, building predictive models, working with live and runtime data...

  • Lead Data Processing Specialist

    15 hours ago

    Islamabad, Islamabad, Pakistan beBeeDataEngineer Full time $70,000 - $100,000

    Job DescriptionWe are seeking an expert in large-scale data processing systems to join our team.The ideal candidate will have a strong background in data engineering, with experience working on complex data-intensive projects using Python and Spark.Develop maintainable and scalable data processing systems.Collaborate with cross-functional teams to design and...

  • Video & Image Data Management Specialist

    2 weeks ago

    Islamabad, Islamabad, Pakistan Cowlardesignstudio Full time

    Video & Image Data Management SpecialistAbout the job Video & Image Data Management SpecialistExpected start date: ASAP (position is open if you can still read this)Positions Open: 05Experience required: 0 to 2 years of working in a rapid startup environmentEducation required: Education is flexible; however, a background in engineering is a plusSalary...