Senior Data Engineer

3 weeks ago

Islamabad, Islamabad, Pakistan SOFTOO Full time

4 days ago Be among the first 25 applicants

Direct message the job poster from SOFTOO

We are seeking a Senior Data Engineer to design and implement the backbone of our data infrastructure for a large-scale Television Audience Measurement (TAM) platform. This role is central to building efficient, reliable, and scalable pipelines that move, transform, and manage data collected from thousands of edge devices and processed through our backend systems.

As a senior member of the engineering team, you will work closely with the Development Manager, Data Scientists, and DevOps team to build batch-oriented ETL workflows, manage large volumes of time-series and signature data, and ensure data integrity from ingestion to reporting.

Key Responsibilities:-

  • Design, develop, and maintain data pipelines for ingesting, validating, and transforming telemetry and audio signature data.
  • Work with tools like Kafka, Airflow, rsync, PostgreSQL, and MongoDB to orchestrate and manage data flow.
  • Collaborate with the TAM box collection system and headend systems to ensure seamless ingestion.
  • Implement efficient batch data workflows for daily audience analytics, panel metrics, and report generation.
  • Develop modular data processing scripts and transformations in Python and Bash.
  • Ensure high reliability, correctness, and fault tolerance in all data movement processes.
  • Implement logging, retry logic, and monitoring to track pipeline health and data anomalies.
  • Support data scientists by preparing clean, structured datasets and integrating model outputs into downstream systems.

Required Skills & Experience:-

  • 6+ years of experience in data engineering, backend development, or similar fields.
  • Strong proficiency with ETL pipelines, time-series data, and batch processing systems.
  • Advanced skills in Python and Bash for scripting and automation.
  • Experience with Kafka, Airflow, PostgreSQL, and MongoDB (or similar alternatives).
  • Solid understanding of data structures, data integrity, schema management, and storage optimization.
  • Comfortable working in Linux environments with on-prem infrastructure and physical servers.
  • Experience writing and optimizing SQL queries, views, and stored procedures.

Preferred

  • Prior experience working on telemetry systems, sensor data, or audience measurement platforms.
  • Familiarity with audio data pipelines or signature matching workflows.
  • Experience with data versioning, metadata tracking, and schema registries.
  • Exposure to the ELK stack, Logstash, and distributed file handling tools.

Bonus Experience

  • Working knowledge of MinIO, TimescaleDB, Parquet, or Columnar DBs.
  • Familiarity with Prefect, Luigi, or similar workflow orchestration tools.
  • Use of monitoring platforms like Prometheus, Grafana, or ELK for pipeline observability.
  • Experience supporting regulatory reporting or compliance-driven data pipelines.
Seniority level
  • Seniority levelMid-Senior level
Employment type
  • Employment typeFull-time
Job function
  • Job functionInformation Technology
  • IndustriesIT Services and IT Consulting

Referrals increase your chances of interviewing at SOFTOO by 2x

Sign in to set job alerts for "Senior Data Engineer" roles.Associate Software Engineer- Data EngineerSenior Software Engineer - .NET (Hybrid, Islamabad, PKR Salary)Senior Executive IT Software Engineering & DevOpsSenior Data Analyst/Reporting - Microsoft Power BISenior Software Engineer - .NET (6-months contract)Senior Software Engineer - .NET (6-months contract)Senior/Principal Software Engineer (.NET+ React)Senior/Principal Software Engineer-Vue.JSSenior Software Development Engineer in TestSenior Odoo Developer (SDE III Technical + Functional)

We're unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.

#J-18808-Ljbffr

  • Senior Data Engineer

    2 weeks ago

    Islamabad, Islamabad, Pakistan Tekrowe Full time

    5 days ago Be among the first 25 applicantsDirect message the job poster from TekroweTalent Acquisition Specialist @ Tekrowe Digital | Technical Recruiter | HR Operations | Head Hunter | Employee Engagement | Corporate CommunicationsThe Company Tekrowe Digital, a rapidly growing Software Development and Technology Innovation company in Pakistan, specializes...

  • Senior Data Engineer

    14 hours ago

    Islamabad, Islamabad, Pakistan beBee Careers Full time

    Senior Data EngineerThis senior-level position involves overseeing the development and maintenance of our data infrastructure. It requires extensive experience in designing, building, and scaling data pipelines as well as managing technical data products for client reporting. The ideal candidate should have 7+ years of experience in building and scaling data...

  • Senior Data Engineer

    4 weeks ago

    Islamabad, Islamabad, Pakistan Tekrowe Full time

    3 days ago Be among the first 25 applicantsGet AI-powered advice on this job and more exclusive features.Direct message the job poster from TekroweTalent Acquisition Specialist @ Tekrowe Digital | Technical Recruiter | HR Operations | Head Hunter | Employee Engagement | Corporate CommunicationsThe Company Tekrowe Digital, a rapidly growing Software...

  • Senior Data Engineer

    6 days ago

    Islamabad, Islamabad, Pakistan beBee Careers Full time

    About the RoleWe are seeking a Senior Data Engineer to join our Data & Analytics team. As a key member of the team, you will play a crucial role in developing high-quality Data Lakehouse products and solutions.The ideal candidate will have solid data engineering and software development skills, as well as experience with data modelling, integration, and...

  • Senior Data Engineer

    2 weeks ago

    Islamabad, Islamabad, Pakistan Cloudpso Full time

    Job DetailsJob timings: Mon - Fri 6 PM - 3 AM Pakistan timeJob Location: Islamabad I-9/3 (Onsite)Experience: Over 5 yearsResponsibilitiesAble to work on complex data-intensive projects.Good understanding of Python, Spark best practices, and commonly used modules based on work experience, creating self-contained, reusable, and testable modules and...

  • Senior Data Engineering Specialist

    6 days ago

    Islamabad, Islamabad, Pakistan beBee Careers Full time

    About the RoleThis is a remote, contract position responsible for designing, building, and maintaining large-scale data infrastructure required for data integration, storage, processing, and analytics. The ideal candidate will develop in an Agile environment, contributing to the architecture, design, and implementation of Data products, including migration...

  • Senior Data Engineer

    3 weeks ago

    Islamabad, Islamabad, Pakistan SOFTOO Full time

    Job SummaryWe are seeking a Senior Data Engineer to design and implement the backbone of our data infrastructure for a large-scale Television Audience Measurement (TAM) platform.About the RoleThis role is central to building efficient, reliable, and scalable pipelines that move, transform, and manage data collected from thousands of edge devices and...

  • Data Engineer

    2 weeks ago

    Islamabad, Islamabad, Pakistan Leed | Pakistan Full time

    Direct message the job poster from Leed | PakistanCertified Human Resources Professional -CHRPCompany Description:Leed Pakistan is a software development company established in 2020, headquartered in Canada, with development offices in Germany and Pakistan. Despite our rapid growth to 50+ professionals, we prioritize quality and customer care. We work with...

  • Senior Data Engineer

    2 days ago

    Islamabad, Islamabad, Pakistan Ciklum India Full time

    Ciklum is looking for a Senior Data Engineer to join our team full-time in Pakistan.We are a custom product engineering company that supports both multinational organizations and scaling startups to solve their most complex business challenges. With a global team of over 4,000 highly skilled developers, consultants, analysts and product owners, we engineer...

  • Senior Big Data Engineer

    4 weeks ago

    Islamabad, Islamabad, Pakistan NorthBay Solutions Full time

    We are hiring for Senior Big Data Engineer with solid experience in data engineering and a passion for Big Data. The ideal candidate will have experience of over 4 to 6+ years specifically in Data Engineering.You will have the chance to join a global, multi-cultural team of Data Engineers, and be part of the growth and development of it, participate in a...