Senior Data Engineer

2 days ago


Lahore, Punjab, Pakistan Tkxel LLC Full time
Tkxel is a leading software development company located in Reston, Virginia. We are committed to developing innovative software solutions for leading enterprises in the world, helping them grow their businesses using the latest technology solutions.Job Description

We are seeking an experienced and highly skilled Senior Data Engineer to join our dynamic team. The ideal candidate will have a strong background in building and optimizing large-scale data pipelines, architecting cloud-based data solutions, and managing complex data environments using cloud technologies. You will work closely with cross-functional teams to design, develop, and deploy data solutions that power business insights.

Key Responsibilities:

  • Data Pipeline Development: Design, build, and maintain scalable ETL/ELT pipelines using AWS services such as Glue, EMR, and Athena. Leverage AWS Step Functions to automate complex workflows.
  • Data Warehousing: Architect and manage data warehouses using cloud technologies, while integrating with S3, RDS, and DynamoDB for both structured (SQL) and unstructured (NoSQL) data needs.
  • Big Data Processing: Utilize PySpark on AWS EMR for distributed data processing, ensuring the effective transformation of raw data into business insights.
  • Database Management: Manage both SQL (e.g., PostgreSQL, MySQL) and NoSQL (e.g., DynamoDB, MongoDB) databases, ensuring data is efficiently stored, retrieved, and queried.
  • Real-Time Data Processing: Implement and maintain real-time streaming data solutions using AWS Kinesis or Kafka for low-latency data processing.
  • Cloud Monitoring & Automation: Use AWS CLI and CloudWatch to monitor, automate, and optimize data pipelines and ensure efficient use of cloud resources.
  • Data Governance & Security: Ensure adherence to best practices for data security, governance, and compliance, including data encryption and access controls.
  • Collaboration: Work closely with data scientists, analysts, and business stakeholders to align data infrastructure with business objectives.
  • Documentation: Maintain detailed documentation of data models, data pipeline processes, and system architectures.
Requirements
  • 5+ years of experience as a Data Engineer, with a strong emphasis on cloud-based data warehousing and ETL pipelines.
  • Proficiency with AWS services including EMR, Athena, Glue, Step Functions, ElasticSearch, RDS, Redshift, DynamoDB, and S3.
  • Advanced SQL skills and experience with relational databases like PostgreSQL, MySQL as well as NoSQL databases such as MongoDB and DynamoDB.
  • Hands-on experience in Python for data processing and automation, with knowledge of AWS CLI.
  • Strong proficiency with PySpark for distributed data processing and transformation.
  • Experience with AWS CloudWatch for monitoring and optimizing data workflows.
  • Data warehouse architecture experience, including optimizing query performance in large-scale environments.
  • Strong understanding of data governance, security, and compliance best practices.
  • Bachelor's degree in Computer Science, Engineering, or a related field.

Nice to have:

  • Familiarity with real-time data streaming solutions such as Apache Kafka or AWS Kinesis.
#J-18808-Ljbffr

  • Lahore, Punjab, Pakistan Data Pilot Full time

    Unlocking Business Potential through Cutting-Edge Data Engineering and AnalyticsData Pilot is dedicated to empowering businesses with the data-driven insights they need to drive growth and success. As an Associate Data Architect, you will play a critical role in designing and implementing innovative data architectures that support advanced data engineering...


  • Lahore, Punjab, Pakistan Data Pilot Full time

    Empowering Businesses with Real-Time Insights through Advanced Data ArchitecturesData Pilot is committed to helping organizations unlock the full potential of their data assets and maximize business value through our expert engineering and business acumen. As an Associate Data Architect, you will play a crucial role in designing and implementing scalable,...


  • Lahore, Punjab, Pakistan Data Pilot Full time

    At Data Pilot, we're more than just a data product development and consulting services company – we're your strategic partner on the journey from data to decisions. Our mission is simple: to help businesses unlock the full potential of their data assets and maximize business value through our expert engineering and business acumen.In the dynamic landscape...

  • Senior Data Engineer

    2 weeks ago


    Lahore, Punjab, Pakistan Devsinc Full time

    Devsinc is seeking a talented Senior Data Engineer to join our dynamic team. The ideal candidate will possess a deep understanding of data architecture and engineering principles, along with hands-on experience in developing, optimizing, and maintaining data pipelines and databases. This role is crucial to support our data-driven initiatives and to enhance...


  • Lahore, Punjab, Pakistan FINCA Impact Finance Full time

    Senior Data Engineer FINCA Impact Finance , Pakistan The Senior Data Engineer takes an active part in development and support tasks related to Enterprise Data Warehouse (EDW), Data Lake, and BI/analytical solutions, which integrate data from 20 countries and provide support for decision-making. As a member of the EDW Development Team, the Senior Data...

  • Senior Data Engineer

    3 weeks ago


    Lahore, Punjab, Pakistan kisaangroup Full time

    Expert Talent Sourcer | Building Strong Candidate Relationships | Data-Driven Recruitment Strategies | Emotional IntelligenceWaseela is a catalyst for change, empowering rural communities to drive Pakistan's economy forward. Through data-driven, research-backed projects, we address critical challenges in our rural economy and unlock opportunities for...

  • Senior Data Engineer

    2 weeks ago


    Lahore, Punjab, Pakistan Devsinc, LLC Full time

    Devsinc is seeking a talented Senior Data Engineer to join our dynamic team. The ideal candidate will possess a deep understanding of data architecture and engineering principles, along with hands-on experience in developing, optimizing, and maintaining data pipelines and databases. This role is crucial to support our data-driven initiatives and to enhance...


  • Lahore, Punjab, Pakistan Data Pilot Full time

    Transforming Data into Business Insights through Innovative SolutionsAs an Associate Data Architect at Data Pilot, you will have the opportunity to work on cutting-edge data engineering and analytics projects that drive business growth and success. Our team is passionate about harnessing the power of data to deliver exceptional results and drive business...


  • Lahore, Punjab, Pakistan Data Pilot Full time

    Data Pilot: Unlocking Business Value through Scalable Data ArchitecturesAt Data Pilot, we're more than just a data product development and consulting services company – we're your strategic partner on the journey from data to decisions. Our mission is simple: to help businesses unlock the full potential of their data assets and maximize business value...


  • Lahore, Punjab, Pakistan Avanceon Limited Full time

    Join to apply for the Senior Big Data Engineer (Azure) role at Avanceon Middle East & South Asia.As a Big Data engineer, you will be responsible for data-related implementation tasks that include provisioning of data storage services, ingesting, streaming and batch data, transforming data, implementing security requirements, implementing data retention...

  • Data Engineer

    4 weeks ago


    Lahore, Punjab, Pakistan Systems Limited Full time

    Systems Limited is looking for a Senior Data Engineer with expertise in PySpark, Airflow, and Databricks to join our growing team.Responsibilities:Design, build, and maintain scalable data pipelines using PySpark for ETL/ELT processing, ensuring data is ingested, transformed, and loaded efficiently across multiple platforms.Use Apache Airflow to automate,...


  • Lahore, Punjab, Pakistan NorthBay Solutions LLC Full time

    NorthBay Solutions LLC is hiring a skilled Senior Data Engineer for Cloud Platforms. As a Senior Big Data Engineer, you will be responsible for designing and developing cloud-based Big Data solutions using AWS services such as EMR, Glue, Lambda, and Athena.About the Job:Develop and manage complex ETL jobs and pipelines in AWS-based environments.Collaborate...

  • Data Engineer

    2 weeks ago


    Lahore, Punjab, Pakistan Systems Limited Full time

    Senior Data Engineer:Systems Limited is looking for a Senior Data Engineer with expertise in PySpark, Airflow, and Databricks to join our growing team.Responsibilities:Design, build, and maintain scalable data pipelines using PySpark for ETL/ELT processing, ensuring data is ingested, transformed, and loaded efficiently across multiple platforms.Use Apache...


  • Lahore, Punjab, Pakistan NorthBay Solutions LLC Full time

    We are hiring for Senior Big Data Engineer with solid experience in data engineering and a passion for Big Data. The ideal candidate will have experience of over 4 to 6+ years specifically in Data Engineering.You will have the chance to join a global, multi-cultural team of Data Engineers, participate in a large variety of exciting and challenging projects,...


  • Lahore, Punjab, Pakistan NorthBay Solutions Full time

    We are hiring for Senior Big Data Engineer with solid experience in data engineering and a passion for Big Data. The ideal candidate will have experience of over 4 to 6+ years specifically in Data Engineering.You will have the chance to join a global, multi-cultural team of Data Engineers, and be part of the growth and development of it, participate in a...


  • Lahore, Punjab, Pakistan FINCA Impact Finance Full time

    Senior Data EngineerThe Enterprise Data Team at FINCA Impact Finance seeks a highly skilled Senior Data Engineer to join their team. As a key member of the EDW Development Team, this individual will play a pivotal role in designing and deploying databases, stored procedures, complex queries, and SSIS/automation jobs that integrate data from 20 countries and...

  • Data Engineer

    4 weeks ago


    Lahore, Punjab, Pakistan Devsinc Full time

    Design, implement, and maintain data pipelines to ensure seamless flow of data from various sources to storage systemsWork with large, complex datasets to ensure high performance, availability, and scalability of data systemsCollaborate with data scientists, analysts, and other engineering teams to deliver insights and business value from structured and...


  • Lahore, Punjab, Pakistan Devsinc Full time

    We are looking for a talented Senior Software Developer Data Engineering to contribute to the development and maintenance of our data infrastructure. As a key member of our team, you will play a critical role in designing, developing, and optimizing scalable data pipelines and ETL processes.Main Responsibilities:Developing and Optimizing Data Infrastructure...


  • Lahore, Punjab, Pakistan Data Magnetics Full time

    About the JobWe are seeking a highly skilled Senior .NET Developer to join our team at Data Magnetics Technologies. As a key member of our engineering team, you will be responsible for designing and developing innovative software solutions using C# with .NET Core MVC and Visual Studio.Your primary focus will be on writing high-quality APIs and UI components...

  • Data Engineer

    1 week ago


    Lahore, Punjab, Pakistan Crypthon Technologies Full time

    We're looking for a Data Engineer who thrives on building scalable data solutions, designing robust pipelines, and transforming complex data into actionable insights.Responsibilities:Build and maintain data pipelines and architectureCollaborate with teams to define data requirementsOptimize systems for performance and scalabilityWork with big data tools and...