Data Scientist

We are seeking a highly skilled Big Data Engineer to join our organization.

The ideal candidate will be responsible for designing, developing, and implementing large-scale data processing systems using big data technologies. The primary focus will be on building robust and efficient solutions for managing and analyzing vast amounts of data.

The successful candidate will collaborate with cross-functional teams, including business analysts, customers, data scientists, data analysts, and software engineers, to deliver scalable and high-performance data processing pipelines.

• Data Processing: Designing and developing applications and frameworks for big data processing, storage, and analytics.
• Pipeline Development: Building and maintaining large-scale data processing pipelines using distributed computing frameworks like Apache Hadoop or Apache Spark.
• Data Ingestion: Implementing data ingestion processes to acquire, process, and store structured and unstructured data from various sources.
• Collaboration: Collaborating with data scientists and data analysts to understand their requirements and translate them into scalable and efficient data processing workflows.
• Performance Optimization: Optimizing and tuning the performance of big data applications by implementing parallel processing, distributed caching, and other optimization techniques.
• Data Modeling: Developing and maintaining data models, schemas, and data dictionaries for efficient data storage and retrieval.
• Data Quality: Ensuring data quality, data integrity, and data security throughout the data processing lifecycle.
• Troubleshooting: Monitoring and troubleshooting data processing pipelines, identifying and resolving issues to ensure reliable and efficient operation.

• Programming Language: Strong proficiency in Java programming language and its ecosystem, including frameworks like Spring or Java EE.
• Big Data Technologies: Experience with big data technologies and frameworks such as Apache Hadoop, Apache Spark or similar distributed computing frameworks.
• Data Storage: Familiarity with data processing and storage technologies like Apache Kafka, Apache Hive, Apache HBase, or NoSQL databases.
• Distributed Computing: Solid understanding of distributed computing principles and concepts, including parallel processing, fault tolerance, and scalability.
• Data Modeling Techniques: Knowledge of data modeling techniques and experience with designing and optimizing data schemas for efficient data storage and retrieval.
• Database Management: Proficiency in SQL and experience working with relational databases like MySQL, PostgreSQL, or Oracle.
• Problem-Solving: Strong problem-solving skills and the ability to analyze complex data-related issues and provide effective solutions.
• Cloud Platforms: Familiarity with cloud platforms such as Amazon Web Services (AWS), Microsoft Azure, or Google Cloud Platform (GCP) and visualization tools like Tableau/Power BI is a plus.
• Communication Skills: Excellent communication and collaboration skills to work effectively with cross-functional teams.
• Education: Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
• Experience: 5+ years of experience in big data engineering or a similar role.