High-Performance Data Architect

We are seeking a highly skilled Data Engineer to design, build, and maintain large-scale data architectures for data integration, storage, processing, and analytics. This is a remote contract position.

• Design, develop, test, and maintain high-performance data architectures for data integration (batch and real-time), storage, processing, and orchestration;
• Contribute to detailed architectural discussions and customer requirements;
• Develop clear, maintainable code with automated testing;
• Collaborate with cross-functional teams to understand data requirements and deliver reusable components;
• Evaluate and implement new technologies to improve data integration, processing, storage, and analysis;
• Design and maintain data governance: cataloging, lineage, data quality;
• Monitor and optimize workloads and clusters for performance;
• Mentor junior team members and share best practices;

Our company is committed to diversity and inclusion. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability, or any other characteristic protected by applicable federal, state, or local laws.

• 5+ years of experience as a data engineer with strong expertise in Databricks, Azure, DevOps, or other hyperscalers;
• Strong programming skills in Python, Scala, and writing efficient and optimized code for data integration, migration, storage, processing, and manipulation;
• Strong understanding and experience with SQL and writing advanced SQL queries;
• Thorough understanding of big data principles, techniques, and best practices;
• Experience with scalable and distributed data processing technologies such as Spark/PySpark (Azure Databricks), DBT, and Kafka;
• Databricks development experience with Python, PySpark, Spark SQL, Pandas, NumPy in Azure;
• Designing and implementing efficient ELT/ETL processes in Azure and Databricks; develop custom integration solutions as needed;
• Data integration from APIs, databases, flat files, event streaming;
• Relational Databases (Oracle, SQL Server, MySQL, Postgres) and NonSQL Databases (MongoDB or similar);
• Data modeling and database design principles; design efficient schemas for data architecture;
• Data warehousing, data lake, and data lake house solutions in Azure and Databricks;
• Delta Lake, Unity Catalog, Delta Sharing, Delta Live Tables (DLT);
• SDLC knowledge, Agile methodologies;
• Experience with SDLC tools: Azure DevOps, GitHub; project management (Jira/Azure Boards), source control, CI/CD (GitHub Actions, Azure Pipelines), artifact management;
• DevOps principles: CI/CD, IaC (Terraform, ARM), configuration management, automated testing, performance tuning, cost optimization;
• Cloud computing in Microsoft Azure for data and analytics: ADF, Databricks, Synapse, Data Lake, Data Lake Storage, SQL Database, etc.;
• Orchestration with Databricks workflows and Apache Airflow;
• Data structures and algorithms; strong software engineering practices;
• Debugging and troubleshooting in complex data environments;
• Data quality and governance, including data quality checks and monitoring;
• BI solutions (Power BI) is a plus;
• Strong communication skills for cross-functional collaboration;
• Documentation of processes and deployment configurations;
• Security practices: network security groups, Azure Active Directory, encryption; compliance;
• Ability to implement security controls in data/analytics solutions;
• Mentoring and coaching of team members; willingness to stay updated with trends;
• Ability to work independently in a rapidly changing environment;
• Focus on architecture, observability, testing, and reliable data pipelines;