Azure Data Engineer — Miami, FL (MUST BE LOCAL TO MA)

Hi,

Role: Data Engineer/Analyst-PCP
Work location: USA Miami – On site

JOB DESCRIPTION:
Key Skills: Azure, Databricks, SQL, Python
Overview:
We are seeking a skilled Data Engineer to join our team and contribute to various data projects. The ideal candidate will bring extensive experience with Azure and Databricks, possess deep SQL expertise, and have a solid understanding of Python. You will be responsible for integrating and managing diverse data sources, leveraging cutting-edge cloud technologies, and ensuring the seamless ingestion of data from multiple channels.
Key Responsibilities:
• Data Integration & Ingestion:
o Design, develop, and maintain robust data pipelines using a variety of data ingestion methods including APIs, Kafka streams, FTP file transfers, and Azure Data Factory (ADF).
o Integrate data from multiple sources such as AWS Redshift, S3, Azure Blob Storage, Azure Data Lake, BigQuery, Couchbase, Oracle, SQL Server, and Aurora.
• Cloud & Big Data Technologies:
o Utilize Azure and Databricks for scalable data processing and analytics.
o Collaborate with cross-functional teams to optimize cloud storage solutions and ensure efficient data movement and transformation.
• Database & Scripting:
o Develop, optimize, and maintain complex SQL queries to support data analysis and reporting.
o Use Python for data manipulation, automation, and enhancing data workflows where necessary.
• Collaboration & Continuous Improvement:
o Work closely with data scientists, analysts, and IT teams to identify data needs and improve data reliability.
o Stay updated with emerging trends and technologies in data engineering to continually enhance our data ecosystem.
Required Qualifications:
• Technical Expertise:
o Proven experience working with Azure and Databricks.
o Advanced proficiency in SQL and strong knowledge of relational and non-relational databases.
o Experience with Python for scripting and data manipulation.
• Data Integration Skills:
o Familiarity with various data ingestion methods and tools, including API integration, Kafka streaming, FTP file movement, and Azure Data Factory.
o Experience working with a diverse range of data sources and cloud storage solutions.
• Problem-Solving & Collaboration:
o Strong analytical and problem-solving skills with the ability to troubleshoot complex data issues.
o Excellent communication skills and a proven ability to work collaboratively across teams.
Preferred Qualifications:
• Experience with additional cloud platforms (AWS, Google Cloud) and data warehouses (BigQuery, Aurora).
• Prior experience in a fast-paced, data-driven environment, contributing to the development and optimization of large-scale data systems.