Vanguard Group LLC
Work Authorization: USC & GC
10+ years of IT experience, including deep expertise in data engineering & ETL.
4-6+ years of recent hands-on experience in designing, building, and scaling data pipelines.
Direct experience designing, configuring, coding, and deploying systems.
Strong, hands-on experience with Databricks (Spark, Delta Lake, Unity Catalog) and AWS (S3, IAM, networking).
Highly proficient in Databricks’ core architectural pattern, which combines the best elements of data lakes and data warehouses.
Cost and performance management: Databricks can become highly expensive if left unoptimized. This person must know how to choose the right AWS EC2 instance types for clusters, configure auto-scaling efficiently, use Spot instances safely, minimize data shuffling, and optimize Photon engine performance to get the fastest runtimes for the lowest cost.
Familiar with advanced features like Unity Catalog (for governance), Delta Live Tables (DLT for pipelines), Databricks SQL, and workflow orchestration.
Expertise in SQL, Python, data modeling, and distributed processing.
Experience with CI/CD, DevOps, and modern data architecture (lakehouse, medallion, etc.).
Experience working in an Agile analytics environment where close collaboration with key stakeholders to address evolving requirements is the norm.
Excellent communicator who can be the front-face of the data team and lead and push back on stakeholders during requirements gathering, delivery, and support.
A passion for continued learning and an interest in staying current with the rapidly evolving data and analytics space.
To apply for this job email your details to mani@vanguard-itsolutions.com