Location: Dublin, OH (Onsite)
Duration: 6+ months
Job Description:
Role Descriptions: Re-write existing pipelines to support business need.Designing and building production data pipelines from ingestion to consumption within a hybrid big data architecture| using Java| dataflow| Python| airflow etc.Designing and implement Collibra Workflows using Java. Designing and implementing data transformation| ingestion| and curation functions on GCP cloud using GCP native or custom programming Optimizing data pipelines for performance and cost for large scale data lakes.Ensure technical specifications are aligned with both business needs and technical design standards. Partner with external consultants| solution providers| and managed services organizations to enable productsolution development as well as meeting documented standards. Interact with multiple organizations to track project progress| identify risks| communicate risks and status to leadership| and to assess potential impacts to the business. Generate ideas and suggestions for process and technical improvements for platforms and processes supported by the team Ensure platforms and tools meet or exceed data security standards| including internal and external audits performed. Use strong verbal and written communication skills that non-technical business and end-users can understand. Develop best practices for solution and tool frameworks| leveraging standard naming conventions| scripting| and coding practices to ensure consistency of data solution.
Essential Skills:
Bachelors degree preferred or equivalent work experience.8 years of engineering experience in Big Data systems| Data Analytics and Data Integration related fields. 2 years of hands-on GCP experience in Data Engineering| Cloud Analytics solutions| and experience with operationalizing Enterprise scale solutions. 5 years of Experience in Programming languages Python| Java| and frameworks- Spring Boot| Spring MVC| REST API development expertise.Experience with CICD pipelines such as Concourse| Jenkins.Knowledge of Terraform is a plus
Preferred to have prior experience in Collibra and Atscale. Hands-on experience with Data Ingestion technologies like GCP DataFlow| Fusion| and AirFlowExperience in designing and optimizing data models on GCP cloud using GCP data stores such as BigQuery.Experience integrating GCP or 3rd party KMS| HSM with GCP data services for building secure data solutions.Experience in implementing metadata management on GCPGoogle Cloud Platform certification is a plus
Thanks, and regards
Ganesh Gorak
Itech Us Inc