Data Engineer with Cloudera Onsite

Years of experience needed –

5-10 years

Technical Skills:

1. Data Pipeline Development:

• Design, develop, and implement scalable data pipelines using Cloudera tools like Hadoop, Spark, Hive, Impala & HDFS.

• Write and optimize ETL processes to extract, transform, and load data into data lakes or warehouses.

2. Big Data Application Development:

• Develop applications to process large datasets efficiently using frameworks such as Apache Spark and MapReduce.

• Build solutions for batch and real-time data processing.

3. Cluster Management:

• Work with Cloudera Manager for cluster setup, configuration, monitoring, and performance optimization.

• Ensure high availability and scalability of Cloudera clusters.

4. Data Storage and Management:

• Design and implement data storage strategies using HDFS, HBase, and other Cloudera-supported tools.

• Optimize data storage and retrieval processes to improve performance.

5. Security and Governance:

• Implement security measures using Cloudera tools like Apache Ranger and Sentry.

• Ensure compliance with data governance and security policies.

6. Performance Tuning:

• Monitor and optimize the performance of Hadoop and Spark jobs.

• Troubleshoot and resolve performance bottlenecks in data pipelines.

7. Integration and Collaboration:

• Integrate Cloudera solutions with external systems, databases, and APIs.

• Collaborate with data scientists, analysts, and other teams to understand requirements and deliver data solutions.

8. Documentation and Support:

• Document processes, workflows, and solutions for future reference.

• Provide support and troubleshoot issues related to Cloudera tools and big data applications.

9. Innovation and Optimization:

• Stay updated on emerging technologies and best practices in the big data ecosystem.

• Continuously seek ways to improve data architecture, performance, and reliability

10. Required competency:

• Tech Savvy

• Communicate Effectively

• Optimizes work Process

—
You received this message because you are subscribed to the Google Groups “sys1point” group.
To unsubscribe from this group and stop receiving emails from it, send an email to sys1point+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/sys1point/CANi42trD3LEmmYyu2u%2BFUOVTV_vh1h2myfB_4OxS5U%2BfniVjyA%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.