Hadoop Developer

We are seeking a skilled Hadoop Developer to join our data engineering team. The ideal candidate will have experience in building, managing, and optimizing large-scale Hadoop clusters. You will work on designing and developing solutions that allow for efficient data processing, transformation, and storage. This role requires a deep understanding of Hadoop ecosystem components and their integration with other data processing tools.Key Responsibilities:Hadoop Development: Design, develop, and maintain Hadoop applications and solutions using various tools within the Hadoop ecosystem, such as HDFS, MapReduce, Pig, Hive, HBase, and others.Data Processing: Build and optimize data processing pipelines using Hadoop technologies to handle large volumes of structured and unstructured data.Data Integration: Integrate Hadoop applications with other data sources, databases, and data warehouses.Performance Optimization: Optimize Hadoop clusters for performance, reliability, and scalability, including tuning MapReduce jobs, HDFS, and other Hadoop components.Data Security: Implement data security and encryption within Hadoop clusters to ensure data protection and compliance with regulatory requirements.Collaboration: Work closely with data scientists, analysts, and other developers to understand data requirements and provide appropriate Hadoop-based solutions.Monitoring and Troubleshooting: Monitor Hadoop clusters and applications, troubleshoot issues, and ensure high availability and reliability of the system.Documentation: Create and maintain technical documentation related to Hadoop processes, systems architecture, and workflows.Required Skills and Qualifications:Education: Bachelor's or master's degree in computer science, Information Technology, Engineering, or a related field.Experience:Minimum of 5 years of experience in Hadoop development.Hands-on experience with Hadoop components such as HDFS, MapReduce, Hive, Pig, HBase, and YARN.Experience with data integration tools like Sqoop, Flume, or Kafka.Strong experience with scripting languages like Python, Perl, or Shell scripting.Proficiency in SQL and NoSQL databases.
#J-18808-Ljbffr