Job Title: Hadoop Admin
Duration: 12 Months
Interview: Telephonic & Skype
Experience: 10+ Years’ experience
Tasks and Responsibilities:
Partners with Infrastructure to identify server hardware, software and configurations necessary for optimally running big data workloads (i.e. Spark, Hive, Impala, etc.)
• Plans and executes major platform software and operating system upgrades and maintenance across physical and virtualized environments.
• Work with project teams to integrate Hadoop access points.
• Strong focus on design, build, deployment, security, system hardening and securing services.
• Designs and implements a toolset that simplifies provisioning and support of a large cluster environment.
• Maintains current knowledge of industry trends and standards in a “Big Data” space/ecosystem.
• Creates and maintains detailed up-to-date technical documentation.
• Proactively manages hadoop system resources to assure maximum system performance and appropriate additional capacity for peak periods and growth.
• Reviews performance stats and query execution/explain plans, and recommends changes for tuning Hive/Impala queries.
• Recommends security management best practices including the ongoing promotion of awareness on current threats, auditing of server logs and other security management processes, as well as following established security standards.
• Provides support to the user community using incident and problem management tools, email, and voicemail.
• Periodic off-hours work required including weekends and holidays. Must be able to provide 24 by 7 on-call support as necessary.
• Assists in other areas of the department and company as necessary.
Required Skills, Abilities, and Certifications:
• Understanding of Hadoop and distributed computing.
• Understanding of Hadoop components (i.e. HDFS, YARN, Zookeeper, Sqoop, Hive, Impala, Hue, Sentry, Spark, Kafka, flume, etc.)
• Minimum of 2 years’ experience supporting products running on AIX, Linux or other UNIX variants.
• Experience with supporting production JVMs.
• Knowledge of Relational Databases (DB2, Oracle, SQL Server, DB2 for iSeries, MySQL, Postgres, MariaDB).
• Configuration Management experience (puppet preferred).
• Version control experience (Unix, git, git preferred).
• Proficient with operating system utilities as well as server monitoring and diagnostic tools to troubleshoot server software and network issues.
• Deep knowledge of associated industry protocol standards such as: LDAP, DNS, TCP/IP, etc.
• Understanding of Enterprise level services – Active Directory, PKI Infrastructure (Venafi).
• Understanding of Kerberos, cross-realm authentication and Kerberized services.
• Security experience including SSL certificates, TLS, hardening, PEN tests, etc.
• Understanding of message based architecture.
• Flexibility is a non-negotiable skill that the ideal candidate must have.
Recommended Skills, Abilities, and Certifications:
• Experience with Systems Management and Administration (applying fixes, loading the OS, creating Gold images, resolving system issues, working with vendors to resolve issues, etc.)
• Experience with VMWare (vSphere), virtualized environments and/or cloud providers such as Microsoft Azure, AWS, etc.
• Experience with Tivoli Storage Management and/or Commvault.
• Experience in System/Database Backup and Recovery.
• B.S. degree in Computer Science or equivalent formal training and experience.