Hadoop Developer
Eden Prairie, MN
1 year
Required:
Technologies | Mandatory(M) |
Java | Good to Have |
SQL | M |
Datastage | M |
Hadoop | M |
Map Reduce | Good to Have |
Hive | M |
Pig | M |
Sqoop and MRUnit | Good to Have |
Falcon | Good to Have |
UDF defined functions/ PIGUDF | M |
HDFS file system | M |
Shell scripting | M |
Python | Good to Have |
Tivoli Work Scheduler or any scheduling Tool | M |
Eclipse | Good to Have |
WebSphere | Good to Have |
Oracle/TeraData or any other Database | M |
Responsibilities:
• Worked on live hadoop cluster running on MapR distribution.
• Involved in creating Hive tables, loading with data and writing hive queries which will run internally in MapReduce way.
• Used Sqoop to ingest data from various source systems into HDFS.
• Written Hive queries for data analysis to meet the business requirements
• Familiarized with job scheduling so that CPU time is well distributed amongst all the jobs.
• Executed queries using Hive and developed Map-Reduce jobs to analyze data.
• Developed Pig Latin scripts to extract the data from the web server output files to load into HDFS.
• Developed Hive queries for the analysts.
• Worked on DataGuise and documented the same for internal users.
• Worked on Elastic search and created various scripts.
• Worked on Datameer and created the shell script to run the Datameer job from the command line.
Environment: Eclipse, Oracle 10g, Hadoop, MapReduce, Hive, Linux, MapReduce, HDFS, Hive, MapR, SQL, Toad 9.6.
Posted by: "Eshwar Yerram" <eshwar@3ktechnologies.com>
| Reply via web post | • | Reply to sender | • | Reply to group | • | Start a New Topic | • | Messages in this topic (1) |
