Big Data Engineer

Location: Herndon, VA, United States
Date Posted: 05-20-2016
CLIENT OVERVIEW
Our client is utilizing cutting edge open source technologies and frameworks to derive actionable insights from massive amounts of data. Our solution provides capability to ingest petabytes of structured and unstructured data and build analytic pipelines that crunch through this data. 

JOB OVERVIEW
We are looking for a Big Data Engineer that will utilize our base platform to develop ingestion and analytic pipelines utilizing a multitude of open source technologies. The primary focus will be on implementing optimal solutions utilizing best practices that can be shared across all our clients. This position requires working with technical leads, data scientists and project managers in a Scrum based Agile environment. 

JOB RESPONSIBILITIES
  • Create, configure, implement, document and maintain ingestion, enrichment and analytic pipelines using distributed big data platform
  • Extend base platform functionality by adding new ingestion and analytic sources
  • Identify, evaluate and implement big data tools and frameworks required to provide requested capabilities

JOB EXPERIENCE REQUIRED
  • Java development experience
  • Scripting language experience (Perl, Python, JavaScript)
  • Proficient understanding of distributed computing principles
  • Proficiency with Hadoop v2, MapReduce, HDFS
  • Experience with building stream-processing systems, using solutions such as Storm or Spark 
  • Experience with integration of data from multiple data sources
  • Experience with NoSQL databases, such as Elasticsearch, MongoDB, Cassandra
  • Knowledge of various ETL techniques and frameworks, such as Flume, Logstash
  • Experience with various messaging systems, such as Kafka or RabbitMQ
  • Experience with Cloudera/Hortonworks distributions
  • Bachelor's degree or equivalent professional experience
JOB EXPERIENCE DESIRED 
  • Good knowledge of Big Data querying tools, such as Pig, Hive, and Impala
  • Management of Hadoop cluster, with all included services 
  • Ability to solve any ongoing issues with operating the cluster 
 www.turasgroup.com 
 
or
this job portal is powered by CATS