Spark Senior Developer - Mississauga, Canada - Virtusa

    Virtusa
    Virtusa Mississauga, Canada

    1 week ago

    Default job background
    Description
    Spark Senior Developer / Tech Lead - CREQ185961 Description

    Knowledge/Experience (Must):

    · 5-7+ years of experience in Hadoop/big data technologies.

    · Experience with Spark/Storm/Kafka or equivalent streaming/batch processing and event based messaging.

    · Relational and NoSQL database integration and data distribution principles experience

    · Hands-on experience with the Hadoop eco-system (HDFS, MapReduce, Hive, Pig, Impala, Spark, Kafka, Kudu, Solr)

    · Experience with API development and use of JSON/XML/Hypermedia data formats.

    · Strong development/automation skills

    · Experience with all aspects of DevOps (source control, continuous integration, deployments, etc.)

    · 3+ years of hands-on experience as a Scala developer (with previous Java background)

    Knowledge/Experience (Preferred):

    · Experience with containerization and related technologies ( Docker, Kubernetes)

    · Comprehensive knowledge of the principles of software engineering and data analytics

    · Knowledge of agile(scrum) development methodology is a plus

    · Cloudera/Hortonworks/AWS EMR, S3 experience a plus

    Primary Location Mississauga, Ontario, Canada Job Type Experienced Primary Skills Docker, Kubernetes, PySpark, Kafka, Java, HDFS Years of Experience 7 Qualification

    Knowledge/Experience (Must):

    · 5-7+ years of experience in Hadoop/big data technologies.

    · Experience with Spark/Storm/Kafka or equivalent streaming/batch processing and event based messaging.

    · Relational and NoSQL database integration and data distribution principles experience

    · Hands-on experience with the Hadoop eco-system (HDFS, MapReduce, Hive, Pig, Impala, Spark, Kafka, Kudu, Solr)

    · Experience with API development and use of JSON/XML/Hypermedia data formats.

    · Strong development/automation skills

    · Experience with all aspects of DevOps (source control, continuous integration, deployments, etc.)

    · 3+ years of hands-on experience as a Scala developer (with previous Java background)

    Knowledge/Experience (Preferred):

    · Experience with containerization and related technologies ( Docker, Kubernetes)

    · Comprehensive knowledge of the principles of software engineering and data analytics

    · Knowledge of agile(scrum) development methodology is a plus

    · Cloudera/Hortonworks/AWS EMR, S3 experience a plus

    Travel No