Azure Data Engineer - Calgary, Canada - Integriti Group Inc

Description

Requirements:

• You'll have experience in a Data Engineer role (5+ years), with a Graduate degree in Computer Science, Information Systems, Electrical Engineering or another quantitative field
• You build and maintain optimal data pipeline architecture
• You assemble large, complex data sets that meet functional / non-functional business requirements.
• You build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL, DataBricks, No-SQL
• Develop highly scalable architecture leveraging modern techniques to maximize cost reductions
• You identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, data quality checks, minimize Cloud cost, etc.
• You build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics.
• You document and communicate standard methods and tools used.
• You work with other data engineers, data ingestion specialists, and experts across the company to consolidate methods and tool standards where practical.
• Youre experienced using the following software/tools:
• Big data tools: Hadoop, HDI, & Spark
• Relational SQL and NoSQL databases, including COSMOS
• Data pipeline and workflow management tools: DataBricks (Spark), ADF, Dataflow
• Microsoft Azure
• Stream-processing systems: Storm, Streaming-Analytics, IoT Hub, Event Hub
• Object-oriented/object function scripting languages: Python, Scala, SQL