Data Engineer - Toronto - Soar Consultants

    Soar Consultants
    Soar Consultants Toronto

    1 week ago

    Description

    DESCRIPTION

    We are currently looking for a Data Engineer for a client in Toronto. They are looking for someone with experience working with Azure, DataBricks and experience working in the Banking/Finance sectors.

    RESPONSIBILITIES

    • Analyze and understand business requirements related to data and information, with a particular focus on the financial services industry.
    • Design data integration and ETL processes using tools such as Databricks, Spark, Hadoop, and Hive, following a holistic, scalable, pragmatic, and effective approach to data processing.
    • Ingest and transform data in Databricks for batch and near real-time processing using technologies such as PySpark and SparkSQL, while leveraging orchestration tools like Azure Data Factory within the broader Microsoft Azure ecosystem. Design and implement data pipelines integrated with Continuous Integration/Continuous Delivery (CI/CD) frameworks as required (DevOps).
    • Develop automated unit-testing to ensure data quality and reliability.
    • Develop design and operational documentation for job schedules and data pipelines.
    • Produce estimates for data analysis and mapping, design, development, and testing.
    • Ensure compliance with business, data, and technical requirements.
    • Comply with client's enterprise architecture standards, policies, and procedures are followed.

    REQUIREMENTS

    • Minimum 5 years experience in Information Technology, participating in complex projects.
    • Minimum Bachelors degree in Computer Science.
    • Strong communication skills.
    • Strong knowledge in data analysis, database development, big data, data lakehouse, data warehousing life cycle and data integration methodologies.
    • Strong SQL knowledge of using in Databricks workspace and Delta Lake features.
    • Strong programming skills in Pyspark and Apache Spark Architecture.
    • Experience with DLT for building reliable and maintainable data pipelines for streaming data using Databricks Structured Streaming.
    • Strong understanding of at least one major cloud provider (e.g., AWS, Azure, or GCP) and its data-related services is essential. This includes knowing about services for data storage (like S3 or Azure Blob Storage) and how they integrate with Databricks.
    • Designing schemas for databases and understanding concepts like the medallion architecture (Bronze, Silver, Gold layers) and slowly changing dimensions (SCD).
    • Familiarity with concepts of data quality, security, and governance, often utilizing Databricks's Unity Catalog.
    • Knowledge of ETL development using SSIS, Informatica, Data Stage.
    • Knowledge of and experience with Copilots and LLMs to gain higher productivity.
    • Strong knowledge of Oracle, Microsoft SQL databases.
    • Knowledgeable in both relational and dimensional data modeling (with both Kimball and Inmon approaches).
    • Experience in .NET development and UNIX shell scripting (nice to have).
    • Good overall business knowledge in financial industries (i.e. one or more in retail banking, commercial banking, capital markets, wealth management, insurance, pension fund, and fintech).
    • Ability to work independently and excel in a team environment.
    • Preferably as Databricks Certified Data Engineer Associate.

    #J-18808-Ljbffr

  • Data Engineer

    1 month ago

    Only for registered members Toronto, ON

    We are seeking a skilled Data Engineer to build maintain and optimize data pipelines and data platforms. · ...

  • Data Engineer

    4 days ago

    Only for registered members Toronto, ON

    + Job summary · As a Data Engineer in our Advanced Analytics team, your primary focus will be on ensuring the stability, reliability and performance of ML models in production environments. · + Design develop and manage ETL data pipelines that facilitate the detailed extraction t ...

  • Senior Data Engineer Consultant

    3 weeks ago

    Only for registered members Toronto, Ontario

    +Réjoignez KEYRUS et fañonner l'avenir de la data avec nous · +Comprendre les besoins techniques des clients afin de concevoir des solutions data performantes et évolutives · Concevoir, développer et optimiser des pipelines de données et architectures lakehouse (Databricks, Fabri ...

  • Data Engineer

    1 month ago

    Only for registered members Toronto, ON Remote job

    We are looking for a Data Engineer experienced with the modern data stack to design and scale our data architecture. · ...

  • Data Engineer

    3 weeks ago

    Only for registered members Toronto, ON

    We're looking for a motivated Data Engineer to help build and scale our cloud-native data pipelines using Snowflake and dbt. · Design, develop, and maintain ELT pipelines using dbt and Snowflake. · Build and support data ingestion processes from internal and external sources usin ...

  • Data Engineer

    1 month ago

    Only for registered members Toronto, ON

    We are hiring a Data Engineer to help us make data driven decisions. · Owner of the core data pipelines in mapping · Evolving data model and schema based on business needs · ...

  • Data Engineer

    1 week ago

    Only for registered members Toronto Full time

    We are seeking an experienced Data Engineer with expertise in Dataiku to join our data team. · 8+ years of overall industry experience specifically in data engineeringSolid understanding of ETL processes and toolsFamiliarity with cloud platforms like AWS, Snowflake, dbt, Azure or ...

  • Data Engineer

    1 month ago

    Only for registered members Toronto, Ontario

    As Senior Data Engineer you will be key technical contributor responsible for designing building and maintaining robust scalable data pipelines analytics platforms. · ...

  • Data Engineer

    1 month ago

    Only for registered members Toronto, Ontario

    We are seeking a skilled and motivated Data Engineer to join our team. · ...

  • Data Engineer

    1 month ago

    Only for registered members Toronto, Ontario

    Data Engineer required to support platform growth and scalability using MySQL. · Expertise in schema design enhancements · Collaborate with development teams · ...

  • Data Engineer

    3 weeks ago

    Only for registered members Toronto, Ontario

    Data Engineer responsible for designing and developing scalable data pipelines using Databricks (PySpark, Delta Lake, SQL) and building ETL/ELT workflows for structured and unstructured data. · Design, develop and maintain scalable data pipelines using Databricks. · Build ETL/ELT ...

  • Data Engineer

    2 weeks ago

    Only for registered members Toronto, Ontario

    We are looking for a Data Engineer who can not only build and maintain modern data pipelines, but also explain complex technical concepts in a simple, clear way to non-technical audiences. · 3-4 years of experience creating ELT / ETL data pipelines from scratch, · Experience work ...

  • Data Engineer

    5 days ago

    Only for registered members Toronto, Ontario, Canada

    Tiger Analytics is a fast-growing advanced analytics consulting firm. Our consultants bring deep expertise in Data Science, Machine Learning and AI. · ...

  • Data Engineer

    4 weeks ago

    Only for registered members Toronto, Ontario

    Ready to leverage your data engineering expertise to drive transformative projects? Apply Now. · ...

  • Data Engineer

    1 week ago

    Only for registered members Toronto, Ontario

    Dawn InfoTek Inc. is a professional IT consulting team that partners with major financial institutions, investment firms and government sectors, seeking competent individuals to fulfill the role of Data Engineer to join their dynamic team for their client, one of the major banks. ...

  • Data Engineer, Forecasting Core Data Engineering

    4 days ago

    Only for registered members Toronto, ON

    Amazon is looking for a Data Engineer to join the team. · Create scalable data solutions using AWS and/or Amazon internal tools. · Create and maintain business logic for data transformation and ingestion pipelines. · ...

  • Data Engineer

    6 days ago

    Only for registered members Toronto, ON

    +Data engineer will contribute their expertise to Tarion's newly formed digital analytics team while growing with the evolving strategic direction. · ++ · +Design, build and support the data and ETL pipelines that incorporate Tarion's AI governance framework. · +, Design logical ...

  • Data Engineer

    1 month ago

    Only for registered members Toronto, Ontario

    We're building a world-class cloud, data and API engineering firm with a mission to power the most influential tech of tomorrow through expert-led delivery strong partnerships and a relentless focus on quality. · At Tactable this is what drives us We embed with clients work acros ...

  • Data Engineer

    1 month ago

    Only for registered members Toronto, Ontario

    We are seeking a visionary Principal Data Engineer to serve as a technical cornerstone for our AI and Data organization. · Architectural Leadership: Design, build, and maintain robust scalable data architectures that support high-volume data processing. · ...

  • Data Engineer

    1 month ago

    Only for registered members Toronto, Ontario

    We are looking for an experienced Database Engineer to join our Engineering team. · ...

  • Data Engineer

    1 month ago

    Only for registered members Toronto, Ontario

    We are looking for a mid level data engineer to build data-driven engineering initiatives within the financial engineering mission. · ...

Jobs
>
Data engineer
>
Jobs for Data engineer in Toronto