
aidan west
Engineering / Architecture
About aidan west:
3 years’ experience building distributed systems, CI/CD pipelines, and ETL workflows. Proficient in cloud computing (GCP, Azure), programming (Go, Python, Bash), and data modelling with SQL (Postgres, BigQuery). Experience with Infrastructure as Code (Terraform), networking (Wireguard, Tailscale), virtualisation (Proxmox), and containerization (Docker).
Experience
- Designed and built a pipeline to extract, decompress and load multiple terabytes of data from Google Cloud Storage to Google BigQuery. Orchestrated in Google Workflows, with Cloud Functions written in Python and a Firestore database to track state. The distributed architecture allowed for a ~40% reduction in processing time, whilst ensuring idempotence and zero duplication.
- Designed and implemented a CI/CD (continuous-development-continuous-integration) pipeline to automate the provisioning of GCP resources. Successful pull requests to Git branches would trigger Cloud Build to run Terraform commands. This enabled the creation of production and testing environments, ensuring that deployments to production would be safe.
- Oversaw a cloud-to-hybrid migration, integrating a newly acquired on-premises server into our infrastructure. Administered with Proxmox. Set-up a Postgres server and secured database access by routing all clients through a proxy. This was undertaken to optimise costs and enhance control over internal data storage.
- Integrated Evidence for cost-free data exploration, reducing BigQuery query spikes and cutting cloud costs by 14%.
- Used Go to write an application to detect unexpected shifts in model outputs after quarterly input updates, ensuring consistency. This reduced detection time by ~11% and enabled faster error resolution.
- Built a containerized infrastructure automation pipeline to provision Proxmox VMs using Terraform and cloud-init. Configured cloud-init for automatic VM enrolment into the company tailnet, enabling access via Tailscale SSH. This streamlined deployment by eliminating the need for manual SSH key management.
- Elevated the technical proficiency of the analytics team through the creation and delivery of Python training sessions, enhancing their programming capabilities. This allowed them to better leverage our data.
Education
Durham University, England - MSc Data Science – Distinction - OCTOBER 2020 - OCTOBER 2021
- Dissertation focussed on training an Optical Character Recognition (OCR) model to digitise and analyse foreign office documents.
Durham University, England - BA History - Upper Second Class Honours - SEPTEMBER 2017 - JUNE 2020
- Final year project focussed on mapping the shifting territories of the Dukes of Burgundy, using Python as tool to collate and map data.
Professionals in the same Engineering / Architecture sector as aidan west
Professionals from different sectors near Toronto, Ontario
Other users who are called aidan
Jobs near Toronto, Ontario
-
We are seeking a Senior Data Engineer to join our Retail Media Platform team in building a best-in-class Ad tech solution. · Analyze data to uncover patterns or issues, · build data pipelines and queries that will aggregate the reporting metrics needed in the Retail Media Platfor ...
Toronto2 weeks ago
-
We are seeking a highly skilled Data Engineer who also possesses strong Quality Assurance (QA) knowledge. This role is primarily focused on data engineering and development work, but the candidate will also support QA activities to ensure the delivery of high-quality data solutio ...
Toronto2 weeks ago
-
Data Engineer on the WELL Intelligence Platform will design build and operate reliable secure scalable data pipelines. · ...
Toronto3 weeks ago