Our UK based client is looking for an AWS Data Engineer work out of their Edinburgh office. The role includes crafting, building, deploying, and maintaining a robust automated test framework for their innovative cloud platform.
Job Detail:
As an AWS Data Engineer, you will design, develop, test, and deploy solutions for strategic projects. Your focus includes delivering high-performance systems, creating frameworks, and seamlessly transitioning to a DevOps lifecycle. Leverage your expertise to mentor, contribute direct work, and provide leadership in collaboration with business stakeholders.
Deliverables:
- Data Pipelines using Python, SQL, Lambda, and Step Functions.
- Data Models that are optimised for business usability and comprehension.
- Develop and optimise data tables for performance using best practices for partitioning, parallelisation.
- Develop and maintain metadata to augment their data catalogue and support the creation of documentation to be used by the client’s technical and business customers.
- Collaborate with internal business customers in the troubleshooting and optimisation of SQL and reporting solutions to solve and improve business outcomes.
Experience Required:
Essential:
- Previous experience in the Banking, Financial Asset Management, or a highly regulated environment is necessary.
- Proven Agile/Scrum experience, delivering in sprints and collaborating effectively.
- AWS Serverless technologies must include but not limited to:
- Cognito, Lambda, Amazon Athena, Sage Maker, EventHub, API Gateway, AWS Glue.
- Proficiency in JDBC/ODBC, SQL query processing, and distributed query engines.
- Current experience with Software Development Lifecycle (SDLC) covering Coding Standards, Code Reviews, Source Control (Bitbucket/GitHub), Build Processes, Testing, and Operations.
Desired:
- Having experience in data warehousing and data modelling is advantageous.
- A background in Big Data or Machine Learning is an asset.
- Experience in consultancy work is a plus.
- Experience with the following Azure solutions: Azure Data Factory, Azure Synapse Analytics, Azure Stream Analytics, Azure Event Hubs, Azure Data Lake Storage, Azure Databrick.
You should be proficient in using the following to create data processing solutions:
- Azure Data Factor and/or AWS Data Pipeline
- Azure Synapse Analytics and/or Amazon Redshift or Amazon Athena
- Azure Stream Analytics and/or Amazon Kinesis Analytics
- Azure Event Hubs and/or Amazon Kinesis
- Azure Data Lake Storage and/or Amazon Simple Storage Service (S3) with appropriate configurations
- Azure Databrick and/or Amazon EMR (Elastic MapReduce)
Download
Job Profile