Relevant 3-year tertiary qualification BSC In Computer Science / Information SystemsIn depth knowledge and understanding of:Data governance and data management frameworksBI and Warehouse designMovement of data into the Cloud using the following tools Apache Kafka, Storm, Flume for ingesting data or Amazon Web Services (AWS) Cloud Development Kit (CDK)Operating with real-time streams, data warehouse queries, JSON, CSV, raw dataScripting Data Pipelines for scheduled movement of dataDesigning and developing ETL/ELT processes and data pipelines Experience working with Azure DevOpsExploratory Data Analysis - EDASQLMicrosoft SQL ServerNoSQLMicrosoft SQL ServerPython (PySpark)C# , Json- calling APIsPowerShellKafkaScalaSplunkElk StackData warehousing solutionsData CleansingTerraformResponsibilities:Work as part of an agile data engineering teamEnsure that data systems meet the companys regulatory commitments, and that data is reliable and efficientSupport best business capabilities for high performance database solutioningActively participate in team, cross-discipline and vendor-driven collaboration sessions or forums to increase understanding of the working environment by contribution and participation in the relevant Data Governance FrameworksPartner with Technology, Business and other stakeholders to ensure that database structures, programmes and applications meet business delivery requirementsMonitor adherence to processes which support the prescribed data architectural frameworks and ensure development/delivery teams align to the required standards and methodologiesDesign and implement scalable end-to-end database solutions including:Addressing issues of data migration i.e.
validation, clean-up and mapping and consistently apply data dictionariesComponent design and developmentIdentification and resolution of production and application development constraintsIntegration of new technologies and software into the existing landscapeDevelopment data set processes for data modelling, mining and productionOptimal utilization of big dataDocumentation management and disaster recovery
#J-18808-Ljbffr