Data Engineer

Details of the offer

This roles responsibility is to design, develop and maintain data-based solutions including ensuring that the operationalization of data pipelines and data stores are high-performing, efficient, organized, and reliable, given a set of business requirements and constraints.The Data Engineer will build and maintain secure and compliant data processing pipelines by using different tools and techniques, maintain various data services and frameworks to store and produce cleansed and enhanced datasets for analysis. This includes data store design using different architecture patterns based on business requirements.You will help to identify and troubleshoot operational and data quality issues, and also design, implement, monitor, and optimize data platforms to meet the data pipelines. Collaborate with cross-functional teams (internal and external) to ensure effective implementation, set-up, utilization and governance of the enterprise data platform across the organization.This role contributes to the complete data function for data analytics, business intelligence, and advanced analytics, and requires a strong business focus, and the individual understands the strategy and direction of the business but focuses on how to underpin that with data.REQUIRED MINIMUM EDUCATION/TRAININGBCom or Bachelors degree in Information Technology / Computer Science / Engineering (IT) or similarData Engineering tools (Azure Data Engineer) certification will be an added advantageITIL or COBIT certification will be an added advantageREQUIRED MINIMUM WORK EXPERIENCE2 - 4 years of working experience in an IT environmentExperience in working on IT enterprise Azure-based data solutionsExperience with Python and SQLExperience in gathering requirements and transitioning them to designs and outcomesExperience working in the Agricultural sector will be an advantageKEY PERFORMANCE AREASEnsure scalability of solutionsContinuously improve business data environment design and efficienciesDeliver business benefits from advanced analyticsEnsure the managed data (business intelligence) is fit for purpose relative to customer needsDesign and develop end-to-end data solutions on the Data platform, ensuring optimal performance, scalability, and reliability.Collaborate with data architects, data analysts, and business stakeholders to understand data requirements, define data models, and implement data integration solutions.Implement data pipelines, data lakes, and data warehouses using.Develop and maintain ETL/ELT processes to extract, transform, and load data from various sources into Azure data platforms.Design and optimize data schemas, tables, and indexes for efficient data storage and retrieval.Perform data profiling, data cleansing, and data quality assessments to ensure data accuracy and consistency.Develop and implement data security and privacy measures, ensuring compliance with relevant regulations (e.g., POPIA).Monitor and troubleshoot data pipelines, identifying and resolving performance bottlenecks, data inconsistencies, and system issues.Collaborate with DevOps teams to implement CI/CD pipelines and ensure smooth deployment and release management of data solutions.Stay up to date with the latest data technologies, trends, and best practices, and provide recommendations for continuous improvement.TECHNICAL KNOWLEDGE / COMPETENCIESData and Business intelligence enablement knowledgeBusiness intelligence rendering toolsWorking knowledge of Project Management ToolsExcellent verbal and written communication skillsIT Security would be an added advantageBEHAVIOURAL COMPETENCIESAnalytical abilityAbility to tie together technologies and architectures to address requirements
#J-18808-Ljbffr


Nominal Salary: To be agreed

Source: Whatjobs_Ppc

Requirements

Operations Support Engineer Or Specialist

Operations Support – Modeling and Engineering Engineer/Specialist/Analyst Description The Modeling and Engineering Support Engineer/Specialist/Analyst positi...


Southern Company - Gauteng

Published a month ago

Systems Analyst (Payments)

Systems Analyst (Payments)Apply locations Randburg Johannesburg posted on Posted 3 Days Ago time left to apply End Date: January 31, 2025 (30+ days left to a...


Rmbwestport - Gauteng

Published a month ago

Head Infrastructure & Enterprise Architecture

Position summaryIndustry: Scientific, Research & Development Job category: IT-Architecture Location: Randburg Contract: Permanent Remuneration: Market Relate...


Mintek Pty Ltd. - Gauteng

Published 13 days ago

Specialist Cybersecurity Incident Management Nexio Midrand, Gauteng

Specialist Cybersecurity Incident Management Nexio Midrand, Gauteng Permanent Closing Date 13 December 2024Job Details Division: Solutions Minimum experience...


Info Desk - Gauteng

Published a month ago

Built at: 2024-12-24T00:54:16.630Z