DVT is a software development, technology consulting and related services company, with offices in Cape Town, Johannesburg, Durban, UK, Kenya, Ireland and Switzerland with over 600 staff. We employ highly - skilled individuals in the fields of Software Development (.Net, Mobile and Java), Business Intelligence, Business Analysis, Agile Consulting, Solution Architecture and Project Management. We develop turnkey solutions and provide consulting services to over 100 clients, both locally and abroad.
DVT is currently on a recruitment drive for experienced Data Engineers, Data Science, and Machine Learning Engineers at all levels. The Senior Data Engineer will play a crucial role in designing, developing, and maintaining scalable data solutions and infrastructure for our clients. This position requires a deep understanding of data engineering principles, technologies, and tools, as well as strong problem-solving skills. The Senior Data Engineer will collaborate closely with cross-functional teams to ensure data pipelines are efficient, reliable, and aligned with business objectives within our clients.
DVT Benefits
Access to International Clients and Projects
Medical Aid and Life Cover through Discovery
Access to Discovery's Healthy Company Programme
Mentorship Platform for technical and non-technical skills
High potential program for career acceleration for key employees
Yearly Personalized Personal Development Plans & Detailed Career Paths
Access to Udemy for business
Paid for Training Programs and Certifications
Personalized Business Coaching
Access to DVT's Innovation Center and Innovation Teams & Projects
Access to the Databricks Learning Academy
TECHNICAL KNOWLEDGE
Proficient in big data technologies such as Apache Hadoop, Spark, and Kafka.
Strong programming skills, particularly Python, Java, and SQL.
Solid experience in developing and optimizing data pipelines, architecture, and data sets.
Solid understanding of data warehousing concepts, ETL tools, and data modelling.
Good experience with cloud-based data solutions like AWS (EC2, S3, EMR, Redshift, Glue), Google Cloud (BigQuery, Dataflow, Pub/Sub).
Data pipeline orchestration and workflow management tools: Airflow, Luigi.
Familiarity with Containerization and orchestration platforms: Docker, Kubernetes.
BEHAVIOURAL COMPETENCIES
Shows promising leadership abilities.
Excellent analytical and problem-solving skills
Ability to work in a fast-paced, dynamic, and collaborative environment.
Comfortable with ambiguity and able to adapt to change.
Ability to work independently and as part of a team.
Passion for learning new technologies and staying up to date with the latest trends in Data Engineering.
Excellent written and verbal communication and collaboration skills
Strong business acumen.
RESPONSIBILITIES
Works independently on complex data engineering projects.
Develop and maintain optimal data pipeline architecture.
Collaborate with data scientists and Business Analysts to meet functional data requirements.
Assist the Lead Data Engineer in planning and implementing data engineering projects and be involved in project management and strategic decision-making.
Optimize and tune data pipelines for performance, scalability, and cost-effectiveness.
Ensure data quality and implement data validation and cleansing processes.
Monitor and troubleshoot data pipelines to identify and resolve issues promptly.
Evaluate and recommend appropriate data storage and processing technologies.
Develop and maintain documentation for data engineering processes, data flows, and system architectures.
Stay updated with emerging technologies, industry trends, and best practices in data engineering and analytics.
Provide technical guidance and mentorship to junior data engineers.
MINIMUM EXPERIENCE REQUIRED
4+ years of experience in data engineering, with a focus on building large-scale data processing systems.
Bachelor's or Master's degree in Computer Science, Information Systems, or a related field.
Who we are:
#J-18808-Ljbffr
