DVT is a software development, technology consulting and related services company, with offices in Cape Town, Johannesburg, Durban, UK, Kenya, Ireland and Switzerland with over 600 staff.
We employ highly - skilled individuals in the fields of Software Development (.Net, Mobile and Java), Business Intelligence, Business Analysis, Agile Consulting, Solution Architecture and Project Management.
We develop turnkey solutions and provide consulting services to over 100 clients, both locally and abroad.DVT is currently on a recruitment drive for experienced Data Engineers, Data Science, and Machine Learning Engineers at all levels.
The Senior Data Engineer will play a crucial role in designing, developing, and maintaining scalable data solutions and infrastructure for our clients.
This position requires a deep understanding of data engineering principles, technologies, and tools, as well as strong problem-solving skills.
The Senior Data Engineer will collaborate closely with cross-functional teams to ensure data pipelines are efficient, reliable, and aligned with business objectives within our clients.DVT BenefitsAccess to International Clients and ProjectsMedical Aid and Life Cover through DiscoveryAccess to Discovery's Healthy Company ProgrammeMentorship Platform for technical and non-technical skillsHigh potential program for career acceleration for key employeesYearly Personalized Personal Development Plans & Detailed Career PathsAccess to Udemy for businessPaid for Training Programs and CertificationsPersonalized Business CoachingAccess to DVT's Innovation Center and Innovation Teams & ProjectsAccess to the Databricks Learning AcademyJob Description: Senior Data EngineerPosition OverviewWe are seeking a highly skilled and experienced Senior Data Engineer to join our dynamic team.
The ideal candidate will have a strong technical background in big data technologies, cloud-based data solutions, and data pipeline orchestration.
This role requires excellent leadership abilities, analytical skills, and a passion for staying up-to-date with the latest trends in data engineering.Technical KnowledgeProficient in big data technologies such as Apache Hadoop, Spark, and Kafka.Strong programming skills, particularly in Python, Java, and SQL.Solid experience in developing and optimizing data pipelines, architecture, and data sets.Solid understanding of data warehousing concepts, ETL tools, and data modeling.Good experience with cloud-based data solutions like AWS (EC2, S3, EMR, Redshift, Glue), Google Cloud (BigQuery, Dataflow, Pub/Sub).Familiarity with data pipeline orchestration and workflow management tools: Airflow, Luigi.Familiarity with containerization and orchestration platforms: Docker, Kubernetes.Bonus point for Databricks ExperienceBehavioral CompetenciesShows promising leadership abilities.Excellent analytical and problem-solving skills.Ability to work in a fast-paced, dynamic, and collaborative environment.Comfortable with ambiguity and able to adapt to change.Ability to work independently and as part of a team.Passion for learning new technologies and staying up-to-date with the latest trends in data engineering.Excellent written and verbal communication and collaboration skills.Strong business acumen.ResponsibilitiesWork independently on complex data engineering projects.Develop and maintain optimal data pipeline architecture.Collaborate with data scientists and business analysts to meet functional data requirements.Assist the Lead Data Engineer in planning and implementing data engineering projects and be involved in project management and strategic decision-making.Optimize and tune data pipelines for performance, scalability, and cost-effectiveness.Ensure data quality and implement data validation and cleansing processes.Monitor and troubleshoot data pipelines to identify and resolve issues promptly.Evaluate and recommend appropriate data storage and processing technologies.Develop and maintain documentation for data engineering processes, data flows, and system architectures.Stay updated with emerging technologies, industry trends, and best practices in data engineering and analytics.Provide technical guidance and mentorship to junior data engineers.Minimum Experience Required6+ years of experience in data engineering, with a focus on building large-scale data processing systems.Bachelor's or Master's degree in Computer Science, Information Systems, or a related field.Bonus points for Databricks Certifications
#J-18808-Ljbffr