About the job Intermediate Data EngineerResponsibilities:Develop algorithms to transform data into useful, actionable informationDesign, construct, install, test and maintain highly scalable data management systemsEnsure systems meet business requirements and industry practicesBuild high-performance prototypes and proof of conceptsResearch opportunities for data acquisition and new uses of existing dataDevelop data set processes for data modelling, mining and productionIntegrate new data management technologies and software engineering tools into existing structuresCreate custom software components (e.g., specialized UDFs) and analytics applicationsEmploy a variety of languages and tools (e.g., scripting languages) to marry systems togetherInstall and update disaster recovery proceduresRecommend ways to improve data reliability, efficiency and qualitySupport the achievement of the business strategy, objectives and valuesParticipate and support corporate responsibility initiatives for the achievement of business strategyBuild, test, and maintain database pipeline architecturesBuild data products for the data analytics team to ingest and train models onEnsure data systems (e.g., Clickhouse, MySQL) are functioning correctly and performing to expectationsCollaborate with management to understand company objectivesCreate new data validation methods and data analysis toolsEnsure compliance with data governance and security policiesDocument all created artifacts, create ERDs and any other needed documentation to ensure data analytics are working at capacityExplore new technology and look into new ways of reaching specific objectivesWork closely with our in-house team as well as outsourced service providers and partnersRequirements:Degree / Diploma in Data Engineering preferred but not required2+ years of cloud experience3+ years of experience building, maintaining and administering a DBMS, such as MySQL or Postgres2+ years experience in SQL development, including writing CTEs, and creating schemas and tablesA successful history of manipulating, processing and extracting value from large disconnected datasetsStrong analytic skills related to working with unstructured datasetsPython ORM programming experience (SQLalchemy) would be a bonusProficiency in both written and spoken EnglishA fundamental understanding of Basic Machine LearningKnowledge about server and data security on the web
#J-18808-Ljbffr