Spark Developer (Big Data)
Designs limited enhancements, updates, and programming changes for portions and subsystems of data pipelines, repositories or models for structured/unstructured data.
Analyzes design and determines coding, programming, and integration activities required based on specific objectives and established project guidelines.
Executes and writes portions of testing plans, protocols, and documentation for assigned portion of application; identifies and debugs issues with code and suggests changes or improvements.
Participates as a member of a project team of other data science professionals to develop reliable, cost effective and high-quality solutions for assigned data system, model, or component.
Collaborates and communicates with project team regarding project progress and issue resolution.
Knowledge & Skills
Using data engineering tools, languages, frameworks to cleanse, mine and explore data.
Understanding of NoSQL & relational based systems along with complex, distributed and massively parallel systems.
Ability to apply analytical and problem-solving skills.
Ability to understand complex data structures.
Understanding of database technologies, management systems, and cloud based systems/services.
Database architecture testing methodology, including execution of test plans, debugging, and testing scripts and tools.
Strong written and verbal communication skills; mastery in English and local language.
Ability to effectively communicate design proposals and negotiate options.
Scope & Impact
Collaborates with peers, junior engineers, data scientists and project team.
Typically partners with high-level Individual Contributors and Managers.
Supports projects requiring data engineering solutions expertise.
Education & Experience
Bachelor’s or Master’s degree in Computer Science, Information Systems, Engineering, or equivalent.
Typically 2-4 years’ experience.