Responsibilities:

1. Design, develop, and maintain Java-based backend applications, ensuring scalability, reliability, and performance.
2. Utilize Apache Spark to process and analyze large volumes of data efficiently, implementing complex data processing pipelines.
3. Collaborate with cross-functional teams to gather requirements, design solutions, and integrate backend systems with other components of the software ecosystem.
4. Optimize Spark jobs and backend processes for maximum performance and resource utilization, while adhering to best practices and industry standards.
5. Implement data validation, cleansing, and transformation techniques to ensure the quality and integrity of the data processed by the system.
6. Troubleshoot and debug issues in backend code and Spark jobs, identifying root causes and implementing effective solutions.
7. Stay updated with the latest developments in Java, Spark, and Big Data technologies, and apply relevant advancements to enhance the existing systems.
8. Participate in code reviews, providing constructive feedback to peers and contributing to the overall improvement of code quality and maintainability.
9. Document design decisions, technical specifications, and implementation details for reference and knowledge sharing within the team.
10. Proactively identify areas for optimization, automation, and process improvement within the backend development and Big Data processing workflows.