Code Reviewer – Java (LATAM)

About the Role

We are seeking experienced, tech-lead-level software engineers to review and work with high-quality public GitHub repositories. This short-term contract involves hands-on software engineering, including development environment automation, issue triage, and evaluating test coverage and quality. The focus is on Java and assessing codebases in bug-fixing scenarios for large language models.

Responsibilities

• Analyze and triage GitHub issues across trending open-source libraries.
• Set up and configure code repositories, including Dockerization and development environments.
• Evaluate unit test coverage and quality.
• Modify and run codebases locally to assess LLM performance in bug-fixing scenarios.
• Collaborate with researchers to identify repositories and issues that are challenging for LLMs.

Requirements

• At least 3 years of overall experience.
• Strong experience with Java.
• Proficiency with Git, Docker, and basic software pipeline setup.
• Ability to understand and navigate complex codebases.
• Comfort running, modifying, and testing real-world projects locally.
• Experience contributing to or evaluating open-source projects is a plus.
• Previous participation in LLM research or evaluation, or experience building or testing developer tools and automation agents, is a plus.

Arrangement and Terms

• Fully remote; applicants must be located in LATAM.
• Short-term contractor assignment of approximately 2–4 weeks.
• Requires at least 40 hours per week, with 4 hours of overlap with PST.
• Payment: $25 per approved task; average task handling time is 45–60 minutes.
• 20 positions available.
• Assessment-based review process.