Similar Jobs
See allSWE-Bench AI Task Auditor
Project World Wide
Global
Software Engineering
Analytical Skills
Problem Solving
Python Developer
Anyone AI
Brazil
Python
JavaScript
TypeScript
AI-Assisted Developer Workflows (Trace) AI Task Auditor - Freelance AI Trainer Project
Unknown
Greece
Developer Tooling
AI Coding Assistants
Problem Solving
Sr Software Engineer
Mitratech
US
Go
Java
Python
Junior AI Engineer – AI Code Quality Validation for T Cloud Public
Deutsche Telekom System Solutions Slovakia
Slovakia
Python
AI Coding Assistants
Cloud Platforms
Responsibilities:
- Review coding tasks derived from real GitHub issues and pull requests to assess technical soundness and reproducibility.
- Evaluate unit tests for correctness, coverage, and robustness, identifying flaky tests, missing dependencies, version conflicts, and environment issues.
- Determine whether tasks can be reliably reproduced across environments and provide clear recommendations on acceptance.
Requirements:
- 3+ years of professional software engineering experience, with strong experience in multi-file codebases.
- Strong understanding of unit testing, test coverage, and dependency management.
- Experience with Git and GitHub workflows, and ability to analyze complex technical problems.
Nice to Have:
- Contributions to open-source projects or experience with SWE-Bench benchmarks.
- Experience with Python projects like Django, Flask, scikit-learn, or SymPy.
- Knowledge of Docker, CI/CD, pip, conda, or dependency pinning.
Engagement:
- Work Type: Remote, part-time project-based consulting.
- Focus on software engineering, open-source code review, testing, and technical evaluation.
Anyone AI
Anyone AI is a company that focuses on AI-related projects, particularly in evaluating software engineering tasks. They are a smaller organization with a culture that values technical expertise and attention to detail.