Autograder
The EduLinq Autograder is an open source tool used by several courses at UCSC to safely and quickly grade programming assignments. Grading student code is something that may seem simple at first (you just need to run their code!), but quickly becomes exceeding complex as you get more into the details. Specifically, grading a student’s code securely while providing the “last mile” service of getting code from students and sending results to instructors/TAs and the course’s LMS (e.g., Canvas) can be very difficult. The Autograder provides all of this in a free and open source project. The LINQS Lab has made many contributions to the maintain and improve the Autograder.
As an open source project, there are endless opportunities for development, improvements, and collaboration. Here, we highlight some specific projects that will work well in the summer mentorship setting.
LLM Detection
- Topics:
AI/MLLLMResearchBackend - Skills: software development, backend, systems, data munging, go, docker
- Difficulty: Challenging
- Size: Large (350 hours)
- Mentors: Eriq Augustine, Fabrice Kurmann, Lise Getoor
As Large Language Model (LLM) tools like ChatGPT become more common and powerful, instructors need tools to help determine if students are the actual authors of the code they submit. More classical instances of plagiarism are often discovered by code similarity tools like MOSS. However these tools are not sufficient for detecting code written not by a student, but by an AI model like ChatGPT or GitHub Copilot.
The task for this project is to create a system that provides a score indicating the system’s confidence that a given piece of code was written by an AI tool and not a student. This will supplement the existing code analysis tools in the Autograder. There are many approaches to completing this task that will be considered. A more software development approach can consist of levering exiting systems to create a production-ready system, whereas a more research approach can consist of creating a novel approach complete with a paper and experiments.
See Also:
Code Analysis GUI
- Topics:
Frontend - Skills: software development, frontend, data munging, js, css, go
- Difficulty: Easy
- Size: Medium or Large (175 or 350 hours)
- Mentors: Eriq Augustine, Fabrice Kurmann, Lise Getoor
The Autograder has existing functionality to analyze the code in a student’s submission for malicious content. Relevant to this project is that the Autograder can run a pairwise similarity analysis against all submitted code. This is how most existing software plagiarism systems detect offending code. The existing infrastructure provides detailed statistics on code similarity, but does not currently have a visual way to display this data.
The task for this project is to create a web GUI using the Autograder REST API to display the results of a code analysis. The size of this project depends on how many of the existing features are going to be supported by the web GUI.
See Also:
Web GUI
- Topics:
Frontend - Skills: software development, frontend, js, css
- Difficulty: Easy
- Size: Medium or Large (175 or 350 hours)
- Mentors: Eriq Augustine, Fabrice Kurmann, Lise Getoor
The Autograder contains dozens of API endpoints, most directly representing a piece of functionality exposed to the user. All of these features are exposed in the Autograder’s Python Interface. However, the Python interface is a purely command-line interface. And although command-line interface are objectively (read: subjectively) the best, a web GUI would be more accessible to a wider audience. The autograder already has a web GUI, but it does not cover all the features available in the Autograder.
The task for this project is to augment the Autograder’s web GUI with more features. Specifically, add support for more tools used to create and administer courses.
See Also:
