Garden Recall
As part of the Current Trends in Computer Science course at Grand Canyon University, I developed a Unity-based mobile memory app. The course aims to provide students with a comprehensive understanding of current trends and practices in computer science through practical experiences. It incorporates Scrum practices to guide a term project assisted by AI, which serves as a basis for analyzing the effectiveness of AI in software development with the goal of producing a publishable paper.
I developed Garden Recall, a visual memory game that challenges the user to recall object locations. The user is briefly shown a set of objects before they are hidden. To progress, they must tap where the objects were previously located. As the game advances, the number of objects to remember increases, gradually raising the difficulty.
Game Instructions:
- Memorize the Positions: A set of outlined objects will appear on the screen. You have limited time before the objects disappear. Focus and memorize their locations before the timer hits zero!
- Find the Hidden Objects: Once the objects vanish, click on the spots where you remember them being. Be precise!
- Lives: You have 3 lives per round. If you miss an object’s location, you lose a life. Don’t worry, you can try again!
- Progression: Each round adds an extra object to challenge your memory. Keep going to see how many rounds you can complete!
Development Process:
The class followed an agile methodology, working in two-week sprints to implement core game mechanics while working with AI assistants. Each sprint ended with a stand-up meeting and a retrospective to assess progress and plan improvements.
We didn’t create a full game design document for this project. Instead, the game’s development started with developing and refining a concept for the game to meet the project requirements using a user persona and then developing a UI Flow diagram. Game development concluded with a playtest, and combining individual projects into a single game app.
PROJECT TIMELINE:
Alternative Link to View the Project Timeline
DATA COLLECTION METHODS:
Throughout the project, students had the option to use AI assistance, but were required to do so at least four times per sprint. To evaluate overall and indiviual AI performance, the same query was asked to both AI assistants; while both AIs received the same initial question, follow-up interactions varied, leading to differing response lengths.
To ensure consistency in evaluating AI performance, the class collaboratively developed a Likert scale assessing AI response quality, ranging from 1 (the response was completely unusable) to 5 (exceeded expectations). In addition to rating responses, students documented the date, initial prompt, and notable response characteristics. This data was later analyzed to compare AI effectiveness in software development tasks.
DATA ANALYSIS PROCEDURES:
After the project concluded, AI interactions were analyzed based on multiple qualitative factors, which were then sorted and analyzed quantitatively. These factors included code quality, explanation clarity, complexity, time saved, and usage type. AI usage trends were also examined over time, with data collected over eight sprints (spanning three months) divided into two halves: the first half (Sprint 1-4) and the later half (Sprint 5-8) of project development.
RESULTS AND DISCUSSION:
Alternative Link to View the Results PDF
After completing individual analyses, our findings were combined to evaluate the overall effectiveness of AI in software development. Key insights from the group analysis included:
- Non-IDE AI assistants performed better than IDE-integrated tools across most tasks.
- Paid and free AI models performed similarly, with minimal differences in effectiveness.
- AI was most useful for debugging and code completion but struggled with more complex, creative tasks.
- This research highlighted both the strengths and limitations of AI in real-world software development, reinforcing the need for human oversight and problem-solving.
Gameplay Demonstration