Guide to Judging: Team Interviews

Overview

The Team Interview Rubric is used for all team interviews. Judges may use the Team Interview Tips and Sample Questions and Team Interview Notes to assist in team interviews. Judges will interview the teams that have been assigned to them by the Judge Advisor. Teamwork, professionalism, interview quality, and team conduct shall be considered in nominating and ranking teams for all judged awards.

Initial Team Interviews can be conducted in the team pit area. This allows Judges to observe teams at work and quickly move from team to team. Another alternative is for the Initial Team Interviews to be conducted in a hallway or some other still-public place, such as a library room or cafeteria. This can be a quieter venue for interviews, but care should be taken that the interview format remains intact and does not become a prepared presentation and also keep in mind that this setting could come across as intimidating for some teams.

Initial team interviews can be conducted without notice to teams, be scheduled by the Judge Advisor, or be conducted at a time of the team’s choosing (i.e. via a signup sheet or first-come-first-served queue). All teams at the event must have their initial interviews scheduled in the same way, and teams are not allowed to choose a particular set of judges. A best practice for a self-service model for assigning interviews would be to assign teams to one of several groups of judges based on a queuing method, with modifications made in case conflicts of interest arise between a team and a judge.

Judges need to talk to the student members of the team. Occasionally enthusiastic adults may want to answer the Judge’s questions. If this is encountered, politely remind the adult(s) that the Judges are there to interview the students. All teams at an event must have an opportunity to be interviewed at least once.

Some teams may be hard to find at an event: if they are not in their pit space, another approach may be to find them as they come off the field for their match. 

Some teams may want to share parts of their engineering notebook during their interview. This is permissible, but depending on how and when notebooks are collected, this may not be possible. Teams should be prepared to answer the judges’ questions without their notebook.

Award finalists may be cross interviewed by different Judge Teams as a part of the deliberation process. The Judge Advisor will assign additional interviews as needed during the event. Follow-up interviews for any award contenders should be conducted without notice, preferably in the competition or pit areas. This allows judges to see the team in their workspace and does not give any teams an advantage via prior notice.

Note: Some students, whether it be from individual or cultural differences, may have varying styles of interacting with judges during the interview process. Maintaining eye contact, speaking in a loud enough voice to be easily heard, and other engagement norms, may differ between students. Judges should do their best to give all teams an opportunity to share their design process during the interview and should strive to not allow factors that are beyond students’ control to bias their evaluation of the team.

Note: A team may decline to be interviewed – that team would no longer be eligible for any Judged Award with the exception of Balloted Awards if they are being offered at the event.

Note: Judges should avoid using humor or language that could be interpreted as disparaging: For example: “I can’t believe you came up with this on your own!” might have been intended as a compliment to the team, but could be misinterpreted to indicate that the judges believe the team is violating the Code of Conduct by claiming work that is not their own.

Note: Some Judge Advisors may wish to create a list of questions for judges to ask that are common for all interviews at an event. This could be particularly helpful to ensure that all aspects of the robot and competition are being addressed, or to assist inexperienced judges with the interview process. This should not be construed as a “script” – judges should be free to follow up questions based on student responses.

Step 1 – Conducting the Team Interview

• All teams should be interviewed for roughly the same amount of time – the Judge Advisor will create a schedule based on the number of teams and Judges at an event.
• Typically, a Team Interview lasts about 10-15 minutes – staying on schedule is important to ensure all teams are interviewed and there is sufficient time to conduct deliberations. Teams that may need an interpreter to communicate with judges may need more time, and should notify the Event Partner upon registration.Team interviews are based around Judges directly asking students open-ended questions about their robot and design process in order to give students an opportunity to share their design process, teamwork, and journey throughout the season. Follow-up questions are asked as needed.
• Teams can use their Robot and its associated equipment, Engineering Notebook (optional), and Programming device to show their code (optional) during the interview. It is the intent of the interview for judges to engage with students and their robot and not with audio/visual aids such as presentations or displays.
• Judges should take notes during interviews and observations to support their evaluations and assist with deliberations – The Team Interview Notes form can be used to keep track of notes for each team.
• Judges should consider taking a picture of each robot with the team number visible to help recall details about robot designs mentioned in their notes.
• If Judges are unable to locate an assigned team for an interview after several visits to the team’s pit area, they will leave a Judges’ Note to Missed Teams on the team pit table.
• If Judges are unable to locate an assigned team’s pit area, they should contact the Judge Advisor for assistance.
• Judges should remember that younger students communicate their ideas differently than older students. Judges should use age-appropriate language when asking questions and considering student responses.
• The Judging Single Page Reference may additionally be used by Judges to look up award description briefs and other useful information.

Step 2 – Complete Team Interview Rubric

After the interview, each Judge group should complete the Team Interview Rubric and optionally the Initial Award Candidate Ranking Sheet for each team. Judges should go somewhere private to discuss and fill out these forms and should take care that their discussions are not overheard by any other party.

Judges should identify student-centered teams with positive, respectful, and ethical conduct during the team interviews and team observations; conversely, they should also make note of any teams that are not demonstrating these principles – including teams that are not being directly interviewed.

IMPORTANT: The Team Interview Rubric is a tool for initial team interview evaluations through quantitative comparison. The final determination of all award candidates and winners are done through further qualitative deliberation among judges based on award descriptions and criteria. As such, a team earning a particular or overall score on a rubric is not an automatic disqualification or threshold for any judged award.

Step 3 – Identify Initial Candidate Teams Within Judge Group

Where additional Judged Awards are offered at an event (beyond the Excellence, Design, and Judges Awards), the Judge Advisor may provide the Initial Award Candidate Ranking Sheet to Judge groups assigned to interview teams. The Judge groups will use both Team Interview Rubric and the Initial Award Candidate Ranking Sheet as they interview their group of teams. This form may also be useful when initial team interviews are being done remotely (see Guide to Judging: Remote Judging) as a way to log nominations from each judging group.

On the Initial Award Candidate Ranking Sheet, Judges will write down the team numbers of the teams they are assigned to interview on the left side and fill in any additional Judged Awards being offered at the event. Awards should be listed according to precedence from left to right, with qualifying awards in the leftmost columns, followed by the non-qualifying awards. The precedence of Qualifying Awards is listed in the REC Foundation Qualifying Criteria document. The Judge groups will then use the spaces provided to indicate a candidate for each of the additional Judged Awards being offered at the event.

As Judges interview teams, they may want to use multiple stars or checks on the Initial Award Candidate Ranking Sheet to give weight to a recommendation. This is done by adding check marks to rank teams – for example, the first team interviewed received one check mark, and if the second team interviewed would be a better candidate, they would receive one check mark, and the first team would receive a second check mark, ranking them 1 & 2. This would continue until all teams are interviewed – the end result would be a ranking of teams.

Below is an example of how this sheet might be filled out by one Judge group, judging a subset of teams at a larger event. In this example the Innovate, Think, and Judges awards have been filled in below.

This is a simple way for Judges to preliminarily rank their recommendations as they go, with final rankings done after their set of interviews are completed. Additionally, Judges can also make notes on the Team Interview Notes Sheet.

Continue to the next section, Guide to Judging: Award Deliberations