Cheating in Online Chess (Part I): Suspicions of Engine Assistance

We randomly paired players to play a two-game (changing colours) match of 10’+5” (10 minutes with an increment of 5 seconds). None of them knew the identity of the opponent; neither his exact Elo rating; nor the condition of his experimental group. Players used an online platform, using nicknames with pseudonyms provided by the researchers. We recorded the games and conducted private semi-structured interviews with each player right after the match ended. We asked them about their impressions of the two-game match, how they assessed their play and their opponent’s play, and we specifically asked them about their impression on the possibility that the opponent could have used a chess engine. Besides, we asked players using chess engines (conditions B and C) about their strategies and impressions on how the use of the engine affected their way of playing and the whole interaction during the game. Group B and C players were also subject to questioning on whether they had any type of suspicions regarding their opponent’s play and the possibility of them also having access to external help.

After the entire round of interviews, we conducted a colloquium with all the participants of the specific session in which we disclosed the experimental conditions of each player. In the colloquium we oriented the conversation towards the broader topic of cheating in chess, both over the board and online.

A qualitative content analysis of the interviews and colloquiums provided findings in two major topics: (1) emergence of suspicions of cheating during the games by every participant; (2) strategies and impressions on engine assistance by those participants in conditions B and C.

This first post deals with the preliminary findings about (1) the emergence of suspicions of cheating during the games.

First, from the subjective impressions of players, we could not say with certainty when cheating was taking place or not. Nonetheless, we could predict (or at least say that it was more likely to emerge) when suspicions of cheating would emerge in the participants, regardless of their personalities. Suspicions of cheating emerged when performative expectations about chess playing were broken. Performative expectations refer to what is considered as normal play for a human player with certain characteristics (i.e., specific Elo rating). Performative expectations were projected depending on different factors: (i) stratification; (ii) interaction; (iii) experimental conditions.

(i) Stratification: this term referred first and foremost to chess hierarchical status expressed in the Elo rating. A specific Elo rating projects an expected playing strength. When the playing strength in the actual performance of the game does not match what is expected for the Elo rating, suspicions are more likely to occur. During the experiment, hierarchical status did not influence much the participants’ suspicions because they were unaware of the exact Elo rating of the opponent. The possible range (2100-2500) was so broad that it could not project a defined expectancy to be tested against the actual performance of the adversary. Nonetheless, in the colloquium, a mismatch between Elo rating and actual performance appeared frequently in the narratives of suspicions in cheating cases. Also, during the colloquium, participants talked about other qualifying stratification elements (age, gender, and nationality) that altered the performative expectancies bound to the hierarchical status of Elo rating, thus affecting the emergence of suspicions of cheating.

(ii) Interaction: during the actual chess games, participants expected to find normally occurring chess events in relation to questions such as the human-like logic of moves; consistency of playing strength and style; and time management. Such normal appearance of play was disrupted when awkward or incompressible moves appeared; when inconsistency in the opponent’s play appeared; or when time management of moves was erratic and/or variable. When these non-normal chess events occurred, the suspicion of engine assistance emerged more often.

Other interaction elements, present in over the board games, such as awkward emotional responses (e.g., too nervous, or too calm) were not available in online game. Thus, they did not appear in the interview narratives of the participants, even though they were raised in the colloquium discussion on cheating afterwards.

Even though participants could not make a post-hoc analysis of the games, elements such as a high percentage of precision in chess moves (related to Elo rating) and correlation between the opponent patterns and engine patterns were also raised in the colloquium as clear indexes of suspicion of cheating by engine assistance.

(iii) Experimental conditions: The experiment was presented as a typical psychological experiment that studied common topics (decision making in chess), carried out by university researchers, one of them known to the participants. Performative expectations about what it entailed to participate in scientific experiments implied that all participants took the test under the same conditions, that they were not deceived or harmed. That is why those who were subjects in condition A (human) could not conceive the suspicion of cheating in his adversary; engine assistance would break the expectation that all participants were under the same conditions and that they (humans in condition A) would not be deceived. However, precisely the condition of those who used the engine in conditions B and C projected the expectation that, since all experimental subjects were under the same conditions, everyone could be using chess engines. That explains why subjects using engine in conditions B and C were more suspicious of their human counterparts in condition A than the other way around. In fact, only one subject in condition A suspected about the use of engine of his opponent.

Concluding this first post of preliminary findings from the study, we found that the mere suspicion of cheating (engine assistance) by the opponent was enough to alter the player’s capacity to engage the game, negatively affecting his performance. Perhaps the most negative impact on the current sensation of extended cheating in online chess (Zaksaitė, 2020, p. 68), qualified by many participants in the experiment as paranoia, is precisely this: many players are underperforming on their chess play due to the suspicions of cheating.

References

Zaksaitė, S. (2020). Cheating in chess: a call for an integrated disciplinary regulation. Kriminologijos studijos, 8, 57-83.

Author biographies