Introduction

Don’t judge a book by its cover. Instead, according to the implications of this popular adage, it would be better to judge it by its content. However, these are not the only ways in which books are being judged. Many books are being judged based on other criteria even before being read, when potential readers rely on book reviews or ratings to decide whether or not to read a specific book. How a book has been judged already may influence how it is judged by others; literary criticism is layered and interwoven. This literary criticism may originate from different sources, from either professional or layperson critics. Although Pierre Bourdieu (1993) has argued that the consecration by authorised gatekeepers is decisive for the symbolic capital of a literary text , such as literary prizes, layperson critics act as new literary gatekeepers and cultural transmitters regarding the evaluative talk of literature. As such they rely on the proliferation of social media and peer-to-peer-recommendation platforms responsible for the digitisation of the public sphere to take part in the literary criticism (De Greve and Martens 2021a, 2021b, in press, 2022; Kostial 2021). Beth Driscoll calls literary prizes one of two increasingly influential phenomena “[a]mong the rapid changes that characterise publishing in the twenty-first century” (Driscoll 2013, 103). The other one is social media and she argues that “both [...] draw together participants from multiple areas of literary culture” (Driscoll 2013, 103). Indeed, in the past decades, the academic interest in particular and in literary prizes in general as consecrators of literature has increased (Todd 1996; Heymans 2001; Ulmer 2006; Meyers 2007; English 2009; Emmerich 2012; Ducas 2013; Braun 2014; Irsigler and Lembke 2014; Chenaux and Beck 2015; Sapiro 2016; Childress, Rawlings, and Moeran 2017; Kennedy-Karpat and Sandberg 2017; Auguscik 2017; Dekker and Jong 2018) and so has the interest – both positive and negative – in layperson literary criticism and social media or social platforms (Steiner 2008; Driscoll 2013, 2014; Allington 2016; Kellermann, Mehling, and Rehfeldt 2016; Kousha and Thelwall 2016; Kellermann and Mehling 2017; Kousha, Thelwall, and Abdoli 2017; Thelwall and Kavyan 2017; Álvarez-López et al. 2018; Schneider 2018; Thomalla 2018; Jaakkola 2019; Wang, Liu, and Han 2019; Weber and Driscoll 2019; Pressman 2020; Walsh and Antoniak 2021), e.g. book blogs, Goodreads, Twitter, Instagram and Amazon.

One such literary prize which receives a considerable amount of academic interest (Moser 2004; Leinen 2010; Rebien 2012; Bogaert 2017; Röhricht 2016; Rahmann 2017) and social media attention is the German Ingeborg-Bachmann-Preis. This prominent prize is awarded annually during the Tage der deutschsprachigen Literatur or TDDL (translation: “Days of German-language Literature”), a multi-day literary festival and competition that takes place in Klagenfurt, Austria. The professional jury nominates 14 authors to write a short narrative text for the competition. During the event all nominated contenders read their unpublished text in front of a live audience and the jury. Afterwards, the text is discussed and criticised by the professional jury in the presence of the author and the live audience, but increasingly so by an online audience as well, under the #tddl-hashtag. The participation of the audience is stimulated by the organisers, on the one hand because the high monetary and symbolic value of the audience award “raises its prestige and the stakes, as the audience’s participation and decision carry more weight” (De Greve and Martens 2021b, 99) and, on the other hand “indem sie die Verwendung des offiziellen Hashtags, #tddl, bei der Diskussion über den Preis in sozialen Medien zunehmend fördern” (De Greve and Martens in press, 2022, translation: “By increasingly promoting the use of the official hashtag, #tddl, when discussing the award on social media”). In 2021, they even expanded the online participation and made “eine Auswahl an Postings [...] Teil der Sendung” (translation: “a selection of posts [...] part of the programme”) and included “eine ‘Frage des Tages’ [...], die über Social-Media-Kanäle öffentlich debattiert werden kann und die in der Mittagspause, moderiert von Cecile Schortmann aufgegriffen und besprochen wird” (translation: “a ‘question of the day’ [...] which can be publicly debated via social media channels and which is taken up and discussed during the lunch break, moderated by Cecile Schortmann”).¹

In “#Bookstagram and Beyond” (De Greve and Martens 2021a), we discussed the position and distinctive features of the Ingeborg-Bachmann-Preis within the field of literary prizes, as well as their influence on the online presence of the prize. Our corpus for this article consisted of all Tweets and Instagram posts from 2007 to 2017 that contained a TDDL-related query or hashtag and the Dutch, German and English reviews of all winning texts (as well as resulting novels) on the peer-to-peer recommendation platform Goodreads from this time period. We also argued that these three social media platforms, due to their distinct limitations and expectations concerning the length, type and subject which shape the content of the contributions, have a distinct way of communicating. We therefore examined the corpora by means of a digital corpus analysis and an examination of word frequencies using both Voyant Tools and AntConc.² In this article, however, we will employ a different method in order to gain deeper insight in the content of the literary discourse surrounding the TDDL by performing a fine-grained aspect-based sentiment analysis (ABSA) on a smaller manually annotated corpus consisting of TDDL-related Tweets, Instagram posts and Goodreads reviews from 2019, along with descriptions of the jury discussions from the same year. In future steps in our research, the annotated corpora presented here will be used as training data to set up a semi-supervised learning system that will be used to perform an automatic aspect-based sentiment analysis on the corpora containing all data from 2007-2017.³ We previously employed this annotation method in “Wertung von Literatur 2.0” (De Greve and Martens in press, 2022), where we exclusively analysed the content of the 2019 TDDL-Twitter-discourse. The results of this analysis will here be supplemented with the previously mentioned three additional corpora, which will enable us to compare the evaluative literary criteria of both professional and layperson critics as well as the differences within layperson criticism across social media platforms. Consequently, we will detect which sentiment is expressed about a certain “aspect” or topic (e.g. nominated author, book, jury, audience etc.) and by whom. In this article, we will thus expand the scope of our research by performing a fine-grained aspect-based sentiment analysis in order to compare the previously annotated 2019 Twitter corpus with three additional newly annotated corpora, consisting of Instagram posts, Goodreads reviews and the jury discussion. We argue that there are noticeable differences between the Twitter, Instagram, Goodreads and jury discourse surrounding the Ingeborg-Bachmann-Preis, as well as regarding the evaluative criteria of professional and layperson critics’ literary criticism.

Composition of Corpora

Because of our aim to study the evaluative literary criteria used by both professional and layperson critics and to engage with the differences in evaluation practices across platforms and media in the context of the Ingeborg-Bachmann-Preis, these corpora consist, on the one hand, of the official description of the jury discussions, and, on the other hand, of the lay discourse surrounding the prize on Twitter, Instagram and Goodreads.⁴ We thus examine four distinct corpora hailing from four different sources and/or platforms. For this case study, in order to narrow down the corpora, we decided to focus on the data and posts from 2019. In the future, this will be expanded to data from 2007 to 2017 as well. As mentioned previously, the jury discussions on the nominated texts are broadcast live. However, an official description and summary of each jury discussion per nominated text is also published on the Prize’s website. As only textual data is annotated for our research project, these descriptions serve as a direct representation of the jury discussions. The Twitter corpus consists of all Tweets created during the TDDL (26th-30th June) in 2019 that contained either the query “tddl” or the official #tddl-hashtag, resulting in a total of 4352 Tweets.⁵ Similar criteria were used to select the Instagram posts, namely all posts created during the literary festival in 2019 that include the #tddl-hashtag, comprising 191 posts.⁶ The reason for only taking those Tweets or posts into account that were posted in this period is twofold.⁷ Firstly, in both cases, the majority of Tweets and posts is written during the TDDL and these therefore contain the so-called “Sofortkritik/-kommentierung” or “Stehgreifkritik” (translation: “immediate criticism” or “criticism on the spot”) that resembles the set-up of the prize itself, in which the jury immediately and (relatively) spontaneously discuss and criticise the texts that have just been read.⁸ Secondly, posts and Tweets posted before or after the event tend to contain more irrelevant information or focus less on the TDDL themselves. Lastly, we extracted all German Goodreads reviews of the novels based on the texts that were nominated for the Bachmann-Preis that year.⁹ This constitutes to the reviews for Leander Fischer’s Die Forelle (Text: “Nymphenverzeichnis Muster Nummer eins Goldkopf”), Tom Kummer’s Von schlechten Eltern, Lukas Meschik’s Vaterbuch (Text: “Mein Vater ist ein Baum”) and Martin Beyer’s Und ich war da. It is necessary to bear in mind that because the competing texts are (at the moment of the TDDL) unpublished, short texts that are not always transformed into a published novel, not all of them may have a Goodreads book page, as is the case here (De Greve and Martens 2021b, 107–8).

Annotation of the Corpora and Results

The annotation system was designed specifically to be applicable not only to Tweets about the Bachmann-Preis, but – providing some adjustments – to both other prizes and social media platforms as well (De Greve and Martens in press, 2022), which is exactly what was executed here by including Instagram posts, Goodreads reviews and the jury discussions. We distinguish eight main aspect categories, namely “Text”, “Reference”, “Reading”, “Onsite Audience”, “Meta”, ‘Jury’’, “Irrelevant”, “Contender”, and 40 subcategories (see Figure 1). The subcategories are relatively self-explanatory but will be briefly touched upon. The “Text”-category refers to the nominated texts and has ten subcategories that encompass different elements of the texts, such as its title, quotes, the point of view or narration, motifs or themes, the language use or style, the general content or plot, the text in general, the form or structure, the flow, rhythm and punctuation and its characters. The “Reference”-category encompasses the references or comparisons to other authors or literary works, musicians or music, film or television etc. and is limited to two subcategories, one for the comparisons made by the layperson critics themselves (which implies an evaluation of the text) and one for those of the professional jury. When the canon or references or the jury are mentioned, this goes hand in hand with an evaluation of the jury discussion and valuation. The third category concerns the author readings and can be divided into mentions of the pronunciation, intonation and understandability, of the reading in general and of the flow, rhythm and punctuation of the reading. “Onsite Audience” represents the live audience that is present at the studio during the event. This category has three subcategories: the audience’s behaviour (e.g. coughing, taking pictures, applauding…), the audience in general and its age, appearance and clothing. Then there is the “Meta”-category as well, which refers to any kind of reference to the circumstances of the event or prize itself, for example the environment, namely the weather and location, the video portraits of the competing authors that are shown before the author readings, technology and social media (e.g. the livestream, website troubles, …), ritualised side events like the TDDL swimming competition in Klagenfurt’s topical Wörthersee,¹⁰ the opening speech, music played during the event, the montage of the broadcast and livestream, the event or prize itself, literature and literary prizes in general and all aspects related to the competition, such as discussions about the long- and shortlist, the voting, the winner and the award ceremony. It is important to keep in mind that when the competition (the “Meta – Competition”-subcategory) is evaluated, this is at the same time an indirect value judgement regarding the professional jury’s own valuation. If the social media user expresses their happiness that a certain competitor won the Bachmann-Preis, they implicitly convey their agreement with the jury’s decision. The Tweet “#tddl das freut mich sehr! Bachmannpreis für #birgitbirnbacher“ (translation: “#tddl I am very pleased! Bachmann-Preis for #birgitbirnbacher”) expresses a positive sentiment about two explicit aspects, namely (the result of) the competition and the contender.¹¹ The Twitter-user is happy that Birgit Birnbacher was voted the winner of the competition. Additionally, this Tweet also implicitly expresses their approval of the jury’s value judgement in pronouncing Birnbacher as the winner. Besides this, there is a main category regarding the jury with six subcategories for their voice or language use (the latter often concerns the use of dialect), quotes, the jury in general, the jury discussions and their evaluation of the texts, their behaviour as well as their age, appearance and clothing. The “Contender”-category is used for mentions of the nominated authors and has five subcategories, namely their voice or language use, primarily mentioned with regards to their author readings and video portraits, quotes,¹² the author in general, their gender as well as their age, appearance and clothing. And lastly, there is a category for irrelevant Tweets as well, specifically Tweets that either clearly did not discuss the TDDL or in which it was not immediately evident how they could be connected to the event.

Results from other annotation projects have often shown that annotating fine-grained aspect categories is complex and that the accuracy of the automatic category prediction tends to decrease as complexity increases. However, this does not mean a fine-grained annotation of aspect categories is out of reach. In their article “A Proposal for Book Oriented ABSA: Comparison over Domains”, Álvarez-López et al. (2018) have illustrated that ABSA, using multiple subtasks such aspect extraction, category detection, and sentiment analysis, can be used to analyse a data set consisting of Amazon book reviews and to identify multiple aspect categories related to the book and its content, such as “general”, “author”, “title”, “audience”, “quality”, “structure”, “length”, “characters”, “plot”, “genre” etc., similar categories as the ones we employ within the “Text”-category. A lot depends on the consistency of the annotation, especially for categories that are less distinct or whose target words are more varied, such as the “Text – General Content & Plot” or the “Jury – Discussion & Valuation” subcategories.¹³ The exact description of and the vocabulary used to describe the content of a text or the jury discussion often consist of a longer span containing more and diverse vocabulary. For many other categories the same target words are repeatedly used to refer to the aspect, such as the variants of “Text” or “Buch” (“book”) for the “Text – General”-category, mentions of “Thema(tisch)” (“theme/thematically”), “Motif” (“motif”) etc. regarding the “Text – Themes & Motifs”-subcategory, references to contender or jury names, etc. In the case of more “vague” or varied target words, we therefore decided to annotate the parts of these longer spans that are more likely to reoccur and be automatically detected. In this sentence from an Instagram post, for example: “Es geht um den Tod, Vergehen, schlechtes Gewissen, weil man sich vom Vater abwendete“ (translation: “It is about death, transgression, bad conscience, because they distanced themselves from the father”),¹⁴ we would solely label “Es geht um” as “Text – General Content & Plot”, which is a more frequently recurring phrase. Regarding the jury discussion, if it pertains to a description of the discussion instead of a direct reference of a word like “Diskussion”, we can annotate the mention of the name of a jury member in combination with a verb that communicates the expression of an opinion. In “Winkels meint, der Text bringe Mut auf. #tddl” (translation: “Winkels maintains that the text inspires courage. #tddl”),¹⁵ we would only label “Winkels meint” as “Jury – Discussion & Valuation”. We have already tested the performance of automatic aspect term category prediction using the manually annotated 2019 TDDL-Twitter corpus, achieving a rather high accuracy of 83% for the prediction of the main aspect categories and 73% for the prediction of the fine-grained subcategories (De Greve et al. 2021), illustrating that the application of the categories is indeed fairly reliable and effective.

We annotated the Instagram posts in the exact same manner as the Tweets, as both are generally shorter social media contributions with a length limitation, and tagged the aspects and sentiment on a Tweet or post level and included implied aspects as well (De Greve and Martens in press, 2022). Some adjustments were made, however, regarding the corpora of Goodreads reviews and the description of the jury discussions. Firstly, both tend to be longer than the Tweets or posts, especially so in the case of the discussion descriptions, and tagging on a review or article level would therefore exclude more information. Consequently, for these two corpora, each aspect was tagged, regardless of any repeated mentions. Secondly, in the description of the jury discussion, the jury members are naturally mentioned frequently. However, as we employ the description as a stand-in for the actual jury discussion, mentions of other jury members in this corpus were only tagged and annotated if they were either mentioned in a quote or if a jury member commented on, agreed or disagreed with the other members of the jury. This means that in the sentence “Klaus Kastberger vermisste eine österreichische Note des Texts” (translation: “Klaus Kastberger missed an Austrian touch in the text”) the word group “Klaus Kastberger vermisste” did not receive a “Jury – Discussion/Valuation”-label, as it is only a pointer attributing the utterance to a jury member.¹⁶ Contrary to this, the word groups such as “Hildegard Keller verstand den Einwand” (translation: “Hildegard Keller understood the objection”)¹⁷ and “Dem widersprach Wilke” (translation: “Wilke contradicted this”)¹⁸ do describe jury members (dis)agreeing with each other and thus evaluating each other’s arguments in the discussion. As a consequence, these would be tagged as “Jury – Discussion/Valuation”, with a positive and a negative sentiment expression respectively.

To facilitate the comparison of the differently sized corpora, the results will be shown as percentages, as opposed to absolute numbers, and the irrelevant Tweets are excluded from the graph (De Greve and Martens in press, 2022).¹⁹ The first four graphs (Figures 2, 3, 4 and 5) visualise how often a main aspect category is mentioned in each of the corpora and which percentage of the mentions is either positive, neutral or negative. When comparing the results, it becomes clear that the corpora have different focal points. In the Twitter discourse, attention is divided over several main categories, mainly the “Meta”-, “Text”-, “Jury”- and “Contender”-categories. In her research on the Bachmann-Preis, Xiana Bogaert compared the thematic tendencies of the jury discussions between 2010 and 2014 with hand-selected Tweets (Bogaert 2017, 7) and came to the conclusion that Twitter-users “hauptsächlich die Jurydiskussionen des Bachmannpreises zum Gegenstand ihrer Kritik herananziehen” (Bogaert 2017, 54, translation: “mainly draw on the jury discussions of the Bachmann-Preis as the object of their critique”) in order to comment and criticise the texts indirectly. Consequently, their evaluative process and criteria are influenced by those of the jury (Bogaert 2017, 56).²⁰ Although they indeed discuss the jury most frequently (28,63%), the large percentage of “Text”-mentions (25,72%) indicates that the Twitter-users also discuss and criticise the texts themselves (De Greve and Martens in press, 2022). This discourse also contains comparatively more negative sentiments than the others: 41,83% of the mentions are negative in the Twitter discourse, in comparison to 28,51% (more than 10% less) in the jury discussions, and only 8,89% on Instagram and 12,03% on Goodreads. The Instagram corpus, on the other hand, appears to focus most on the event itself – the “Meta”-category (54,13%)- and to a smaller degree on the text (16,21%), jury members (10,71%), nominated authors (9,48%) and author readings (7,35%). Besides this, the mentions on Instagram are primarily neutral (59,94%).²¹ The Goodreads reviews and jury discussions, however, are more positive (55,69% and 46,62%) and both focus predominantly on the participating texts (87,98% and 79,49%). As a result, the Goodreads reviews and the valuation of the professional jury appear to be more similar in their shared fixation on the texts themselves. In itself, this aligns with the expectations regarding the task of the jury as well as the focus of a book recommendation platform: it is the jury’s prerogative and responsibility to discuss the text – and, to a lesser extent, the author’s reading. Similarly, on a peer-to-peer book recommendation platform, users are expected to write book reviews, focusing on the literary work. Twitter- and Instagram-users, on the other hand, do not have this restriction and they are therefore free to comment on more aspects of the literary competition. For this research, we will now expound on the four most dominant main aspect categories, namely the text, which will grant insight into the various literary criteria at work in the different corpora, as well as the “Meta”-category, the jury and the contenders.

By addressing and examining the “Text”-subcategories, it is possible to examine which aspects of a literary text are being mentioned most often, which implies how relevant this aspect is to a certain group, as well as whether it is mainly criticised or praised. The diagrams (Figures 6, 7, 8 and 9) illustrate how often a certain “Text”-subcategory is brought up in the Tweets, posts, reviews and jury discussions. In contrast to the previous graphs, in the context of the “Text”-category, the similarity between the Twitter discourse and the jury discussion is slightly more pronounced when compared to the other corpora. And in fact, contrary to common prejudice against fan communities and layperson criticism, the Twitter discourse is generally more negative (52,96%) than the actual jury discussion (26,37%), containing twice as many negative mentions in terms of percentage. Both of these discourses are comparatively more negative than the others (Instagram: 15,09%; Goodreads: 13,67%). Furthermore, the aspect distributions in the Tweets and jury discussion correspond more closely to one another. In both cases, they primarily discuss the text in general (Twitter: 41,70%; jury: 37,03%), secondly the content or plot (Twitter: 20,53%; jury: 24,78%) and thirdly the language use and style (Twitter: 12,74%; jury: 12,39%). The other aspect subcategories are not mentioned as often. Nevertheless, despite the similarities, there are some differences as well. The jury pays somewhat more attention to the characters (Twitter: 3,41%; jury: 5,76%), the motives and themes (Twitter: 3,41%; jury: 5,76%), the form of the text (Twitter: 1,42%; jury: 4,90%) as well as the narration (Twitter: 2,83%; jury: 6,77%), whilst the Twitter-users focus more on the quotations (Twitter: 11,78%; jury: 1,59%). The literary criteria expressed in the Tweets do not appear to be more superficial than those of the professional jury.

We can conclude that the literary evaluation criteria of the Twitter-users and the professional jury are in fact relatively similar, apart from some minor differences, and that these lay critics are even more critical than the professional jury, thus contradicting Bogaert’s assumption that “das Potenzial eines Textes [...] nicht beim Bewerten in Betracht gezogen” is (Bogaert 2017, 48, translation: “the potential of a text (...) not taken into account when evaluating it”). The subcategory aspects in Instagram posts are generally more often mentioned in a neutral context (58,49%) and are frequently informative statements. Only the text in general (15,09% positive out of 22,64%, equalling 66,65% of this total percentage) and the text’s flow and rhythm are (predominantly) being praised, and the style and language (3,77% negative out of 7,55%, equalling 49,93% of this total percentage) are criticised more often. In comparison to the other corpora, they mention the plot and content of the text more frequently (35,85%). When examining the Goodreads reviews, the percentage of positive aspect mentions stands out: this is the most positive corpus regarding the “Text”-category (58,99%). In this corpus the text in general (26,62%) and its content (24,46%) are discussed almost equally often. Striking, however, is the fact that the characters are mentioned so frequently (14,39%), as they receive far less attention in the other three corpora (Twitter: 3,41%; Instagram: 5,66%; jury: 5,76%). The diagrams of the Instagram posts and Goodreads reviews illustrate a somewhat different hierarchy of aspect importance than illustrated by those of the Tweets and jury discussions: the text in general loses significance in comparison to its content. From these data can be concluded that many aspects of the competing texts are being discussed in all of the corpora, though each corpus somewhat has its own focus, once again (De Greve and Martens in press, 2022) disproving Wegmann’s thesis that “Auseinandersetzungen mit ästhetischen Formprinzipien, mit der Poetik von literarischen Texten, ihrer Stilistik, ihren rhetorischen Mitteln” in the Web 2.0 “[t]endenziell eher unterrepräsentiert sind” (Wegmann 2012, 287, translation: “Discussions of aesthetic principles of form, of the poetics of literary texts, their stylistics, their rhetorical devices (...) (t)end to be underrepresented”).

Moving onwards to the “Meta”-category (Fiure. 10, 11, 12 and 13), the differences between the corpora are more pronounced. Contrary to the previous results, in this context the resemblance in focus is greater between the Twitter and Instagram corpus and between the Goodreads reviews and the jury discussions, respectively. The former mainly focus on the TDDL and the Ingeborg-Bachmann-Preis itself (Twitter: 40,23%; Instagram: 36,16%), the “main event”, and discuss every other subcategory as well, though in varying degrees – and that is where their differences lie. Once again, the Twitter discourse contains the largest percentage of negative mentions (33,53%), although the majority are neutral (38,27%), and the Instagram posts are mostly neutral (61,58%). The competition (Twitter: 17,98%; Instagram: 9,04%), concerning the voting, long- and shortlist as well as the award ceremony -mostly in a neutral (Twitter: 7,86%; Instagram: 4,52%) or positive (Twitter: 6,07%; Instagram: 4,52%) context-, the opening speech (Twitter: 6,47%; Instagram: 2,82%) as well as the technology and social media (Twitter: 13,12%; Instagram: 6,21%) are mentioned almost twice as frequently in the Tweets, whereas the Instagram posts more often address the side events (Twitter: 1,16%; Instagram: 11,30%) and, especially so, the weather and location (Twitter: 6,65%; Instagram: 24,29%). The main focus on the event itself for the Twitter and Instagram corpora in addition to the greater focus on location on Instagram than on Twitter confirm the preliminary results from our preceding digital corpus analysis, as “[f]or such a location-oriented visual social media platform [=Instagram], the frequent occurrence of place names is to be expected” (De Greve and Martens 2021a, 15). The Goodreads reviews and jury discussions on the other hand appear to concentrate mostly on the event itself (Goodreads: 33,33%; jury: 28,57%), similar to the Twitter and Instagram corpus, the competition (Goodreads: 33,33%; jury: 28,57%), the technology and social media (on Goodreads, 33,33%) and on the video portraits and the weather and location (jury discussion, two times 14,29%). However, contrary to the Tweets and Instagram posts, they do not address any of the other subcategories. The Goodreads corpus solely contains neutral mentions, whereas the sentiment varies depending on the subcategory in the jury discussions, with a total of 57,14% positive, 28,57% neutral and 14,29% negative mentions. However, looking at the percentages presented here, it is important to keep in mind that the total percentage of mentions of the “Meta”-category is in fact very low for both of these corpora (cf. Figures 4 and 5) and consequently are relatively negligible, in accordance with the conclusion that there were “no explicit references to the Ingeborg-Bachmann-Preis or the TDDL” in the Goodreads reviews (De Greve and Martens 2021a, 19).

The next set of graphs (Figures 14, 15, 16 and 17) shows the percentage of positive, neutral and negative mentions of the jury in the four corpora. Both the Instagram posts and the jury discussions exclusively discuss the jury discussion and valuation (Instagram: 71,43%; jury: 96,34%) and the jury in general (Instagram: 28,57%; jury: 3,66%). Although the discussion of the professional Bachmann-Preis jury focuses mainly on the nominated texts, they occasionally also discuss topics related to themselves. This usually consists of criticising the arguments or opinions of the other jury members or agreeing with them (total of 68,29% negative and 28,05% positive mentions). In comparison, the percentage of neutral mentions (62,86%) is much higher in the Instagram corpus and they address the jury in general more frequently: 28,57% compared to 3,66%. A greater variety of jury-related aspects is being discussed in the Tweets, although their main focus is on the jury discussion and valuation (71,83%) as well. This subcategory is most mentioned in a negative context (37,56% negative out of 71,83%, equalling 52,29% of this total percentage), which indicates that the Twitter-users generally disagree with the jury’s evaluation, even though they were not as condemning of the jury’s decision regarding the competition (Figure 10). In fact, most of the mentions for the “Jury”-category are negative (50,23%). It must be taken into consideration, however, that the Twitter-users use the jury discussions as a stepping stone to take part in the online discussion of the competing texts, by evaluating them indirectly (Bogaert 2017, 54–56). Besides this, the Tweets also simply mention (10,19%) and quote (10,33%) the jury members. In addition, the Twitter discourse is also interested in some non-text evaluation-related aspects of the professional jury, such as their voice or language use (2,35%),²² their behaviour (2,88%) and their age, appearance and clothing (2,41%). This focus on secondary aspects, such as age, appearance, attire and gender also extends to the discussion surrounding the contenders (cf. Figure 18).²³ The reason for this may be twofold: on the one hand, the online audience watching the TDDL-livestream is able to see or is confronted with these secondary aspects and may simply comment on them, just like they comment on the montage of the livestream or the interior decor of the studio. On the other hand, however, the online community also uses Twitter and Instagram (e.g. regarding the gender of the authors) for social activism by means of striking symbols. Comparable in this context is, for example, the role of Hanna Engelmeier’s T-shirt at the Deutscher Buchpreis.²⁴ This attention to detail might strike one as superfluous or as an aberration. But the dress-code of the jurors signals their habitus, just as the major critics of past eras like Reich-Ranicki and Fritz J. Raddatz did with their more formal attire. While everything can be said to be political, the more outspoken nature of politically engaged discourse also coincides with online movements such as #frauenzählen (translation: “#countingwomen”).²⁵ Critics on social media are more preoccupied with the visibility of women and gender equality in the literary field, e.g. “Federer und Heitzler auf der Shortlist und Birkhan fehlt? Unverständlich. Das muss ein Fall von Quotenmännern sein. #tddl” (translation: “Federer and Heitzler on the shortlist and Birkhan missing? Incomprehensible. This must be a case of token men. #tddl”).²⁶ In comparison, the jury is not mentioned at all in the corpus of Goodreads reviews (De Greve and Martens 2021a), which focus mainly on the text itself.

The final main aspect category we discuss in this article concerns the nominated authors (Figures 18, 19, 20 and 21). Once more, the Twitter discourse remains the corpus with the greatest percentage of negative mentions by far (Twitter: 26,40%; versus Instagram: 3,23%; Goodreads: 0%; and jury: 2,78%), even though the percentage of positive (40,72%) and neutral (32,88%) mentions of this main aspect category prevail. The Twitter corpus is comparatively more varied and discusses a wider range of topics, namely the competitors’ voice and language use (1,21%), quotes (1,81%), and – as addressed in the previous paragraph – their gender (3,92%), as well as their age, appearance and clothes (4,98%). Nevertheless, the main focus for all four corpora are the contenders in general (Twitter: 88,08%; Instagram: 93,55%; Goodreads: 100%; jury: 88,89%). In the corpora of Instagram posts, Goodreads reviews and jury discussions, the mentions are mostly neutral, informative statements (Instagram: 58,06%; Goodreads: 54,55%; jury: 66,67%) or positive references (Instagram: 38,71%; Goodreads: 45,45%; jury: 30,56%), with little to no negative sentiment. In the Goodreads reviews, no other aspects related to the nominated authors are mentioned, but the Instagram posts periodically discuss the gender (cf. previous paragraph) as well (6,45%), and the jury sometimes quotes the contender (2,78%) or addresses his or her voice and language use (8,33%), which can be connected to the author readings.

Conclusion

This article has addressed the literary criticism surrounding the German Ingeborg-Bachmann-Preis in order to gain deeper insight into the content of the evaluative talk about literature by professional and layperson critics ‘in real life’ as well as on various social media platforms by performing a fine-grained aspect-based sentiment analysis on a manually annotated corpus consisting of Tweets, Instagram posts, Goodreads reviews and descriptions of the jury discussions. The goal of this article was to identify which topics regarding the TDDL (e.g. nominated texts, jury, authors, the prize itself etc.) were being discussed in the different corpora and which sentiment was expressed about them. We wanted to analyse the possible differences between the Twitter, Instagram, Goodreads and jury corpora and between the evaluative literary criteria of professional and layperson critics, as well as to expand upon and compare the new findings to the results presented in our previous articles on the Bachmann-Preis (De Greve and Martens 2021a, in press, 2022). In addition to this, we also briefly discussed the corpus composition and annotation method.

First, we compared the four corpora based on seven main aspect categories before zooming in on the subcategories of the four most frequently discussed main categories, namely the “Text”-, “Meta”-, “Jury”- and “Contender”-categories. The visualisations focusing on the main aspect categories illustrated the main overarching differences and similarities regarding discussed topics and sentiment between the discourses. We discovered that, on an overarching level, there is a greater similarity between the Twitter and Instagram discourses on the one hand and between the Goodreads reviews and the jury discussions on the other hand. Due to the lack of restrictions and expectations regarding the main focus of the discourse, the Tweets and Instagram posts are free to focus on more diverse topics, whereas both the Goodreads reviews and jury discussions focused predominantly on the nominated texts. The latter were, percentage-wise, comparatively more positive than the others. Overall, the Instagram corpus contained more neutral mentions and the Twitter-users were the most critical. The examination of the “Text”-subcategories enabled us to gain a deeper insight into the evaluative literary criteria at work in each discourse. We discovered that these were relatively similar, although the corpora each had their own focal points to which they paid more attention than the others. The differences were more substantial, however, for the other three subcategories, regarding the event itself, the jury and the competing authors, where the Tweets tended to discuss more diverse topics than the other corpora. The analysis of the manual annotation of the four corpora consequently enabled us to confirm that there are multiple distinctions between the literary discourse and the respective evaluative literary criteria on Twitter, Instagram, Goodreads and of the professional jury discourse, as well as to disprove the prejudice and hypothesis that layperson literary criticism does not concern itself with aesthetic principles, such as form, style etc. (Wegmann 2012), or only makes an assessment based on the judgement of the professional jury and is generally less critical. This study has delved into the content of the literary discourse surrounding the Ingeborg-Bachmann-Preis and has provided an in-depth qualitative and quantitative analysis of the selected and manually annotated corpora. A larger quantitative analysis will be needed to investigate whether these findings also apply to the discourse of other editions of the TDDL. However, the data presented here can be used to train a semi-supervised learning system to automatically examine and explore the TDDL-discourse over a larger time period.