What’s that noise? Theory on comics and the production of sound

Sound effects -or visual “representations of sound” (Hague, 2014, p. 64)- found in comics is practically unique for the format (Catricalà & Guidi, 2015; Warner, 2008), even though other media have adapted the convention as well (see, e.g., Hague, 2014). Catherine Khor-doc (2001) analyses an example panel from Asterix, and the use of the sound effect “Tch-ac!” written in bold capital letters, to show how a company of Roman legionaries work “in perfect harmony” (p. 169). Khordoc explains that without background words such as these, comics would appear quiet and unrealistic. In comparison, David Kunzle (2001) -in the same essay collection- analyses and discusses “silent” comics, which use neither speech bubbles nor sound effects, effectively demonstrating Khordoc’s idea that comic book images without sound effects indeed appear to be muted. The most comprehensive list of sound effects to be found, the Comic Book Sound Effect Database (Ijima-Washburn), includes US Silver Age Marvel comics 1936-1963 (so a rather limited supply thus far). The database currently lists 2217 sound effects from a collection of 233 comics, from punctuation such as “!” to letter strings such as “ZZZZ ZZZZ”. Thus, sound effects are plentiful in comics, and they exist in many different formats - in fact, sound effects are commonly expressed without words, merely through the use of pictures, pictograms or icons to represent sound (see, e.g., McCloud, 1993), for example the depiction of swearing through the use of different symbols. In comics “very few of these sound pictures form words that are in the dictionary and are consequently associated with specific meanings. They constitute a language on the other side of semantics” (Warner, 2008, p. 112). There are various debated terms in use to describe sound effects. For example, sound effects and onomatopoeia -words that mimic or otherwise represent sounds and noises, such as meow, swish, crunch, etc. (Catricalà & Guidi, 2015)- are not synonymous. The stylistic use of onomatopoeia and the terms sound text and graphic text are not synonymous either

- indeed it could be argued that all text produced within comics is graphic text in some sense. However, the purpose of this text is not to discuss the semantics of these terms, but rather to investigate the social literacy practices in the classroom, of which these visual representations of sound are a part. For simplicity, I will utilize the term sound effect here, as it seems to be most commonly used, and covers most types of sound effects. The discussions here relate primarily to literary research on sound effects in Western comics and largely omits research around specific types of comics, such as manga (see, for example, Petersen, 2009), this is because the materials and the sound effects discussed in the current paper are specifically from Western comics.

The problem with discussing sound effects is of course that as ink on the page, these visual representations do not, in fact, make any sound at all. As Scott McCloud illustrates in Understanding comics, characters are often (mis)represented as “saying” something, even though in reality “no one said a word” (1993, p. 25), and the reader is assumed to imagine this sound for themselves. Similarly, Joost Pollmann (2001) argues that “comics can be quite noisy” (p. 9), referring to the many visual representations of sounds, speech and music that exist in them - even though they do not produce any sound. As Ian Hague explains, ”the reader is tasked […] with the conversion of sounds from visual representations to audible manifestations” (2014, p. 80).

Figure 1 Sound effects without paired imagery. McCloud (1993, p. 87, cropped for space) © Scott McCloud. 

While discussing the aspect of closure -the reader constructing a narrative from the story elements both shown and not shown- McCloud (1993) demonstrates how speech bubbles and sound effects can be the primary elements in comics narratives (see Figure 1). His point is that the reader makes up what is missing in the pictures, by way of reading the visualisations of sound and dialogue text in the panels and constructing a story without the rest of the panel. Although McCloud’s narrative above is highly unusual in its format, it does demonstrate that the sound effects can have a definite impact on the way the reader constructs the story, just as other images and text. What McCloud does not do is show how the reader constructs these effects in the process of putting them together with text and images (if these are present).

Subir Dey and Prasad Bokil (2016) ask for a measurement of readers’ cognitive processes in order to fully understand the literary effects of sound effects. In general, comics could be considered materials for single-person reading (see, e.g., Sutliff Sanders, 2013), and so the sounds constructed would be difficult to study. However, in a classroom setting the material is often used collaboratively as read alouds or to produce stories, and so the process of making meaning out of the visuals is displayed publicly. Thus, the current study addresses the challenge from Dey and Bokil, but, instead of measuring cognition, aims to discuss this as situated classroom practice and investigate participants’ construction of sound effects as part of comics literacy (see also Wallner, 2017b). The combination of text and image to create different representations of sound makes these aspects pertinent for studying multimodal literacy practices - how pairs of readers construct these representations in their interaction with each other and the material. Previous studies on sound effects are few and primarily focused on the text and image qualities of sound effects and how this could or should influence the interpretation of the text (e.g. Catricalà & Guidi, 2015; Warner, 2008). The current study instead takes the perspective of the readers of comics, and how classroom participants’ situated meaning making of these visual representations are constructed as social actions (see also Wallner, 2017a).

Aim

The aim of the current paper is to increase knowledge on how comics sound effects are co-constructed as multimodal literary devices in classroom interaction. The paper investigates classroom literacy events wherein visual representations of sound are an essential part, and what comics literacy practices are done when participants co-construct these as sound effects.

Analysing comics literacy in social interaction

Wallner (2017b) demonstrates how a class of 3^rd graders and their teachers make meaning out of speech and thought bubbles that the students create, combining different interactional modalities, such as sound, gestures, image and text. The study concludes that these devices are a relevant topic of teaching, and the teacher makes meaning out of the visual stylistics of the bubbles through the use of her body movements, gesture, text and talk. Already in preschool, the visual literacy of bubbles is important. Angela Yannicopoulou (2004) studies 178 pre-schoolers’ knowledge of visual literacy prior to developing textually literacy, and the calligraphic and visual elements of these mirror those used in much of sound effects. Similar to the use of bubbles, the font size, type and colour of sound effects discussed in the current paper are visual indicators to the reader of how to read the textual message (Warner, 2008).

As part of New Literacy Studies (Gee, 1992; Street et al., 2011), and new literacies (Jewitt, 2011; Ripley, 2012), the current paper combines the sociocultural study of literacy and multimodality (Kress, 2010), building on the theory, methodology, and argumentation utilized by, e.g., Wallner (2017a). Wallner’s dissertation argues for the recognition of a comics literacy, as sequential art has its own unique literary style, as well as cultural and social history, making the reading and production of comics significantly different from other literacies. This is in no way intended to diminish aspects of visual or textual literacy also present in comics meaning making. Sound effects combine the graphic, pictographic, and textual qualities of most comics and, as such, are good representations of this literacy. The current study utilizes discursive psychology (Edwards & Potter, 1992; Potter, 1996; Wiggins, 2017) in order to explore situated video-filmed literacy events, as they unfold sequentially. Discursive psychology argues that participants utilize aspects of cognition and psychology to perform social action in everyday talk (Potter, 1996; Wiggins, 2017; Wiggins & Potter, 2007). In other words, the things we say in conversation do something, and the way we say them, and the way others respond to them, indicates what these actions are. During analysis of data the researcher reviews the total material -watching videos with the help of annotated script transcriptions- looking for interesting aspects of interaction (Wiggins, 2017), in this case in regards to the comic book material. For this study the use of sound effects employed in reading was deemed particularly interesting to focus on. A collection of six sequences was built, two longer (1-3 minutes), and four shorter (less than 30 seconds). These were reviewed, transcribed in multimodal detail (including gestures, prosody and use of materials - for a transcription guide, see Appendix I), and analysed. The research question guiding the analysis at this point was: what social actions are the participants doing with their narrative construction of sound effects? After analysis of these sequences, the complete material was rewatched for further examples. In the whole of the material, few reading events occur, making these sequences rare. Although there are other sequences of reading visual representations of sound from these depicted events, the excerpts included here are the primary examples of the explored actions. The analysis is not meant to produce generalizable or exhaustive data, but to provide prominent examples in order to discuss and further investigate the phenomenon.

Method, data, and participants

The data in this paper are drawn from video observations (see, e.g., Wiggins, 2017), totalling 36 hours filmed in one primary school and one secondary school, including 77 students and 6 teachers. Four cameras and audio recorders were placed in the classrooms and students and teachers worked in pairs, groups or whole class. The study focuses on what Potter and Wiggins (2007) call naturalistic material, meaning interaction taking place in a naturalistic context (i.e. in the classroom), rather than in a laboratory. Neither the setting nor the action is designed by the researcher, and the interaction would occur regardless of the presence of the researcher (see also Potter, 2010). The researcher then presents this in detailed interactional transcripts, enabling the reader to make their own interpretation of the analysed interaction (Potter, 2010). Frame grabs from the video data are included to display gestures, gaze and materials, edited for confidentiality.

In the data presented in the current paper, the participants were part of a publisher’s project trying out two comic book albums and a teacher’s guide (Egmont Publishing, 2015). Over the course of three weeks and ten lessons, the teachers freely chose what activities and comics to use in the classroom, in what subject and for what purpose (see also Wallner, 2017a). This liberty is evinced in the current paper by the use of materials provided by the publisher as well as other comics not part of the publisher’s study. During the time of the study, there was minimal contact between the publisher and the researcher, and the researcher is under no obligation to produce results beneficial to the publisher. The researcher asked permission to film the classroom activities, and teachers and students generously volunteered to be filmed, with written consent given from students’ legal guardians.

Analysis

The following excerpts display different aspects of co-construction of comic sound effects. These excerpts are all taken from the same class in Grade 3 during one lesson and an after-school recreation centre. The analytical focus is placed on the use of the comic book graphic text to see what is narratively or interactionally done by participants.

Establishing sound effects as narrative devices through verbal and embodied actions

In the first excerpt, the topic of the lesson is speech and thought bubbles (see also Wallner, 2017b). The student Andreas has been instructed to find and copy a bubble from a comic book to a sheet of paper. Wilma is one of the teachers in the class working with Andreas. They are using a Donald Duck (DD) comic book (Carlsson, 2013), containing both old and newly produced stories.

Excerpt 1: Embodied action by teacher (Wilma & Andreas)

1 A ah ha ha ha ha [ha .hh]

2 W <[ye:a ]> so I think outside- also outside the bubbles there are of

course a lot #glug glug glug glug# you know when you ( ( Figure 2 drin[k ]

3 A [ ( ( nods ) ) ]

4

Figure 2 Wilma drinking 

Figure 3 Donald drinking © Disney, used with permission. 

5 W and there ( ( Figure 5 ) ) kind of #spsch# [#splash# ]

[ ( ( Figure 4 ) ) ]

Figure 4 Wilma’s splat 

Figure 5 Donald’s splat © Disney, used with permission. 

Wilma here pinpoints the “outside” of the speech and thought bubbles, making special note of two visualizations produced in the comic that Andreas and she are reading: “glug” and “splat”. These two effects are illustrated differently in the comic. They are onomatopoetic (Catricalà & Guidi, 2015), and the graphic text represents this in different ways: “glug” (Figure 3) is written in black soft font, while “splat” (Figure 5) is written in red with black, sharp edges. Both are written in capital letters followed by exclamation points, indicating that they should be read loudly and with emphasis (see, e.g., Wallner, 2017b; Eisner, 2008). The sounds that Wilma makes also represent this; her pronunciation of “glug” is open, glottal, and rounder, while “splat” is palatal, flatter and harder. Wilma’s reading of the words is also accompanied by embodied movements, which mimic the illustrations in the comic - she cups her hand around an invisible flask (Figure 2) and lays her hand flat in the air (Figure 4). Thus, Wilma both physically and audibly acts out the sound effects as a public display for the benefit of the student she is working with. This serves to illustrate to Andreas her interpretation and reenactment of the visual representation in the comic book. As McCloud puts it “comics is a monosensory medium. It relies on only one of the senses to convey a world of experience” (p. 89, emphasis in original). Here, the comic readers engage the material with their eyes, and translate this experience into other modalities (Stolpe, Frejd, & Wallner, 2016) - making the experience not only visual but audible and sensational as well. Wallner (2017b) shows how teachers and students negotiate multimodal readings of speech and thought bubbles. Here, Wilma demonstrates how her multimodal reading of the graphic text -with its font, font size, punctuation and its context within the comic- becomes constructed in the context of the lesson as real-life movement and sound. The topic of the ongoing lesson here is speech and thought bubbles -another multimodal aspect of comics narratives- and Wilma focuses on the sound effect as an aspect of comics similar to bubbles “also outside the bubbles there are of course a lot” (line 2). This recognizes that visual representations of sound in comics occupy a similar narrative space, or serve a similar narrative function, regardless if they are within bubbles or not. Indeed, as Figure 8 (below) demonstrates, sound effects can often be produced in bubbles if they pertain to a particular character.

Another aspect to note here is that the sound text represented here gets reformulated by Wilma, in that she in line 2 reads the text as “glug glug glug glug”, where there is only two “glug” (Figure 3) in the actual comic, and further in line 5 she reads the text as “splasch” or “spsch”, rather than the textual “splat” (Figure 5). This demonstrates that even the text gets reinterpreted here, even though these are fairly easy words to read. Wilma’s readings could be argued to produce different types of effects (of varying degree) from what the text actually says - but she gets no feedback on this from Andreas. This is an aspect that I will return to below.

Wilma and Andreas continue their work of flipping through the comic book looking for speech and thought bubbles. In Excerpt 2, they come across an image of the villainous Magica de Spell kicking the doors open to Scrooge McDuck’s vault. Numbers in brackets (X.X) indicate a hearable paus of X.X seconds, see also Appendix 1.

Excerpt 2. Embodied construction of image by student (Wilma & Andreas)

1 W and what is she doing there

( ( Figure 6 ) ) you think? ( 0.4 )

2 W that means of course that she-

( 0.6 ) ki:cks open the door there

(1.1)

3 A yea (0.8)) it says #KICK# ( 0.5 )

[ PAH.: ]

[ ( (Figure 7 ) ) ]

Figure 6 Magica kicking © Disney, used with permission. 

Figure 7 Andreas kicking 

In this excerpt, Wilma asks specifically for Andreas to explain the action performed by Magica De Spell in Figure 6. Andreas nods, and Wilma answers her own query by pointing to the panel and explaining what “it means” (line 3, whether “it” refers to the illustrations or the written word “kick” is uncertain). Andreas then pinpoints and reads the graphic text (“it says KICK!”) and illustrates the comic action for his teacher through an embodied action and a further verbal re-enactment (“PAH.:”, line 3), not only explaining -“yea it says” (line 4)- but also translating this, loudly emphasizing and acting out the sound and motion of the action displayed. In the comic, this action is illustrated both through white movement lines showing the motion (McCloud, 1993) of the doors and a yellow flash by Magica’s foot. Andreas’ response employs his whole body, although because of the table it is impossible to discern any leg movement. He also produces a sound effect two times: the first time, in line with his explanation, he says “KICK” loudly, and then, coordinated with an embodied reenactment, he also exclaims a louder “PAH”, making a further verbal emphasis on the force of the action. In relation to Excerpt 1, this embodied illustration from Andreas indicates a pedagogical resonating of Wilma’s illustrations in Excerpt 1, where she indicated that acting out the motion and sound of the comic illustration is a way of demonstrating understanding of the narrative event. Here, Andreas’ lack of a verbal response could be interpreted as a token of not understanding (thus, Wilma herself answers her question). However, Andreas then follows Wilma’s explanation with an illustration of his own -repeating the word twice in different ways, as well as physically performing the motion depicted in the comic illustration- constructing what he considers the illustration of the character’s action, the text, and the colours and shapes of the panel to collectively represent. This way, Wilma’s rather flat description of the scene, “that means of course that she- (0.6) ki:cks open the door there” (2), becomes reimagined as a forceful and living illustration.

Sound effect without visual representation

As this paper has demonstrated so far, visual representations of sound can be read in different ways from how they are spelled out on the page (such as PAH! for instance). In the following, the two participants are continuing their search for bubbles, but Wilma is now looking for something specific that she has not found yet. Throughout the transcript, Wilma and Andreas are flipping the pages of the comic book.

Excerpt 3. Sound effect without visual representation (Wilma & Andreas)

W I just wanted to see if we can find someone who’s sleeping

(0.2)

A H [ E:RE ] ( ( holds on a page ) )

3 W [ if ] you can see the difference of someone who is awake and when you’re

sleeping and dreaming

(2.6)

4 W =°(but there is n[ o one who ]’s sleeping)°

5 A [ hahahahah ] ( ( points in the comic ) )

(5.5)

W ^No:one who’s sleeping anywhere^ (4.7)

7 W °nope°

(1.4)

8 W .hno

(4.9)

9 A when they’re sleeping there are z:: (1.0)

10 W [ .hyea ] [ I ]’m thi [ nking when they’re

11 A [ ye:s ] it was somewher[ e ] [I can’t

12 W dreaming ]

13 A remember wha]t boo:k (0.3)

14 W yes yes it [ ’s- I know ]

15 A [ in my book ] in there there was=

16 W =mm ( 0.7 ) yea but I know that with when you snore there’s [ kind of ]

17 A [ there are ]

ze [ ds ]

18 W [ z: ] .hyea

(2.0)

19 W .hyea no I can’t find one

Initially, Wilma is searching for “someone sleeping” (line 1) for the purpose of discovering whether or not there is a difference between “someone who is awake and when you’re sleeping and dreaming” (line 3). This is possibly done in relation to a discussion that takes place a few minutes earlier between the teachers where the topic was the difference between a general thought bubble and a bubble representing a character dreaming. When Wilma cannot find a character sleeping, Andreas suggests that this kind of character is paired with a line of zeds - indeed a common convention in comic books (as depicted in Figure 8, taken from a different DD comic than the one the pair are looking in). They do not find one, but Andreas’ suggestion reveals that he is aware of this convention, and not only that this visual representation indicates the sound of snoring, but also that this is connected with a character who is sleeping.

Figure 8 Snoring. Ferrari, Pochet, and Zanotta (2012, p. 26) © Disney, used with permission. 

The two participants are in effect searching for two different things, Wilma is searching for a bubble (which is the topic of the lesson) and Andreas is instead searching for the snoring sound effect - possibly as this follows on their earlier talk of sound effects such as glug and splat. Andreas’ suggestion indicates that a visual representation of snoring is suggestive of a character sleeping - and where there is sleeping there might also be dreaming. Thus, he makes a connection between these two visual representations, the displayed character action of sleeping on the one hand, and the produced text and visual representation of the sound they produce whilst sleeping on the other. It could be argued that a sleeping character do not necessarily need to be snoring, but in comics these two are often combined, possibly to differ between characters merely lying down, or a dead character.

Furthermore, Andreas pronounces the sound “z::” as a prolonged consonant, which indicates a search for several Zs in a line, a common way to indicate this particular sound (un-like Figure 8, where there is only one Z with an exclamation mark). In this excerpt, the two participants do not in fact make “snoring sounds” the way a person would actually sound while snoring (in the way that Andreas does in Excerpt 2, for instance), but rather they produce the long, palatal, sounded zzz written in many comics. This differs somewhat from previous excerpts, and perhaps this stands to reason as here they do not in fact have any illustrations in front of them to “mimic” or reinterpret, and so it is more interactionally effective to describe the effect verbatim (sounding out the letters) than it is producing the actual effect, in order to know that you are looking for the same thing. Thus, here the relation between audible sensation and visual representation is reversed from previous excerpts; the two participants co-construct sounds verbally whilst searching the comic book in order to find their visual and textual equivalence.

These excerpts show how the student and teacher co-construct the visual representations into sound, using their bodies and their voices in order to represent the actions depicted in the text. This enables the readers to create an animated narrative, where they collaborate on the meaning of the text and the visuals, which enriches their construction of the story and provides more joy and life to their reading.

Re-interpreting visual representations of sound collaboratively

The excerpts below take place during a different lesson, and participants Jonna (teacher) and Charles (student) have chosen a DD comic to read aloud, and the story “Mästerskaps-mysteriet” (Swe: The Championship Mystery, Figure 9).

Figure 9 Half page. Ferrari, Pochet, & Zanotta, 2012a, p. 5 © Disney, used with permission. 

Jonna and Charles have their backs to the camera, making it difficult to discern movements and gaze. They have been reading a few frames when the following takes place. A hearable pause shorter than 0.2 seconds is indicated with a (.), see also Appendix 1.

Excerpt 4. Visual representations co-constructed (Jonna & Charles)

1 C I’ll read it too=

2 J =yea ( . )

3 C #s:pruo:ing# [ °#spruoing#° ] ( ( Figure 10 ) )

4 J [ #SPRO:ING# ] you see what happens here in the picture (2.0)

5 J what happens- what is it that comes flying=

6 C =he’s a fatty [ man ]

7 J [ yes ] yes but

look? ( 0.4 ) what is it that comes

flying ( 0.9 ) what happens

(0.9)

8 C he sits on it then it goes down=

9 J =i(h)t br(h)ea(h)ks

Figure 10 Sproing © Disney, used with permission. 

In this excerpt, Charles initiates the reading of a visual representation of a sound (SPROING!, Figure 10) - made as a rotund character (called Putte) sits down on the far side of a sofa, producing a see-saw effect where the smaller character (called Alfred) on the other side of the sofa flies up in the air (Alfred’s socks are visible on the upper-right side of the panel). This is the first sound text that is read aloud in this sitting, and this text has clearly drawn Charles’ attention (it is also the only text in this panel). Unlike Andreas in previous excerpts, Charles reads the sound text atonally and carefully, basically reading the word verbatim - this reading does not differ significantly from his reading of other words. In line 4, Jonna then loudly rephrases the sound that Charles produces (in overlap with his repetition of it). She uses more emphasis, and constructs sensation into the sound, and then asks him to explain what happens in the image. It could be argued that Charles, as a novice reader, focuses on reading the word (sproing) rather than the effect (SPROING!) - this could also be evinced by his re-reading of the same word twice. Emphasising textual loudness, through reading it loudly as it is done here by Jonna, creates sensation with the young reader who might not understand the word this way themselves (cf. Wallner, 2017b). Therefore, whereas Andreas has no problem constructing the sounds as part of the verbal story being told -adding emphasis and sensation to the words and visuals on the page- Jonna instead constructs this for Charles (as he reads the effects without emphasis). This pedagogical work shows the student how the graphic text and the colours and lines of the images can be constructed to add certain sounds and sensations to the narrative - altering the effect of the story and the students’ appreciation of it.

Connecting visual events and actions in panels to visual representations of sound

As Jonna and Charles are reading their comic aloud, rather than searching for certain elements as Andreas and Wilma, other material and narrative aspects become displayed. The following excerpt continues from the previous one with a few seconds in between.

Excerpt 5. The sound connected to events in the panels (Jonna & Charles)

1 J #hm ( 0.3 ) maybe I should sit on a chair?# he says ( ( Figure 11 ) ) (1.3)

2 C #yea do that#= ( ( Figure 11 ) )

3 J =does he think ( 0.3 ) and where does he come from? (0.4)

4 C #ploff:#= ( ( Figure 11 ) )

5 J =^#ploff#^ you see here.

( ( Figure 10 ) ) ( 0.6 ) who is it

hanging up there,

(0.3)

6 C (dickie)? ( 0.6 ) him=

7 J =him yea he of course flies up from the arm rest ( ( Figure 10 ) )

8 C h. h. (0.6)

9 J and then he catches him ( ( Figure 11 ) )

Figure 11 Ploff © Disney, used with permission. 

About forty-two seconds follow where Charles and Jonna continue to read the speech bubbles in Figure 12. The line “Ouch! Do you always have to drag that cello around with you, Meja?” (Figure 12, Huey’s speech bubble on the right side) leads to a conversation about the correct pronunciation of cello and what a cello is, but these lines have been omitted as they contain no work on sound. Charles then returns to the narrative:

Excerpt 6. Connecting sound to events in the panel (Jonna & Charles)

1 C #du-[ ]dump#=

( ( Figure 12 ) )

2 J [and what hap-] =#dump# (0.2)

3 J what izzit that happens there? (2.0)

4 J why does it sound ( 0.3 ) dump (0.6)

5 C he: collides,(0.4)

6 J yea: he gets this case in the head you see, (0.8)

7 C h.

8 J >you? see it<

9 C °(m)°

10 J mm

Figure 12 Dump © Disney, used with permission. 

Throughout these three excerpts, Jonna focuses on the actions in the panels (e.g. “what happens here in the picture” Ex 4, line 4) indicating that her construction of the sound effect takes into account the events of the panels as well as the written words, colours and other effects. The practice depicted in Excerpts 4-6 is similar to other reading training where an experienced reader would be able to sound out words differently with emotion and emphasis, whereas a novice reader’s primary focus would be correct pronunciation. However, Jonna’s reformulation of sounds and her orientation to other visual aspects of the panels are cues for Charles that he needs to take into account the visual clues not only of the sound itself, but also to the previous panel and other adjacent panels, so as to understand the event and read the sound the way it should be read (according to Jonna anyway). In Excerpt 6, Charles reads “dump” (line 1) and Jonna asks “why does it sound (0.3) dump” (line 4, emphasis added). Together, they then connect the visual elements of the panel -the characters, the object of the cello, and the visual representation of sound- co-constructing a causal narrative where the collision between Meja’s cello and Huey’s head makes a noise that sounds like “dump”.

Throughout excerpts 4-5, this is done through correction-work, somewhat typical for classrooms (see, e.g., Macbeth, 2004; McHoul, 1978; Mehan, 1979). In Excerpt 4, Jonna asks “what is it that comes flying” (line 5) and evaluates Charles’ answer (“he’s a fatty-man”, line 6 - referring to Putte) as insufficient or irrelevant, evinced by her response “yea yea, well, look?” (line 8, see also Mehan, 1979). Charles’ answer is indeed incorrect (it is Alfred who comes flying), but his response relates to the visual aspects, just not enough to satisfy his teacher. Similarly, in Excerpt 5 Charles reads the sound text (PLOFF!, Figure

similarly to his other readings -“ploff:” (line 4)- little emphasis or effect added. Jonna rephrases this reading as a higher-pitched “^ploff^” (line 5) but as in Excerpt 4, she seems to have a broader agenda in mind than merely enhancing the sensation of the specific visual representation of sound. Charles again fails to fulfil a request for information from his teacher -“where does he [Alfred] come from” (Ex 5, line 3)- instead replying by reading the sound text on the page. Jonna then rephrases her question, drawing Charles’ visual attention to Alfred in Figure 10 (effectively answering her own request, as she makes the connection between the socks in Figure 10 and the “ploffing down” Alfred in Figure 11). Charles then gives a joking answer “(dickie)?”² (Ex 5, line 6), and then gives an answer that the teacher accepts (Ex 5, lines 6-7). Jonna then gives a full account of the events taking place between Figure 10 and 11: “he [Alfred] of course flies up from the arm chair and then he [Putte] catches him” (Ex 5, lines 7-9) effectively demonstrating how the events of the two panels coincide with the visual representation of sound produced through the different characters’ actions.

In sum, in all three of these excerpts (4-6) Charles reads sound text and Jonna makes attempts to rephrase these readings by focusing on one or more events in the comic book connected to that particular sound text. Thus, Jonna and Charles co-construct both the audible quality of these sound effects (with different suggestions being offered of how the graphic text should be read in relation to the surrounding visuals), as well as their role in the sequential narrative. These excerpts demonstrate two things. First, how the teacher and student co-construct these effects, where the teacher offers second readings of the effects that are displayed, as well as contextualizing for the student the visual representations as part of the ongoing narrative action of the comics. Second, these excerpts show how the orientation of the student to the visual representations provide opportunity for him to partake and make meaning out of the comics reading, where these effects provide an arena of focus for him in the graphic material. This enables the student some control over the reading process and the interpretation of the material.

Concluding discussion

In addition to studies of the individual cognition of the reading of comics (see, e.g., Cohn, Jackendoff, Holcomb, & Kuperberg, 2014; Cohn, Paczynski, Jackendoff, Holcomb, & Kuperberg, 2012), the current paper investigates the process of readers’ social construction of comics narratives, focusing particularly on how readers co-construct visual representations of sound as audible sound effects.

When participants construct sound effects they produce narrative events using verbal sounds, emphasis, and embodied actions in order to translate the visual representations available to them on the page (Stolpe, et al., 2016). The teachers Wilma and Jonna do pedagogical work in this paper demonstrating how they embody and vocalize the graphic text. Thus, they give the silent visual effects audible sensation and emotion - similar to the work with speech and thought bubbles discussed by Wallner (2017b). This shows that the co-constructive work of the teachers and students help these participants to hear the sound, rather than just imagine it. Therefore, the reader is allowed to familiarize them-selves with these visual representations and read and appreciate them in different ways. This way, new readers of comics are given an opportunity to understand and interpret these literary devices, and make them part of their reading repertoire. This can then be used for all types of visual reading.

Furthermore, these representations are constructed not only as different types of sounds, having different types of sensational effect, but also as carrying narrative weight. Excerpts 4-6 show how Jonna responds to Charles’ readings by reorienting his focus to the other visual elements of the panels on the page - other characters, actions and events. This demonstrates to the student how these visual representations are connected as a sequential part of the narrative, and how these different visual elements come together in order to produce the desired effect (cf Cohn et al., 2014; Cohn et al., 2012). Thus, sound effects are also constructed as having meaningful impact on the way that the comic narrative as a whole is read - further enhancing the reading experience.

In sum, when participants co-construct visual representations they become active partakers in constructing the story, and the visual representations are translated by the readers into audible sound effects. This visual reading could allow for students who have not yet mastered textual literacy to be active in the process of meaning making, as they use symbols, font and colour in the storytelling process. Furthermore, learning to read these multimodal aspects is a useful skill to further students’ abilities in reading other multimodal texts as well. The present paper adds to previous literary studies of sound effects through a multimodal interactional study of participants’ in situ meaning making with a multimodal literary format, combining text and image into sound - making silent comics come alive with noise.