How to do things without words. Multisemiotic visualization in LEGO vs. IKEA building instructions


Abstract


This study explores LEGO and IKEA building instructions within the broader landscape of multisemiotic qualitative data visualization practices. Building instructions are defined as procedural texts, in which an encoder plans ahead how a practical action is to be undertaken in the real world, and as cognitive protocols, guiding users in the performing of complex tasks by way of rescaling the latter in a sequence of smaller problems, and therefore turning representation into action. The peculiarity of LEGO and IKEA building instructions lays however in the multisemiotic mix through which they perform their referential and instructional functions, which does not comprise verbal language. By way of multisemiotic visualization strategies, LEGO and IKEA building instructions present numerical, topographical, analytical and processual meanings in synoptic, integrated fashion, so as to allow the grasping of articulated data sets on the part of the user. Incorporating Systemic Functional Grammar, classic Social Semiotics and Cognitive Discourse Analysis, this study analyses and contrasts the ideational and interpersonal processes through which LEGO and IKEA building instructions codify empirical phenomena and procedures in such a way as to get unspecialized users to obtain a complete and concrete object from a box of scattered pieces. Attention is finally given to the overarching cultural and epistemological tendency that may be detected behind the fast-growing diffusion of visualization in today’s information dissemination practices, i.e. the spatialization of temporal processes.

DOI Code: 10.1285/i22390359v40p117

Keywords: qualitative data visualization; multisemioticity; Cognitive Discourse Analysis; procedural discourse; spatialization of temporal processes

References


Arnheim R. 1969, Visual Thinking, University of California Press, Berkeley and Los Angeles.

Benking H. 2005, Using Maps and Models, Supersigns and Superstructures. http://benking.de/systems/codata/CODATA-MIST2005.htm (1.02.2018).

Bergen B., Lindsay S., Matlock T. and Narayanan S. 2007, Spatial and Linguistic Aspects of Visual Imagery in Sentence Comprehension, in “Cognitive Science” 31, pp. 733-764.

Berger S. 2017, The Art of Philosophy: Visual Thinking in Europe from the Late Renaissance to the Early Enlightenment, Princeton University Press, Princeton and Oxford.

Bernstein B. 1981, Codes, Modalities and the Process of Cultural Reproduction: A Model, in “Language and Society” 10, pp. 327-363.

Bertin J. 2011, Graphics and Graphic Information Processing, de Gruyter, Berlin.

Coopmans C. 2014, Visual Analytics as Artful Revelation, in Coopmans C., Vertesi J., Lynch M. and Woolgar S. (eds.), Representation in Scientific Practice Revisited, MIT Press, Cambridge, MA, pp. 37-59.

Diana R.A. and Reder L.M. 2004, Visual Versus Verbal Metacognition: Are They Really Different?, in Levin D.T. (ed.), Thinking and Seeing: Visual Metacognition in Adults and Children, MIT Press, Cambridge, MA, pp. 198-201.

Friendly M. 2009, Milestones in the History of Thematic Cartography, Statistical Graphics, and Data Visualization. http://www.datavis.ca/papers/gfkl.pdf (23.03.2018).

Gotti M. 2003, Specialized Discourse: Linguistic Features and Changing Conventions, Peter Lang, Bern.

Grice H.P. 1975, Logic and Conversation, in Cole P. and Morgan J.J. (eds.), Syntax and Semantics. Vol. 3: Speech Acts, Academic Press, New York, pp. 41-58.

Halliday M.A.K. 2002, On Grammar, Continuum, London.

Halliday M.A.K. 2004, An Introduction to Functional Grammar: Third Edition, Hodder Arnold, London.

Hyland K. 2005, Metadiscourse, Continuum, London.

Iedema R. 2001, Resemiotization, in “Semiotica” 137 [1-4], pp. 23-39.

Iedema R. 2003, Multimodality, Resemiotization: Extending the Analysis of Discourse as Multisemiotic Practice, in “Visual Communication” 2 [1], pp. 29-57.

Jakobson R. 1959, On Linguistic Aspects of Translation, in Brower R.A. (ed.), On Translation, Harvard University Press, Cambridge, MA, pp. 232-239.

Jary M. and Kissine M. 2014, Imperatives, Cambridge University Press Cambridge.

Kress G. and van Leeuwen T. 1996, Reading Images: The Grammar of Visual Design, Routledge, London.

LeVine P. and Scollon R. 2004, Discourse and Technology: Multimodal Discourse Analysis, Georgetown University Press, Washington.

McKay E. 1999, An Investigation of Text-Based Instructional Materials Enhanced with Graphics, in “Educational Psychology” 19 [3], pp. 323-335.

Mitchell W.J.T. 1980, The Language of Images, University of Chicago Press, Chicago and London.

Neuenshwander B. 1993, Letterwork: Creative Letterforms in Graphics Design, Phaidon, London.

O’Halloran K.L. (ed.) 2004, Multimodal Discourse Analysis: Systemic-Functional Perspectives, Continuum, London.

O’Halloran K.L. 2008, Mathematical Discourse: Language, Symbolism and Visual Images, Continuum, London.

O’Halloran K.L., E M.K.L., Podlasov A. and Tan S. 2013, Multimodal Semiotics: The Interaction of Language with Other Resources, in “Text & Talk” 33 [4-5], pp. 665-690.

O’Halloran K.L., Tan S. and E. M.K.L. 2017, Multimodal Analysis for Critical Thinking, in “Learning, Media and Technology” 42 [2], pp. 147-170.

O’Halloran K.L., Tan S. and Wignell P. 2016, Intersemiotic Translation as Resemiotization: A Multimodal Perspective, in “Signata” 7, pp. 199-229.

Pérez-Llantada C. 2020, Preface, in Gotti M., Maci S.M. and Sala M. (eds.), Scholarly Pathways: Knowledge Transfer and Knowledge Exchange in Academia, Peter Lang, Bern, pp. 5-9.

Pillegard M. and Frandsen F. 1996, Text Type, in Östman J.O. and Verschueren J. (eds.), Handbook of Pragmatics: Vol. 2, Benjamins, Amsterdam. https://doi.org/10.1075/hop.2.tex3

Pozzo A. 1995, Espressioni Lineari: Semiotica dell’Episodio 113 di Osvaldo Cavandoli, in Bendazzi G., Cecconello M. and Michelone G. (eds.), Coloriture. Voci, rumori, musiche nel cinema di animazione, Pendragon, Bologna, pp. 209-215.

Pylyshyn Z. 2003, Seeing and Visualizing: It’s Not What You Think. MIT Press, Cambridge, MA.

Rowley-Jolivet E. 2000, Image as Text: Aspects of the Shared Visual Language of Scientific Conference Participants, in “Asp” 27-30, pp. 133-154.

Rowley-Jolivet E. 2002, Visual Discourse in Scientific Conference Papers: A Genre-Based Study, in “English for Specific Purposes” 21 [1], pp. 19-40.

Rowley-Jolivet E. 2004, Different Visions, Different Visuals: A Social-Semiotic Analysis of Field-Specific Visual Composition in Scientific Conference Presentations, in “Visual Communication” 3 [2], pp. 145-175.

Saint-Martin F. 1987, Sémiologie du langage visual, Presses de l’Université de Québec, Québec.

Schnurr S. 2013, Exploring Professional Communication: Language in Action, Routledge, London.

Taylor H.A. and Tenbrink T. 2013, The Spatial Thinking of Origami: Evidence from Think-Aloud Protocols, in “Cognitive Processing” 14, pp. 189-191.

Tenbrink T. and Taylor H.A. 2015, Conceptual Transformation and Cognitive Processes in Origami Paper Folding, in “Journal of Problem Solving” 8. http://dx.doi.org/10.7771/1932-6246.1154.

Tenbrink T. and Maas A. 2015, Efficiently Connecting Textual and Visual Information in Operating Instructions, in “IEEE Transactions on Professional Communication” 58 [4], pp. 346-366.

Tufte E.R. 1997, Visual Explanations: Images and Quantities, Evidence and Narrative, Graphics Press, Cheshire.

Tufte E.R. 2001, The Visual Display of Quantitative Information, Graphics Press, Cheshire.

Van Leeuwen T. 2005, Introducing Social Semiotics, Routledge, London.

Van Olmen D. and Heinold S. 2017, Imperatives and Directive Strategies, Benjamins, Amsterdam.

Vertesi J. 2014, Drawing As: Distinctions and Disambiguation in Digital Images of Mars, in Coopmans C., Vertesi J., Lynch M. and Woolgar S. (eds.), Representation in Scientific Practice Revisited, MIT Press, Cambridge, MA, pp. 15-35.

Werlich E. 1976, A Text Grammar of English, Quelle & Meyer, Leipzig.


Full Text: pdf

Refbacks

  • There are currently no refbacks.
کاغذ a4

Creative Commons License
This work is licensed under a Creative Commons Attribuzione - Non commerciale - Non opere derivate 3.0 Italia License.