Visual Semiotics

Visual semiotics is a subdomain of semiotics that analyzes visual signs.

Studies of meaning evolve from semiotics, a philosophical approach that seeks to interpret messages in terms of their signs and patterns of symbolism. The study of semiotics, or semiology in France, originated in a literary or linguistic context and have been expanding in a number of directions since the early turn-of-the century work of Charles Sanders Peirce in the U.S. and Claude Lévi-Strauss and Ferdinand Saussure in France.

A sign can be a word, a sound, or a visual image. Saussure divides a sign into two components—the signifier (the sound, image, or word) and the signified, which is the concept the signifier represents, or the meaning. As Berger points out, the problem of meaning arises from the fact that the relation between the signifier and the signified is arbitrary and conventional. In other words, signs can mean anything we agree that they mean, and they can mean different things to different people. Nonverbal signs can produce many complex symbols and hold multiple meanings.

The Belgian Mu Group (Groupe µ) (founded 1967) developed a structural version of visual semiotics, on a cognitive basis, as well as a visual rhetoric.

Most signs operate on several levels—iconic as well as symbolic and/or indexical, which suggests that visual semiotic analysis may be addressing a hierarchy of meaning in addition to categories and components of meaning. As Umberto Eco explains, "what is commonly called a 'message' is in fact a text whose content is a multilevelled discourse".

The broadening concept of text and discourse encourages additional research into how visual communication operates to create meaning. Deely explains that "at the heart of semiotics is the realization that the whole of human experience, without exception, is an interpretive structure mediated and sustained by signs." Semiotics now considers a variety of texts, using Eco's terms, to investigate such diverse areas as movies, art, advertisements, and fashion, as well as visuals. In other words, as Berger explains, "the essential breakthrough of semiology is to take linguistics as a model and apply linguistic concepts to other phenomena--texts--and not just to language itself." Anthropologists like Grant McCracken and marketing experts like Sydney Levy have even used semiotic interpretations to analyze the rich cultural meanings of products and consumer consumption behaviors as texts.

Visual texts are an important area of analysis for semioticians and particularly for scholars working with visually intensive forms such as advertising and television because images are such a central part of our mass communication sign system. Linda Scott has deconstructed the images in perfume advertising as well as in Apple's "1984" commercial using close readings of the various messages which can be interpreted from the ads. Shay Sayre has also looked at perfume advertising images and the visual rhetoric in Hungary's first free election television advertisements using semiotic analysis. Also using semiotics, Arthur Asa Berger has deconstructed the meaning of the "1984" commercial as well as programs such as Cheers and films such as Murder on the Orient Express.

Systems of meaning, Culler and Berger tell us, is analyzed by looking at cultural and communication products and events as signs and then by looking at the relationship among these signs. The categories of signs and the relationships between them create a system. Barthes, for example, has analyzed the "fashion system," and classified the system of communication through fashion into two categories: image clothing and descriptive clothing. Likewise, an advertisement has its own system of meaning. We expect an appeal to purchase, either directly or implied, to be made and a product to be shown, for example, as part of the advertising system.

In their book Discourses in Place: Language in the material world, Ron Scollon and Suzie Wong Scollon note that visual semiotics has to do with turning “from the spoken, face-to-face discourses to the representations of that interaction order in images and signs” (82). Interaction order involves the various social interactions that take place in any setting, such as being alone, being with a companion, at a meeting, watching a show etc., and it “is almost always complex,” with various interactions occurring at once (83). When it comes to images, says Scollon and Wong, there are also multiple relationships. These include the relationships between the components of a visual image, the relationships between the producers of the visual image, the relationships between the producers and the components, as well as the relationships between the components of an image and those who are viewing it.

This interaction order, then, has four main semiotic systems, says Scollon and Wong. These include represented participants, modality, composition and interactive participants. Represented participants are elements of a visual image, and are either narrative (“present unfolding actions and events or… processes of change”) or conceptual (“show abstract, comparative or generalized categories”) (Scollon and Wong 86). Modality is how true to reality a visual image is, and main indicators include color saturation, color differentiation, depth, illumination and brightness, among others. With modality, it is often found “that truth, veracity, or sincerity might be expressed in very different ways from one society to another,” with Western cultures favoring naturalistic representation, or as true to seeing it in person as possible (Scollon and Wong 89-90).

Composition is the way in which represented participants within a visual image are arranged in relation to one another, with the three main systems of compositions being ideal-real(top to bottom), given-new(left to right), and center-margin relationships (Scollon and Wong 92). So for example, when reading a menu at a fast food restaurant, the given information would be something such as “hamburger,” and the new information would be the price, read left to right and recognized in that order. As mentioned above, interactive participants, explain Scollon and Wong, are the various relationships that occur around a visual image, such as those between the producers of the image and the represented participants in that image. In this way, these four components work together to help convey the meaning of signs and symbols.