Motion pictures as a source of empirical language data (a case study of counterfactuals)

DOI:

Thus, it appears that any science is meant to observe phenomena as experiences rather than objects ‘out there’, and it proves valid as long as it is experientially grounded, i.e. it can provide explanation of how, when and where this or that experience can recur without fail. In this respect, linguistics should pursue the same objectives. The present article will discuss why it does not happen and why mainstream theory of language has a too heavy legacy of Aristotle’s analytical logic to focus on human experience lived in the dynamics of communication. There will also be given a methodological alternative of studying cinema as language-in-motion which may help the language science to move from the (mono-)logical, text-oriented stance towards the eco-logical and experiential one.

Why linguistics tends to study static objects. Per Linell [2005] in his provocative critique of linguistics gives 101 reasons why mainstream linguistics chooses to approach language as literacy and the art of alphabetic writing rather than as human’s natural bodily behavior situationally embedded in the social and cultural environment. He explains how linguists have become biased in their research towards written, static texts ever since literacy began to play a big cultural and even political role. States have needed standardized national languages to stand out in the political arena, societies have needed educated people with good writing skills. Moreover, literacy has always served as a social distinction between those who can speak the proper and correct language and those who speak the vulgar vernacular. Language issues thus stand ‘proxy’ for wider social issues [Rickford, 1999, p. 272]. “Inscriptions, in the concrete sense of static marks on paper, have been the dominant technology of language. At the same time, it is in the development of literacy, in the schooling needed for learning to read and write, that theories of language structure have become necessary” [Linell, 2005, p. 5].

According to Linell, the age-long tradition of language studies is characterized by scriptism , i.e. the influence of writing on the conceptualization of speech. Speech is a dynamic activity distributed in real time with both speaker and listener being physically present in the immediate context, while texts are offline, displaced and deferred. In terms of the dialogical property of interactivity, the written text is “suspended dialogue” [Peters, 1999] lacking any immediate situational context [Linell, 2005, p. 21]. Suffice it to say that writing and reading belong to the so-called secondary socialization, while the primary empirical interactions in the form of speaking and hearing are overshadowed in texts.

Thus, the structure of speech, or talk-in-interaction, is mainly studied from the perspective of its regular counterparts in conventional writing. Even those linguistic disciplines concerned with the grammar or pragmatics of spoken language treat utterances (a unit of speech) in comparison to sentences (a unit of text) and analyze what is analyzable textually (‘spoken texts’ or ‘spoken grammar’). For example, utterances which do not display full clause structures are interpreted as ‘elliptical’, i.e. incomplete, lacking explicit proposition or even dysfunctional. In fact, any utterance in a spoken interaction may be called ‘elliptical’ because it is natural for participants in a dialogue to exploit properties of each other’s utterances and thereby dynamically co-construct meanings and attitudes [Bakhtin, 1981]. Another case in point is gestures. Gestures cannot be (fully) represented in writing, that is why their role in language analysis is often minimized. Indeed, gestures, bodily postures, facial expressions, gaze behavior are perfectly integrated with verbal conduct. Alongside with linguistic features, gestures help people construct meanings and attitudes and “exploit features of the built environment” [Goodwin, 2000].

It follows that the main methodological gap in scriptism as posited by Linell and his supporters is the failure to depart from dualism. The desire to look at isolated structures as if independently existing objects is a hallmark of the Cartesian philosophy which dualizes our understanding of the world in terms of subject-object relationships. This tradition has persisted in linguistics ever since literacy made it possible to observe inscriptions, marks and other textually demarcated symbols on paper. With writing, language becomes a static, ‘photographic’ (or rather, representational) object existing ‘out there’. “Linguists are not dealing with complex bodily conduct, behaviors, which exhibit recurrent structural properties. <...> Instead, they hypostatize, arrest, paralyze language structures as a body of knowledge, posit them as abstractions existing in and of themselves” [Linell, 2005, p. 9]. Another epistemological trap of scriptism is rationalism. Written texts are by definition logically organized intellectual products because writing is a purely linear affair. Studying texts linguists tend to find explanation not for what, how and why the author organized in his / her writing (it is physically impossible, as the context of writing is never observable), but for how the author’s writing can be described and thus logically organized once again. In such a way, mainstream linguistic theory aims to establish double logical coherences which are detached from the reality of our perceptual acting in the immediate context.

The overview of scriptism in linguistics allows us to arrive at one important conclusion: language studies are basically associated with textology; generalizations made by linguists as a result of their observations of speech are grounded mainly upon what is or is not common to see on paper, not in life . Indeed, language goes far beyond writing. If it were not the case, illiterate people would either be proclaimed a miracle or considered non-linguistic at all.

From texts to experiences. In his studies of children’s cognitive development, Leo Vygotsky became the first Soviet scientist who proved that language should not be approached analytically, because language is synthesis by definition.

Language, he maintained, is not what we write, nor what speak, but is what we do towards, in and through writing, speaking or any other social and cultural interaction. According to Vygotsky, “to understand another’s speech, it is not sufficient to understand his words – we must understand his thought. But even that is not enough – we must also know its motivation” [Vygotsky, 1962, p. 151]. He claims that traditional methods of verbal definition, by which he evidently meant describing things, are totally “inadequate for studying concepts... as they overlook the dynamics and the development of the process of concept formation” [Vygotsky, 1962, p. 96]. To find a term or definition for a concept is to deal with ready-made knowledge as the resulting product rather than the process of thinking or knowing. This method also confines research material to isolated verbalizations as the only source of empirics.

There are two important implications of this finding. First, language with all its structures, components or units cannot be investigated from within these units – it requires a much more holistic effort. Namely, we must look at the generative source of language, the process that takes place in a domain other than the domain of definitions or terms (cf.: [Maturana, Verden-Zöller, 2008, p. 154]). Second, we must integrate our abstractions back into the dynamics of bodily experience to check how they fit the conditions of our living.

Experiential view of language underlies the philosophy of enactivism in which language is equated with the operational dynamics of experience construction, or enactment of the world [Piaget, 1937; Varela, Thompson, Rosh, 1993; Glasersfeld, 1995]. There can be given three reasons to prove this point. First, “experience – that which we distinguish as happening to us and in us – cannot be denied” [Maturana, Verden-Zöller, 2008, p. 13]. Second, language is the root of all experience, i.e. all things happening to us and in us, since “the language I speak is my language, it makes me aware of myself” [Foerster, 2003, p. 297], it makes me an experiencer . Third, experience or language is not something readymade or passively received. It is actively constructed and organized in the course of cognitive development. Otherwise, we humans would not need to learn or live anything, instead we would be a machine waiting to be adjusted for the reception of the right signal.

What would be more appropriately termed as languaging is thus a cycle of changes happening to/in our body to reciprocally maintain the continuity of our focused and unfocused attention on “a sort of universe we shall appear to ourselves to inhabit” [James, 1890, p. 401]. The fact that our attention can be unfocused on some sensorial data and be still ‘experientially active’ is crucial for us humans: our attention once focusing on a perceptual item can refocus on it later by some power of volition and through recognition matrices [Ceccato, Zonta, 1980; Glasersfeld, Ackermann, 2011] which all, in turn, are made possible in and through languaging only [Kravchenko, 2016]. In this way we construct our experience out of the attentionally organized sensorial data, or else, our experiences. They are what we can easily isolate (abstract) from the sensorial flow, displace, or turn upon themselves (reflect upon them), and live again (enact) in a new context of interactions. Thus, languaging as experience construction can be characterized by the following ecology of enaction:

framing (isolating certain items from the bodily dynamics) + reference (interacting with these isolations: reflecting upon them and acting upon these reflections) + attention (selection of frames and their reference through changing interests and desires, or relations).

Thus, to study reference frames , what we commonly understand as names as well as their constellations – texts, means to study experiences , i.e. probe into our bodily operations (sensorimotor functioning and thinking processes) and their attentional triggers (relations).

Method of investigation of experiential dynamics in linguistic research

Thus, a hypothesis about a frame of reference as a linguistic abstraction must be based on the explanation of what, how, when and where we do ‘to make this reference happen’. It implies looking at the practical context of interactions in which this frame of reference is enacted to construct experience.

One way to do this is to study children at different times of their cognitive development. A problem may arise because interrelated complexes in languaging do not take shape by default but are actively constructed and tested through trial and error over a course of time when, later, they become a more or less stable and sustainable structure or pattern of action.

Another path to approach direct experience in its perceptual, empirical sense is to explain the circumstances under which it is enacted as an abstraction, or else, re-enacted as an already viable and stable conceptual structure, internalized pattern of action. In explaining the circumstances (when, where, why?) we will also be able to identify the mechanisms (how?) underlying this cognitive enactment. Such mechanisms are epistemological ways of concept construction in and through abstractions, generalizations, and reflections (for a detailed account see: [Piaget, 1937; Glasersfeld, 1995]).

For the sake of clarity, I have summarized the procedural components of open-ended experiential analysis after Varela et al. [1993] and Glasersfeld [1995] and propose the following step-by-step question-answer algorithm of scientific observation and explanation of languaging-as-experience-construction (see Table 1).

Table 1. Methodological algorithm of holistic research and open-ended experiential analysis of language

(Step 1) Hypothesizing: What experience is enacted in/through a particular frame of reference?

(Step 2) Observation and analysis: When / where is it enacted?

What are the experiential circumstances driving one to refer to this experience? What new emerging experiences and emotional perturbations ‘here and now’ cause one to recreate the old experience in question?

(Step 3) Observation and analysis: Why is it enacted?

What kind of desired result does one seek to achieve? What kind of experiential balance does one strive to restore?

(Step 4) Observation and analysis: How is it enacted?

What operations does one do to experience something again?

• -    Is it a direct experience that is acted out almost automatically, without much thinking? ( = Do I enact an empirical abstraction from raw sensory material? )

• -    Is it an indirect experience, i.e. one not only enacts an isolated item of (abstraction from) raw sensory material, but also reflects upon it to an extent that the experience is not re-created as it was?

• -    How exactly does one reflect upon the abstracted sensory material?

• -    Does one reflect upon the way one has this experience? (Does one change and develop an understanding of an experience to figure out how else one can have this experience? = Do I enact a reflective abstraction ?)

• -    Does one reflect upon the way one abstracts this experience? (Does one change or develop one’s understanding of understanding an experience = Do I enact a reflected abstraction? )

(Step 5) Synthesis and verification of the hypothesis: Does the hypothesis work? Is the hypothesized experience enacted in the observed circumstances non-controversially? Can the deduced hypothesis predict the happening of the experience under the circumstances similar to those observed? How is the investigated experience distinguished from other phenomena? What is a better term for the investigated experience?

Results and discussion

Motion pictures as enactive data for linguistic research

Why a picture in motion is better than a picture in isolation. To be able to grasp the dynamics of languaging, we should find ways to observe the experiential context of our semantic interactions, or else, our semantic living [Maturana, 2006]. The reason why it is not enough to study and describe language in its semantic abstractions is because isolated linear structures will never ‘tell us the whole (holistic) story’: they can only help de-fine, de-termine, find limitations to what we set out to observe (cf. step 1 in our methodological algorithm).

I will try to make it more explicit why motion pictures go beyond familiar textual data and are useful in any linguistic research which is aimed at studying experience.

• 1.    Uninterrupted non-linear dynamics: films help follow non-linear cognito-perceptual changes

  as they unfold in the human mind. There are no visible textual demarcations hindering us from experiencing the projected flow of living as it is. Motion pictures are free from the bias of linear written language where a reader is tempted into becoming a researcher by looking at structures and dialogical practices in isolation, as static product-oriented objects on paper [Linell, 2005] admitting of reading and rereading.

• 2.    Broader contextuality: it should be once again stressed that in films we can observe practical, empirical domains of every semantic operation, while in texts we are confined to the semantic domain of the same semantic operations. Communication like any other interaction must be observed and scientifically explained across the domains of ‘action in’ and ‘action out’ (or generative mechanism and resulting process), otherwise what we would be observing and explaining is not an inter -action.

• 3.    Distinctions between thinking and perceiving at one time: as a rule, films project not only something a character is perceiving, but also something a character is thinking. The director’s camera not only frames the perceptual dynamics of the character’s (or someone else’s) body, but it also captures the flow of attentional focalizations through which the character selects ways of relating the perceived dynamics to his / her earlier experiences. It means that films allow one to observe attentional processes that are unobservable in normal conditions (even in psychological experiments), let alone in texts (where we do not directly observe anything, but are engaged in reading only). Such a narrative technique of the moving image helps the viewer not only ‘experience’ the character’s living as it is, in the flow, but also follow the process of reality construction in the character’s world.

• 4.    Embodiment of knowing: films, as said before, project sensorimotor dynamics, thus making bodily movements (gaze, gestures, etc.) and sensoriality (hearing, seeing) observable.

We should remember that repeating an experience or interrupting the experiential flow is another experience. The director will never replay a series of shots for no particular reason, the director will make this replay a part of the continuous experiential flow as planned and determined by the story. Under ordinary circumstances, a viewer in the cinema cannot stop the movie and go back to the previous shot or the titles unlike a reader who sometimes has to refer to the footnotes, appendices, etc. The author of a book may choose to diverge from the narrative by introducing philosophical reflections that have little to do with the experiential dynamics of the character. The director of a film is unable to show anything but the flow of attention and experience of the characters or / and narrator in the projection of their living. In this way film watching can guarantee natural conditions for non-linear uninterrupted experiential dynamics.

Even if we rely on data from corpora, what we will be studying in the final analysis are written texts, transcripts as linear affairs. Even if we find instances of spoken communication, we will not be able to make explicit the actual conditions in which the interaction in point occurred. As objects of study, texts are always bounded, while experiential background to the object of our study is never bounded; therefore, (experiential) context is not wholly observable in texts. To hypothesize about the experiential nature of this or that semantic structure, we should go deeper into the person’s emotioning, thinking, and sensorimotor being in language. To make it possible, we must make sure we get ahold of the person’s relevant semantic living in totality. How can we make sure what is relevant and what is not in written texts?

A case study: a concept of time reversibility in/through counterfactuals. To demonstrate how effectively motion pictures can be methodologically applied to linguistic research, I will give one example that is connected with the study of some fundamental concepts and grammatical problems connected with them. The concept of time, in particular, can be easily approached from a dynamic perspective in which we can more clearly see the experiential context of time as a way of acting and then deduce (or even observe) the attentional and referential mechanisms underlying this acting. A choice of those films which tell their story around or throughout changing temporal reality will help. For instance, Atonement (2007) is interesting in this sense.

The analysis of how older Briony’s mind ‘moves’ in her reflections on the past as shown by the director in the last scene suggests that the realities constructed are not of the ‘more true’ or ‘less true’ kind, they belong to one and the same level of actuality of the storyline: they both are objects of Briony’s attention. Both the death and the happy life of her sister are depicted sequentially. Their counterfactuals status emerges as a new understanding of the past and a change of relation Briony produces in her actuality (‘here and now’, at the moment of answering the interviewer’s questions, at the moment of regretting her mistakes and getting more dissatisfied with her present life), not in the ‘irreal’ world.

The enactive data provided in the motion picture allow us to look at counterfactuals as a way of semantic living rather than a grammatical construction or logico-semantic structure. Briony lived a ‘counterfactual life’ throughout the second half of the story. Counterfactuals became for her a way of dealing with her sense of guilt, with the world that tragically changed for her the moment she made a life-changing decision to testify against her sister’s boyfriend. She enacted counterfactuals in and through her reflections upon the past where she re-organized the past experience by changing the relational value of this experience from the negative to the positive. The summary of our findings are given in Table 2.

The findings will help us make a few linguistic generalizations on counterfactuals. Linguistically, counterfactuals are frames of reference in which the agent reverses the past time , or rather, his / her lived experiences, to counterbalance what he / she is actually living in the present. In other words, the value of the lived experience is not satisfying for the agent to live on and anticipate the coming changes in a normal, linear direction; he / she begins to (reflectively) change the direction of this experiential flow and act upon this change by constructing new experience out of the old one.

This complex, non-linear process of attentional and experiential counterbalancing may well explain the complex and non-linear grammatical choices we make in our languaging.

Table 2. Counterfactuals as an experience (a case study of Atonement )

Step	Research Procedure	Results
Step 1	Hypothesizing	time is specifically experienced in/through a counterfactual way of dealing with the world
Step 2	Observation and analysis: when / where?	the experience is enacted after the character’s fatal mistake, in the circumstances of repentance and atoning
Step 3	Observation and analysis: why?	the experience is enacted to deal with a sense of guilt, to beg forgiveness from the wrongfully betrayed people
Step 4	Observation and analysis: how?	The experience is enacted attentionally : through refocalizations on the lived experiences, the refocalizations counterbalancing (helping to restore the balance in) the dynamics of the actual living; cognitively : through reflecting upon what happened in the past and acting upon this reflection (constructing a new experience by reverting the way the past experiences happened); relationally : through a change of value attached to the lived experience
Step 5	Synthesis and verification	The investigation fits the following hypothesis: a concept of time reversibility is enacted in/through counterfactuals; the experiential result (the enactment) is counterbalancing reality

Namely, the pattern If I only I had not done such a mistake, everything would have happened differently involves a number of complicated and ‘illogical’ structures for a reason. Contrasting negation (If I had ... = in fact, I don’t have) helps to create a relational change and reverse the experience, past tense forms specify what kind of the lived experience we are reversing (in a reflective abstraction) and ‘future-in-the past’ would frames the result of this reversion, namely the construction of a new, counterbalancing experience. All in all, the counterfactual enactment of the world creates counterbalancing reality in referential frames of which we act to satisfy our changing interests and desires.

To sum it up, the motion picture provided significant data for a holistic understanding of experience construction in terms of non-linear temporal reality. Different techniques of film making and our holistic perception of the story, the character’s living, rather than his / her talking, assured a maximum of methodological integration in the observation and synthetical study of languaging.

Conclusion

Motion pictures remain a good source of enactive data for linguistic research unless they are treated as texts. If they are, linguists find themselves “epistemologically trapped in the fragmentary understanding of language, culture and cognition” [Kravchenko, 2016]. Texts will never provide a whole picture of what human language is because in texts a human cannot be observed. In particular, texts are what we have already produced in the process of our experience organization, but they are far from this process itself, they have very little to say about how, where and when we organize our experience. If we cannot investigate such factors, we cannot construct a valid hypothesis about this or that experiential item generating our recurrent communicative behavior, or languaging dynamics. If we cannot do so, we fail to meet the requirements of scientific research to explain “how something works always”.

Viewed holistically, motion pictures can safely be used in the study of languaging as experience construction because in them we can observe actors acting upon, and enacting, bodily dynamics. Just as our ‘moving’ attention selects some part of our perceptual field to frame for reference, the moving camera selects an image to capture for the meaningful development of the story. Just as the flow of our living causes us to construct meaningful experience by relating one experiential item to another, the developing story of a film makes sense to let one event happen after, before or during another. Just as for a scientific understanding of reality we explain the recurrent coherences of our experiential dynamics across separate relational domains, films allow us to look at the experiential dynamics of the characters in totality which gives us a better understanding of the latter’s behaviors.

Linell expresses hopes that computer-supported visualization of languaging dynamics will offer a methodological alternative to the product-oriented approach of mainstream linguistics [Linell, 2005, p. 220]. Some work has been done in this regard with Druzhinin et al. [2020] attempting to investigate such relational phenomena as dysphemisms and euphemisms in experiential contexts on the basis of TV drama shows and concluding that the distinction between the ‘phemes’ depends upon the experience of the distinguisher. A lot of work is still to be done in the area of linguistic and cross-disciplinary research, and once we approach language in experiential totality, as where and how we humans organize our being, we will be able to write the long sought-after linguistic “recipes that work always”.