Academic literature on the topic 'Urban design; Artificial Intelligence; extended reality technologies'

Up until recently, learning affordances (possibilities) offered by immersive digital technology in education, such as augmented reality (AR) and virtual reality (VR), were addressed and considered in isolation in educational practice. In the past five to ten years this has shifted towards a focus on integrating digital affordances around particular learning contexts and/or settings, creating a mixed reality (MR) ‘continuum’ of digital experiences based on the combination of different technologies, tools, platforms and affordances. This idea of a ‘digital continuum’ was first proposed during the mid 1990s by Milgram and Kishino (1994), conceptualised as an immersive continuum going from the real environment (RE) end, where no digital immersion exists in the real world, all the way to the fully digitally immersive VR end, where digital immersion is at its full. Recent literature expands the original digital continuum view – rooted in Milgram and Kishino (1994), to now consider MR environments extending to a multi-variety of sensorial dimensions, technological tools and networked intelligent platforms, and embodied user engagement modes, creating interconnected learning ecosystems and modes of perception (see for example Mann et al., 2018; and Speicher, Hall & Nebeling, 2019). This new approach to MR is referred to as XR, where the X generally stands for ‘extended reality’ (referring to all the points along the MR continuum and beyond), or for ‘anything reality’ (accounting for the range of existing immersive technologies and denoting the imminently yet-to-come new digital affordances). XR as a multi-dimensional immersive learning environment can be approached and understood as a dynamic and culturally-responsive ‘medium’, offering targeted, flexible and adaptable user experiences coming from user-centric learning design strategies and pedagogy (Aguayo, Eames & Cochrane, 2020). Today, XR as an emergent learning approach in education invites us to re-conceptualise technology-enhanced learning from a completely different epistemological stand. We have moved from focusing on the individual and isolated use of immersive digital technology like AR and VR as ‘learning tools’ that can enhance and augment learning experiences and outcomes in education; to now going beyond hardware and software and consider perception, cognition, aesthetics, emotions, haptics, embodiment, contexts (space), situations (time), and culture, among others, as critical components of a purposefully designed XR learning ecosystem (Aguayo et al., 2020; Liu et al., 2017; Maas & Hughes, 2020). Imagine the educational possibilities when artificial intelligence (AI) learning algorithms connected to internet of things (IoT) devices come into play with XR in education (Cowling & Birt, 2020; Davies, 2021). The challenge remains in knowing how to ground such epistemological and technological innovation into authentic, contextual, and tangible practice, while facilitating the balancing with non-technology mediated lived experiences in the real world (i.e. real reality (RR), Aguayo, 2017). Here, a set of XR research and practice case studies from Auckland University of Technology’s AppLab are presented to showcase and discuss how XR as a new paradigm is leading the exploration of digital innovation in education.

The "information revolution" of our time affects our entire generation. While a vision of the "Information Society," with its financial, legal, business, privacy, and other aspects has emerged in the past few years, the "traditional scene" of information technology, that is, industrial automation, maintained its significance as a field of unceasing development. Since the old-fashioned concept of "Hard Automation" applicable only to industrial processes of fixed, repetitive nature and manufacturing large batches of the same product1)was thrust to the background by keen market competition, the key element of this development remained the improvement of "Machine Intelligence". In spite of the fact that L. A. Zadeh already introduced the concept of "Machine Intelligence Quotient" in 1996 to measure machine intelligence2) , this term remained more or less of a mysterious meaning best explicable on the basis of practical needs. The weak point of hard automation is that the system configuration and operations are fixed and cannot be changed without incurring considerable cost and downtime. Mainly it can be used in applications that call for fast and accurate operation in large batch production. Whenever a variety of products must be manufactured in small batches and consequently the work-cells of a production line should be quickly reconfigured to accommodate a change in products, hard automation becomes inefficient and fails due to economic reasons. In these cases, new, more flexible way of automation, so-called "Soft Automation," are expedient and suitable. The most important "ingredient" of soft automation is its adaptive ability for efficiently coping with changing, unexpected or previously unknown conditions, and working with a high degree of uncertainty and imprecision since in practice increasing precision can be very costly. This adaptation must be realized without or within limited human interference: this is one essential component of machine intelligence. Another important factor is that engineering practice often must deal with complex systems of multiple variable and multiple parameter models almost always with strong nonlinear coupling. Conventional analysis-based approaches for describing and predicting the behavior of such systems in many cases are doomed to failure from the outset, even in the phase of the construction of a more or less appropriate mathematical model. These approaches normally are too categorical in the sense that in the name of "modeling accuracy," they try to describe all structural details of the real physical system to be modeled. This significantly increases the intricacy of the model and may result in huge computational burden without considerably improving precision. The best paradigm exemplifying this situation may be the classic perturbation theory: the less significant the achievable correction is, the more work must be invested for obtaining it. Another important component of machine intelligence is a kind of "structural uniformity" giving room and possibility to model arbitrary particular details a priori not specified and unknown. This idea is similar to that of the ready-to-wear industry, whose products can later be slightly modified in contrast to the custom-tailors' made-to-measure creations aiming at maximum accuracy from the beginning. Machines carry out these later corrections automatically. This "learning ability" is another key element of machine intelligence. To realize the above philosophy in a mathematically correct way, L. A. Zadeh separated Hard Computing from Soft Computing. This revelation immediately resulted in distinguishing between two essential complementary branches of machine intelligence: Hard Computing based Artificial Intelligence and Soft Computing based Computational Intelligence. In the last decades, it became generally known that fuzzy logic, artificial neural networks, and probabilistic reasoning based Soft Computing is a fruitful orientation in designing intelligent systems. Moreover, it became generally accepted that soft computing rather than hard computing should be viewed as the foundation of real machine intelligence via exploiting the tolerance for imprecision, uncertainty and partial truth to achieve tractability, robustness, low solution cost and better rapport with reality. Further research in the past decade confirmed the view that typical components of present soft computing such as fuzzy logic, neurocomputing, evolutionary computation and probabilistic reasoning are complementary and best results can be obtained by their combined application. These complementary branches of Machine Intelligence, Artificial Intelligence and Computational Intelligence, serve as the basis of Intelligent Engineering Systems. The huge number of scientific results published in journals and conference proceedings worldwide substantiates this statement. Three years ago, a new series of conferences in this direction was initiated and launched with the support of several organizations including the IEEE Industrial Electronics Society and IEEE Hungary Section in technical cooperation with IEEE Robotics & Automation Society. The first event of the series hosted by Bdnki Dondt Polytechnic, Budapest, Hungary, was called "19997 IEEE International Conference on Intelligent Engineering Systems " (INES'97). The Technical University of Vienna, Austria hosted the next event of the series in 1998, followed by INES'99 held by the Technical University of Kosice, Slovakia. The present special issue consists of the extended and revised version of the most interesting papers selected out of the presentations of this conference. The papers exemplify recent development trends of intelligent engineering systems. The first paper pertains to the wider class of neural network applications. It is an interesting report of applying a special Adaptive Resonance Theory network for identifying objects in multispectral images. It is called "Extended Gaussian ARTMAP". The authors conclude that this network is especially advantageous for classification of large, low dimensional data sets. The second paper's subject belongs to the realm of fuzzy systems. It reports successful application of fundamental similarity relations in diagnostic systems. As an example failure detection of rolling-mill transmission is considered. The next paper represents the AI-branch of machine intelligence. The paper is a report on an EU-funded project focusing on the storage of knowledge in a corporate organizational memory used for storing and retrieving knowledge chunks for it. The flexible structure of the system makes it possible to adopt it to different SMEs via using company-specific conceptual terms rather than traditional keywords. The fourth selected paper's contribution is to the field of knowledge discovery. For this purpose in the first step, cluster analysis is done. The method is found to be helpful whenever little or no information on the characteristics of a given data set is available. The next paper approaches scheduling problems by the application of the multiagent system. It is concluded that due to the great number of interactions between components, MAS seems to be well suited for manufacturing scheduling problems. The sixth selected paper's topic is emerging intelligent technologies in computer-aided engineering. It discusses key issues of CAD/CAM technology of our days. The conclusion is that further development of CAD/CAM methods probably will serve companies on the competitive edge. The seventh paper of the selection is a report on seeking a special tradeoff between classical analytical modeling and traditional soft computing. It nonconventionally integrates uniform structures obtained from Lagrangian Classical Mechanics with other simple elements of machine intelligence such as saturated sigmoid transition functions borrowed from neural nets, and fuzzy rules with classical PID/ST, and a simplified version of regression analysis. It is concluded that these different components can successfully cooperate in adaptive robot control. The last paper focuses on the complexity problem of fuzzy and neural network approaches. A fuzzy rule base, be it generated from expert operators or by some learning or identification schemes, may contain redundant, weakly contributing, or outright inconsistent components. Moreover, in pursuit of good approximation, one may be tempted to overly assign the number of antecedent sets, thereby resulting in large fuzzy rule bases and much problems in computation time and storage space. Engineers using neural networks have to face the same complexity problem with the number of neurons and layers. A fuzzy rule base and neural network design, hence, have two important objectives. One is to achieve a good approximation. The other is to reduce the complexity. The main difficulty is that these two objectives are contradictory. A formal approach to extracting the more pertinent elements of a given rule set or neurons is, hence, highly desirable. The last paper is an attempt in this direction. References 1)C. W. De Silva. Automation Intelligence. Engineering Application of Artificial Intelligence. Vol. 7. No. 5. 471-477 (1994). 2)L. A. Zadeh. Fuzzy Logic, Neural Networks and Soft Computing. NATO Advanced Studies Institute on Soft Computing and Its Application. Antalya, Turkey. (1996). 3)L. A. Zadeh. Berkeley Initiative in Soft Computing. IEEE Industrial Electronics Society Newsletter. 41, (3), 8-10 (1994).

According to the top-priority trends and challenges in the mineral sector, and as per the mining science strategy, it is highly critical to arrange enhanced control, prediction and safety of production objects and their functioning for the preservation of automation sustainability. Improved control of databases, regulatory bonds, management, logistics and principles of sustainable development in mining makes it possible to reduce technological deviations and accidents at large mining and processing plants. Most procedural violations and accidents in surface and underground mines occur because of the unskilled actions of process flow operators. Damage in this case can be considerable, especially as compared with the expenses connected with qualitative training and persistent development of personnel engaged with supervisory control and data acquisition for the efficient operation of SCADA-systems within the automation framework of mining and processing plants. Definition of digital systems and their interrelation with multilevel automated control can be incorrect. The review of new principles can awaken interest in the conceptual assessment of digitalization processes using such notions as: numerical models, simulator, and artificial intelligence. Often applied formulations and principles of a digital model are substituted without justification of functional connections. On the other hand, a digital system today can be assumed as robotic lines and other numerical models and smart technologies, for instance, machining stations with numerical program control. It is necessary to define the practical significance of conceptual modifications and digital transformation regarding objects of the mineral sector, using Big Data; to understand how a digital twin can influence a changeable process situation; to provide prompt prediction; to eliminate an accident; and to preserve the physical balance in the whole production system. Such intelligent and flexible productions particularly need computerbased simulators and digital twins based on technologies of Industry 4.0–extended and virtual reality on the basis of digital twins. Digital twins allow maximal simulation of real-life activity of process flow operators. The skills acquired by personnel after such simulation training enable operators to master the optimized procedure for functioning in emergency situations in mineral mining and processing. This paper exemplifies the remote training and control of process flows, which is of concern in the current international situation.

Augmented Reality—The Liminal Zone Within the larger context of the post-desktop technological philosophy and practice, an increasing number of efforts are directed towards finding solutions for integrating as close as possible virtual information into specific real environments; a short list of such endeavors include Wi-Fi connectivity, GPS-driven navigation, mobile phones, GIS (Geographic Information System), and various technological systems associated with what is loosely called locative, ubiquitous and pervasive computing. Augmented Reality (AR) is directly related to these technologies, although its visualization capabilities and the experience it provides assure it a particular place within this general trend. Indeed, AR stands out for its unique capacity (or ambition) to offer a seamless combination—or what I call here an effect of convergence—of the real scene perceived by the user with virtual information overlaid on that scene interactively and in real time. The augmented scene is perceived by the viewer through the use of different displays, the most common being the AR glasses (head-mounted display), video projections or monitors, and hand-held mobile devices such as smartphones or tablets, increasingly popular nowadays. One typical example of AR application is Layar, a browser that layers information of public interest—delivered through an open-source content management system—over the actual image of a real space, streamed live on the mobile phone display. An increasing number of artists employ this type of mobile AR apps to create artworks that consist in perceptually combining material reality and virtual data: as the user points the smartphone or tablet to a specific place, virtual 3D-modelled graphics or videos appear in real time, seamlessly inserted in the image of that location, according to the user’s position and orientation. In the engineering and IT design fields, one of the first researchers to articulate a coherent conceptualization of AR and to underlie its specific capabilities is Ronald Azuma. He writes that, unlike Virtual Reality (VR) which completely immerses the user inside a synthetic environment, AR supplements reality, therefore enhancing “a user’s perception of and interaction with the real world” (355-385). Another important contributor to the foundation of AR as a concept and as a research field is industrial engineer Paul Milgram. He proposes a comprehensive and frequently cited definition of “Mixed Reality” (MR) via a schema that includes the entire spectrum of situations that span the “continuum” between actual reality and virtual reality, with “augmented reality” and “augmented virtuality” between the two poles (283). Important to remark with regard to terminology (MR or AR) is that especially in the non-scientific literature, authors do not always explain a preference for either MR or AR. This suggests that the two terms are understood as synonymous, but it also provides evidence for my argument that, outside of the technical literature, AR is considered a concept rather than a technology. Here, I use the term AR instead of MR considering that the phrase AR (and the integrated idea of augmentation) is better suited to capturing the convergence effect. As I will demonstrate in the following lines, the process of augmentation (i.e. the convergence effect) is the result of an enhancement of the possibilities to perceive and understand the world—through adding data that augment the perception of reality—and not simply the product of a mix. Nevertheless, there is surely something “mixed” about this experience, at least for the fact that it combines reality and virtuality. The experiential result of combining reality and virtuality in the AR process is what media theorist Lev Manovich calls an “augmented space,” a perceptual liminal zone which he defines as “the physical space overlaid with dynamically changing information, multimedia in form and localized for each user” (219). The author derives the term “augmented space” from the term AR (already established in the scientific literature), but he sees AR, and implicitly augmented space, not as a strictly defined technology, but as a model of visuality concerned with the intertwining of the real and virtual: “it is crucial to see this as a conceptual rather than just a technological issue – and therefore as something that in part has already been an element of other architectural and artistic paradigms” (225-6). Surely, it is hard to believe that AR has appeared in a void or that its emergence is strictly related to certain advances in technological research. AR—as an artistic manifestation—is informed by other attempts (not necessarily digital) to merge real and fictional in a unitary perceptual entity, particularly by installation art and Virtual Reality (VR) environments. With installation art, AR shares the same spatial strategy and scenographic approach—they both construct “fictional” areas within material reality, that is, a sort of mise-en-scène that are aesthetically and socially produced and centered on the active viewer. From the media installationist practice of the previous decades, AR inherited the way of establishing a closer spatio-temporal interaction between the setting, the body and the electronic image (see for example Bruce Nauman’s Live-Taped Video Corridor [1970], Peter Campus’s Interface [1972], Dan Graham’s Present Continuous Pasts(s) [1974], Jeffrey Shaw’s Viewpoint [1975], or Jim Campbell’s Hallucination [1988]). On the other hand, VR plays an important role in the genealogy of AR for sharing the same preoccupation for illusionist imagery and—at least in some AR projects—for providing immersive interactions in “expanded image spaces experienced polysensorily and interactively” (Grau 9). VR artworks such as Paul Sermon, Telematic Dreaming (1992), Char Davies’ Osmose (1995), Michael Naimark’s Be Now Here (1995-97), Maurice Benayoun’s World Skin: A Photo Safari in the Land of War (1997), Luc Courchesne’s Where Are You? (2007-10), are significant examples for the way in which the viewer can be immersed in “expanded image-spaces.” Offering no view of the exterior world, the works try instead to reduce as much as possible the critical distance the viewer might have to the image he/she experiences. Indeed, AR emerged in great part from the artistic and scientific research efforts dedicated to VR, but also from the technological and artistic investigations of the possibilities of blending reality and virtuality, conducted in the previous decades. For example, in the 1960s, computer scientist Ivan Sutherland played a crucial role in the history of AR contributing to the development of display solutions and tracking systems that permit a better immersion within the digital image. Another important figure in the history of AR is computer artist Myron Krueger whose experiments with “responsive environments” are fundamental as they proposed a closer interaction between participant’s body and the digital object. More recently, architect and theorist Marcos Novak contributed to the development of the idea of AR by introducing the concept of “eversion”, “the counter-vector of the virtual leaking out into the actual”. Today, AR technological research and the applications made available by various developers and artists are focused more and more on mobility and ubiquitous access to information instead of immersivity and illusionist effects. A few examples of mobile AR include applications such as Layar, Wikitude—“world browsers” that overlay site-specific information in real-time on a real view (video stream) of a place, Streetmuseum (launched in 2010) and Historypin (launched in 2011)—applications that insert archive images into the street-view of a specific location where the old images were taken, or Google Glass (launched in 2012)—a device that provides the wearer access to Google’s key Cloud features, in situ and in real time. Recognizing the importance of various technological developments and of the artistic manifestations such as installation art and VR as predecessors of AR, we should emphasize that AR moves forward from these artistic and technological models. AR extends the installationist precedent by proposing a consistent and seamless integration of informational elements with the very physical space of the spectator, and at the same time rejects the idea of segregating the viewer into a complete artificial environment like in VR systems by opening the perceptual field to the surrounding environment. Instead of leaving the viewer in a sort of epistemological “lust” within the closed limits of the immersive virtual systems, AR sees virtuality rather as a “component of experiencing the real” (Farman 22). Thus, the questions that arise—and which this essay aims to answer—are: Do we have a specific spatial dimension in AR? If yes, can we distinguish it as a different—if not new—spatial and aesthetic paradigm? Is AR’s intricate topology able to be the place not only of convergence, but also of possible tensions between its real and virtual components, between the ideal of obtaining a perceptual continuity and the inherent (technical) limitations that undermine that ideal? Converging Spaces in the Artistic Mode: Between Continuum and Discontinuum As key examples of the way in which AR creates a specific spatial experience—in which convergence appears as a fluctuation between continuity and discontinuity—I mention three of the most accomplished works in the field that, significantly, expose also the essential role played by the interface in providing this experience: Living-Room 2 (2007) by Jan Torpus, Under Scan (2005-2008) by Rafael Lozano-Hemmer and Hans RichtAR (2013) by John Craig Freeman and Will Pappenheimer. The works illustrate the three main categories of interfaces used for AR experience: head-attached, spatial displays, and hand-held (Bimber 2005). These types of interface—together with all the array of adjacent devices, software and tracking systems—play a central role in determining the forms and outcomes of the user’s experience and consequently inform in a certain measure the aesthetic and socio-cultural interpretative discourse surrounding AR. Indeed, it is not the same to have an immersive but solitary experience, or a mobile and public experience of an AR artwork or application. The first example is Living-Room 2 an immersive AR installation realized by a collective coordinated by Jan Torpus in 2007 at the University of Applied Sciences and Arts FHNW, Basel, Switzerland. The work consists of a built “living-room” with pieces of furniture and domestic objects that are perceptually augmented by means of a “see-through” Head Mounted Display. The viewer perceives at the same time the real room and a series of virtual graphics superimposed on it such as illusionist natural vistas that “erase” the walls, or strange creatures that “invade” the living-room. The user can select different augmenting “scenarios” by interacting with both the physical interfaces (the real furniture and objects) and the graphical interfaces (provided as virtual images in the visual field of the viewer, and activated via a handheld device). For example, in one of the scenarios proposed, the user is prompted to design his/her own extended living room, by augmenting the content and the context of the given real space with different “spatial dramaturgies” or “AR décors.” Another scenario offers the possibility of creating an “Ecosystem”—a real-digital world perceived through the HMD in which strange creatures virtually occupy the living-room intertwining with the physical configuration of the set design and with the user’s viewing direction, body movement, and gestures. Particular attention is paid to the participant’s position in the room: a tracking device measures the coordinates of the participant’s location and direction of view and effectuates occlusions of real space and then congruent superimpositions of 3D images upon it. Figure 1: Jan Torpus, Living-Room 2 (Ecosystems), Augmented Reality installation (2007). Courtesy of the artist. Figure 2: Jan Torpus, Living-Room 2 (AR decors), Augmented Reality installation (2007). Courtesy of the artist.In this sense, the title of the work acquires a double meaning: “living” is both descriptive and metaphoric. As Torpus explains, Living-Room is an ambiguous phrase: it can be both a living-room and a room that actually lives, an observation that suggests the idea of a continuum and of immersion in an environment where there are no apparent ruptures between reality and virtuality. Of course, immersion is in these circumstances not about the creation of a purely artificial secluded space of experience like that of the VR environments, but rather about a dialogical exercise that unifies two different phenomenal levels, real and virtual, within a (dis)continuous environment (with the prefix “dis” as a necessary provision). Media theorist Ron Burnett’s observations about the instability of the dividing line between different levels of experience—more exactly, of the real-virtual continuum—in what he calls immersive “image-worlds” have a particular relevance in this context: Viewing or being immersed in images extend the control humans have over mediated spaces and is part of a perceptual and psychological continuum of struggle for meaning within image-worlds. Thinking in terms of continuums lessens the distinctions between subjects and objects and makes it possible to examine modes of influence among a variety of connected experiences. (113) It is precisely this preoccupation to lessen any (or most) distinctions between subjects and objects, and between real and virtual spaces, that lays at the core of every artistic experiment under the AR rubric. The fact that this distinction is never entirely erased—as Living-Room 2 proves—is part of the very condition of AR. The ambition to create a continuum is after all not about producing perfectly homogenous spaces, but, as Ron Burnett points out (113), “about modalities of interaction and dialogue” between real worlds and virtual images. Another way to frame the same problematic of creating a provisional spatial continuum between reality and virtuality, but this time in a non-immersive fashion (i.e. with projective interface means), occurs in Rafael Lozano-Hemmer’s Under Scan (2005-2008). The work, part of the larger series Relational Architecture, is an interactive video installation conceived for outdoor and indoor environments and presented in various public spaces. It is a complex system comprised of a powerful light source, video projectors, computers, and a tracking device. The powerful light casts shadows of passers-by within the dark environment of the work’s setting. A tracking device indicates where viewers are positioned and permits the system to project different video sequences onto their shadows. Shot in advance by local videographers and producers, the filmed sequences show full images of ordinary people moving freely, but also watching the camera. As they appear within pedestrians’ shadows, the figurants interact with the viewers, moving and establishing eye contact. Figure 3: Rafael Lozano-Hemmer, Under Scan (Relational Architecture 11), 2005. Shown here: Trafalgar Square, London, United Kingdom, 2008. Photo by: Antimodular Research. Courtesy of the artist. Figure 4: Rafael Lozano-Hemmer, Under Scan (Relational Architecture 11), 2005. Shown here: Trafalgar Square, London, United Kingdom, 2008. Photo by: Antimodular Research. Courtesy of the artist. One of the most interesting attributes of this work with respect to the question of AR’s (im)possible perceptual spatial continuity is its ability to create an experientially stimulating and conceptually sophisticated play between illusion and subversion of illusion. In Under Scan, the integration of video projections into the real environment via the active body of the viewer is aimed at tempering as much as possible any disparities or dialectical tensions—that is, any successive or alternative reading—between real and virtual. Although non-immersive, the work fuses the two levels by provoking an intimate but mute dialogue between the real, present body of the viewer and the virtual, absent body of the figurant via the ambiguous entity of the shadow. The latter is an illusion (it marks the presence of a body) that is transcended by another illusion (video projection). Moreover, being “under scan,” the viewer inhabits both the “here” of the immediate space and the “there” of virtual information: “the body” is equally a presence in flesh and bones and an occurrence in bits and bytes. But, however convincing this reality-virtuality pseudo-continuum would be, the spatial and temporal fragmentations inevitably persist: there is always a certain break at the phenomenological level between the experience of real space, the bodily absence/presence in the shadow, and the displacements and delays of the video image projection. Figure 5: John Craig Freeman and Will Pappenheimer, Hans RichtAR, augmented reality installation included in the exhibition “Hans Richter: Encounters”, Los Angeles County Museum of Art, 2013. Courtesy of the artists. Figure 6: John Craig Freeman and Will Pappenheimer, Hans RichtAR, augmented reality installation included in the exhibition “Hans Richter: Encounters”, Los Angeles County Museum of Art, 2013. Courtesy of the artists. The third example of an AR artwork that engages the problem of real-virtual spatial convergence as a play between perceptual continuity and discontinuity, this time with the use of hand-held mobile interface is Hans RichtAR by John Craig Freeman and Will Pappenheimer. The work is an AR installation included in the exhibition “Hans Richter: Encounters” at Los Angeles County Museum of Art, in 2013. The project recreates the spirit of the 1929 exhibition held in Stuttgart entitled Film und Foto (“FiFo”) for which avant-garde artist Hans Richter served as film curator. Featured in the augmented reality is a re-imaging of the FiFo Russian Room designed by El Lissitzky where a selection of Russian photographs, film stills and actual film footage was presented. The users access the work through tablets made available at the exhibition entrance. Pointing the tablet at the exhibition and moving around the room, the viewer discovers that a new, complex installation is superimposed on the screen over the existing installation and gallery space at LACMA. The work effectively recreates and interprets the original design of the Russian Room, with its scaffoldings and surfaces at various heights while virtually juxtaposing photography and moving images, to which the authors have added some creative elements of their own. Manipulating and converging real space and the virtual forms in an illusionist way, AR is able—as one of the artists maintains—to destabilize the way we construct representation. Indeed, the work makes a statement about visuality that complicates the relationship between the visible object and its representation and interpretation in the virtual realm. One that actually shows the fragility of establishing an illusionist continuum, of a perfect convergence between reality and represented virtuality, whatever the means employed. AR: A Different Spatial Practice Regardless the degree of “perfection” the convergence process would entail, what we can safely assume—following the examples above—is that the complex nature of AR operations permits a closer integration of virtual images within real space, one that, I argue, constitutes a new spatial paradigm. This is the perceptual outcome of the convergence effect, that is, the process and the product of consolidating different—and differently situated—elements in real and virtual worlds into a single space-image. Of course, illusion plays a crucial role as it makes permeable the perceptual limit between the represented objects and the material spaces we inhabit. Making the interface transparent—in both proper and figurative senses—and integrating it into the surrounding space, AR “erases” the medium with the effect of suspending—at least for a limited time—the perceptual (but not ontological!) differences between what is real and what is represented. These aspects are what distinguish AR from other technological and artistic endeavors that aim at creating more inclusive spaces of interaction. However, unlike the CAVE experience (a display solution frequently used in VR applications) that isolates the viewer within the image-space, in AR virtual information is coextensive with reality. As the example of the Living-Room 2 shows, regardless the degree of immersivity, in AR there is no such thing as dismissing the real in favor of an ideal view of a perfect and completely controllable artificial environment like in VR. The “redemptive” vision of a total virtual environment is replaced in AR with the open solution of sharing physical and digital realities in the same sensorial and spatial configuration. In AR the real is not denounced but reflected; it is not excluded, but integrated. Yet, AR distinguishes itself also from other projects that presuppose a real-world environment overlaid with data, such as urban surfaces covered with screens, Wi-Fi enabled areas, or video installations that are not site-specific and viewer inclusive. Although closely related to these types of projects, AR remains different, its spatiality is not simply a “space of interaction” that connects, but instead it integrates real and virtual elements. Unlike other non-AR media installations, AR does not only place the real and virtual spaces in an adjacent position (or replace one with another), but makes them perceptually convergent in an—ideally—seamless way (and here Hans RichtAR is a relevant example). Moreover, as Lev Manovich notes, “electronically augmented space is unique – since the information is personalized for every user, it can change dynamically over time, and it is delivered through an interactive multimedia interface” (225-6). Nevertheless, as our examples show, any AR experience is negotiated in the user-machine encounter with various degrees of success and sustainability. Indeed, the realization of the convergence effect is sometimes problematic since AR is never perfectly continuous, spatially or temporally. The convergence effect is the momentary appearance of continuity that will never take full effect for the viewer, given the internal (perhaps inherent?) tensions between the ideal of seamlessness and the mostly technical inconsistencies in the visual construction of the pieces (such as real-time inadequacy or real-virtual registration errors). We should note that many criticisms of the AR visualization systems (being them practical applications or artworks) are directed to this particular aspect related to the imperfect alignment between reality and digital information in the augmented space-image. However, not only AR applications can function when having an estimated (and acceptable) registration error, but, I would state, such visual imperfections testify a distinctive aesthetic aspect of AR. The alleged flaws can be assumed—especially in the artistic AR projects—as the “trace,” as the “tool’s stroke” that can reflect the unique play between illusion and its subversion, between transparency of the medium and its reflexive strategy. In fact this is what defines AR as a different perceptual paradigm: the creation of a convergent space—which will remain inevitably imperfect—between material reality and virtual information.References Azuma, Ronald T. “A Survey on Augmented Reality.” Presence: Teleoperators and Virtual Environments 6.4 (Aug. 1997): 355-385. < http://www.hitl.washington.edu/projects/knowledge_base/ARfinal.pdf >. Benayoun, Maurice. World Skin: A Photo Safari in the Land of War. 1997. Immersive installation: CAVE, Computer, video projectors, 1 to 5 real photo cameras, 2 to 6 magnetic or infrared trackers, shutter glasses, audio-system, Internet connection, color printer. Maurice Benayoun, Works. < http://www.benayoun.com/projet.php?id=16 >. Bimber, Oliver, and Ramesh Raskar. Spatial Augmented Reality. Merging Real and Virtual Worlds. Wellesley, Massachusetts: AK Peters, 2005. 71-92. Burnett, Ron. How Images Think. Cambridge, Mass.: MIT Press, 2004. Campbell, Jim. Hallucination. 1988-1990. Black and white video camera, 50 inch rear projection video monitor, laser disc players, custom electronics. Collection of Don Fisher, San Francisco. Campus, Peter. Interface. 1972. Closed-circuit video installation, black and white camera, video projector, light projector, glass sheet, empty, dark room. Centre Georges Pompidou Collection, Paris, France. Courchesne, Luc. Where Are You? 2005. Immersive installation: Panoscope 360°. a single channel immersive display, a large inverted dome, a hemispheric lens and projector, a computer and a surround sound system. Collection of the artist. < http://courchel.net/# >. Davies, Char. Osmose. 1995. Computer, sound synthesizers and processors, stereoscopic head-mounted display with 3D localized sound, breathing/balance interface vest, motion capture devices, video projectors, and silhouette screen. Char Davies, Immersence, Osmose. < http://www.immersence.com >. Farman, Jason. Mobile Interface Theory: Embodied Space and Locative Media. New York: Routledge, 2012. Graham, Dan. Present Continuous Past(s). 1974. Closed-circuit video installation, black and white camera, one black and white monitor, two mirrors, microprocessor. Centre Georges Pompidou Collection, Paris, France. Grau, Oliver. Virtual Art: From Illusion to Immersion. Translated by Gloria Custance. Cambridge, Massachusetts, London: MIT Press, 2003. Hansen, Mark B.N. New Philosophy for New Media. Cambridge, Mass.: MIT Press, 2004. Harper, Douglas. Online Etymology Dictionary, 2001-2012. < http://www.etymonline.com >. Manovich, Lev. “The Poetics of Augmented Space.” Visual Communication 5.2 (2006): 219-240. Milgram, Paul, Haruo Takemura, Akira Utsumi, Fumio Kishino. “Augmented Reality: A Class of Displays on the Reality-Virtuality Continuum.” SPIE [The International Society for Optical Engineering] Proceedings 2351: Telemanipulator and Telepresence Technologies (1994): 282-292. Naimark, Michael, Be Now Here. 1995-97. Stereoscopic interactive panorama: 3-D glasses, two 35mm motion-picture cameras, rotating tripod, input pedestal, stereoscopic projection screen, four-channel audio, 16-foot (4.87 m) rotating floor. Originally produced at Interval Research Corporation with additional support from the UNESCO World Heritage Centre, Paris, France. < http://www.naimark.net/projects/benowhere.html >. Nauman, Bruce. Live-Taped Video Corridor. 1970. Wallboard, video camera, two video monitors, videotape player, and videotape, dimensions variable. Solomon R. Guggenheim Museum, New York. Novak, Marcos. Interview with Leo Gullbring, Calimero journalistic och fotografi, 2001. < http://www.calimero.se/novak2.htm >. Sermon, Paul. Telematic Dreaming. 1992. ISDN telematic installation, two video projectors, two video cameras, two beds set. The National Museum of Photography, Film & Television in Bradford England. Shaw, Jeffrey, and Theo Botschuijver. Viewpoint. 1975. Photo installation. Shown at 9th Biennale de Paris, Musée d'Art Moderne, Paris, France.

For Socrates the act of communication is grounded in the world of original forms, archetypes, or abstract ideas. These ideas exist independently of the human mind and reflect a reality that is truer than the world of everyday experience. The task of the speaker is to draw the listener closer to the truth of these ideas, and this requires an intimate coupling of the form of speech to the character of the listener. In Phaedrus, Socrates explains, ". . . a would-be speaker must know how many types of soul there are. The number is finite, and they account for a variety of individual characters. When these are determined one must enumerate the various types of speech, a finite number also." The types of soul must then be carefully paired with the types of speech. This theoretical knowledge by itself, however, is not sufficient. A speaker must also know when ". . . he has actually before him a specific example of a type which he has heard described, and that this is what he must say and this is how he must say it if he wants to influence his hearer in this particular way" (Plato 91-92). Thus, the aspiring speaker must sharpen his powers of observation. Exactly how a speaker goes about discerning the various types of souls in his audience is not discussed in Phaedrus, but one assumes it is by mastering the art of face reading, or physiognomy. The science of physiognomy was of particular importance to the ancient Greeks. Nearly all the well-known Greek writers had something to say about the subject. Pythagoras is claimed to be the first Greek to formalize it systematically as a science. Hippocrates wrote voluminously on the subject, as did Aristotle. Socrates not only recommended physiognomy to his students (Tytler) but he is also reported to have demonstrated his facility with the science at least twice: once in predicting the promotion of Alcibiades and once upon first meeting Plato, whom he immediately recognized as a man of considerable philosophical talent (Encyclopedia Britannica). Writing is inferior to speech, Socrates tells Phaedrus, precisely because it cannot see and adapt the message to the reader. Like a painting of an object, writing is merely the image of dissociated speech. What is missing in writing-and what writing seems ever intent on reconstructing-is the human face. Pressing Faces Behind the Typeface Although physiognomy was banned by the Church, as it was associated with paganism and devil worship (practitioners of the science were burnt at the stake), it was revived in the Renaissance and became an obsession with the advent of the printing press. The printing press heralds democracy. But as human rights grew, urban centers developed, and new professions and classes emerged, people were no longer able to divine their own destiny or to predict the behavior and destiny of others. It became imperative to find other more reliable means of identifying the good and the bad, the talented and the unremarkable. Two books were considered indispensable: the Bible and Lavater's Essays on Physiognomy (Juengel). Physiognomy was the science that helped people decipher class and profession. It became the spelling book of character, one that people diligently studied so that they could learn to read not only the marks of character in others but also the signs of talent and potential in their own faces and in the faces of their children. Face reading was egalitarian and leveling (Juengel). The heads of state could be read and debunked in the flourishing art of caricature, and people delighted in decoding the physiognomy of the ordinary faces that crowded the pages of the popular press. The populace applauded the artists that succeeded in revealing the whole spectrum of a character-class, intentions, profession-in the masterly strokes of the pen (Wechsler). Unfortunately, so intense was the interest in face reading that many people were forced to cover their faces when out in public (Zebrowitz). Inside the religious, medical, educational, and criminal justice institutions, authorities scanned faces to identify the virtuous and the vile. People were hanged because of the shape of their jaws ("A physiognomic auto-da-fé,") and sometimes convicted of crimes because of an unfortunate physiognomy, even before any crimes were committed (Lichtenberg. 93). Mass Consumption of the Face Open a magazine. What do you see? I counted over 200 faces in the September 15, 2003 issue of Newsweek, 120 faces in the September 29, 2003 issue of Forbes, 124 faces in the September 15, 2003 issue of Time, and 37 faces in the November, 2003 issue of Handgunner (I included the masked faces). Whereas, in the 19th century, face reading was used by the religious, medical, and criminal justice authorities to identify a person's character, in the modern world face reading becomes face righting. Early in the century, people came to be viewed less as individuals than as masses that were dynamically statistical with fluctuations of opinions and tastes that could be sampled and manipulated. It quickly became apparent to the behemoth advertising industry that was erected with the advent of mass media that product designs and packages could be collated with viewer reactions. The audience is scrutinized, labeled, and targeted. What people are fed are fleshless images of themselves. Horkheimer and Adorno have observed that the media have reduced the individual to the stereotype. Stereotypes package people, typically in unflattering boxes. Mediated faces are used to mirror, to prime, and to manipulate the audience (Kress and van Leeuwen). On television and in print, images of canned faces proliferate. Not all stereotypes are unsavory. Nothing recommends itself nor sells like a beautiful face, but even beautiful faces must be retouched, even recomposed from features extracted from databases of perfect facial features. So important is the image of the face that media icons routinely visit the plastic surgeon. Michael Jackson is the most extreme example of what has been derogatorily termed a "scalpel slave." Plastic surgery is not exclusive to celebrities; countless millions of ordinary Americans feel compelled to undergo various cosmetic surgeries. The 20th-century consumption of the face has ended by consuming the face. Facing the Face Interface Text has made a comeback in hypertext. Empowered by the hyperlink, readers have become writers as they assemble texts with the clicks of a mouse (Landow). Electronic texts are pushed as well as pulled. Businesses have learned to track and to query users, building individual profiles that are then used to assemble personalized pages and email messages. Socrates' objection that writing is unable to perceive the reader no longer holds. The virtual text is watching you. And it is watching you with virtual eyes. There is a growing interest in developing face interfaces that are capable of perceiving and talking. The technical requirements are enormous. Face interfaces must learn to make eye contact, follow speaking faces with their eyes, mirror emotion, lip synch, and periodically nod, raise eyebrows, and tilt the head (Massaro). Face interfaces are also learning to write faces, to map rhetorical forms to the character of their interlocutors in ways Socrates could not have imagined. Socrates did not teach his students to consider the rhetorical effects of their faces: the speaker's face was thought to be fixed, a true reflection of the inner soul. Virtual faces are not so constrained. Smart faces are being developed that are capable of rendering their own appearances from within a statistical model of the users' impressions of faces. The goal is for these virtual faces to learn to design, through their interactions with users, facial appearances that are calibrated to elicit very specific impressions and reactions in others (Brahnam). Some people will disapprove of virtual faces. Just as the media use faces to manipulate the viewer and perpetuate facial stereotypes, smart faces run the risk of doing the same. Some may also worry that virtual faces will be attributed more intelligence and social capacity then they actually possess. Do we really want our children growing up talking to virtual faces? Can they satisfy our need for human contact? What does it mean to converse with a virtual face? What kind of conversation is that? For the present at least, virtual faces are more like the orators and bards of old. They merely repeat the speeches of others. Their own speech is nearly incomprehensible, and their grammatical hiccups annoyingly disrupt the suspension of disbelief. On their own, without the human in the loop, no one believes them. Thus, the virtual face appears on the screen, silently nodding and smiling. Not yet a proper student of classical rhetoric, it is much like the virtual guide at artificial-life.com that recently greeted her visitors wearing the following placard: A virtual guide that greeted visitors at artificial-life.com. Access date: October 2003. Works Cited Brahnam, Sheryl. "Agents as Artists: Automating Socially Intelligent Embodiment," Proceedings of the First International Workshop on the Philosophy & Design of Socially Adept Technologies, in conjunction with CHI 2002. Minneapolis, MN, 2002: 15-18. Encyclopedia Britannica. "Physiognomy." LoveToKnow Free Online Encyclopedia, 1911. Available: http://21.1911encyclopedia.org Horkheimer, Max, and Theodor W. Adorno. Dialectic of Enlightenment. Trans. John Cumming. New York: Seabury, 1944. Juengel, Scott Jordan. "About Face: Physiognomics, Revolution, and the Radical Act of Looking." Ph.D. dissertation. University of Iowa, 1997. Kress, Gunther, and Theo van Leeuwen. Reading Images: The Grammar of Visual Design. London: Routledge, 1996. Landow, P. George. Hypertext 2.0: The Convergence of Contemporary Critical Theory and Technology. Baltimore: John Hopkins U P, 1997. Lavater, Johann Caspar. Essays on Physiognomy: For the Promotion of the Knowledge and the Love of Mankind. Trans. Thomas Holcroft. London: printed by C. Whittingham for H. D. Symonds, 1804. Lichtenberg. Quoted in Frey, Siegfried. "Lavater, Lichtenberg, and the Suggestive Power of the Human Face." The Faces of Physiognomy: Interdisciplinary Approaches to Johann Caspar Lavater. Ed. Ellis Shookman. Studies in German Literature, Linguistics, and Culture. Columbia, S.C.: Camden House, 1993. 64-103. Massaro, D. M. Perceiving Talking Faces: From Speech Perception to a Behavioral Principle. Cambridge, MA: MIT P, 1997. Plato. Phaedrus and the Seventh and Eighth Letters. Trans. Walter Hamilton. London: Penguin, 1973. Tytler, Graeme. Physiognomy in the European Novel: Faces and Fortunes. Princeton, N.J.: Princeton U P, 1982. Wechsler, Judith. A Human Comedy: Physiognomy and Caricature in 19th Century Paris. London: U of Chicago P, 1982. Zebrowitz, Leslie A. Reading Faces: Window to the Soul? Boulder, Col.: Westview, 1998. Web Links http://vhost.oddcast.com/vhost_minisite/ http://mrl.nyu.edu/~perlin/facedemo/ http://www.faceinterfaces.com http://www.artificial-life.com Citation reference for this article MLA Style Brahnam, Sheryl. "Type/Face" M/C: A Journal of Media and Culture <http://www.media-culture.org.au/0401/04-brahnam.php>. APA Style Brahnam, S. (2004, Jan 12). Type/Face. M/C: A Journal of Media and Culture, 7, <http://www.media-culture.org.au/0401/04-brahnam.php>

Introduction This article explores the changing relational quality of “the shadow of hierarchy”, in the context of the merging of platforms with infrastructure as the source of the shadow of hierarchy. In governance and regulatory studies, the shadow of hierarchy (or variations thereof), describes the space of influence that hierarchal organisations and infrastructures have (Héritier and Lehmkuhl; Lance et al.). A shift in who/what casts the shadow of hierarchy will necessarily result in changes to the attendant relational values, logics, and (techno)socialities that constitute the shadow, and a new arrangement of shadow that presents new challenges and opportunities. This article reflects on relevant literature to consider two different ways the shadow of hierarchy has qualitatively changed as platforms, rather than infrastructures, come to cast the shadow of hierarchy – an increase in scalability; and new socio-technical arrangements of (non)participation – and the opportunities and challenges therein. The article concludes that more concerted efforts are needed to design the shadow, given a seemingly directionless desire to enact data-driven solutions. The Shadow of Hierarchy, Infrastructures, and Platforms The shadow of hierarchy refers to how institutional, infrastructural, and organisational hierarchies create a relational zone of influence over a particular space. This commonly refers to executive decisions and legislation created by nation states, which are cast over private and non-governmental actors (Héritier and Lehmkuhl, 2). Lance et al. (252–53) argue that the shadow of hierarchy is a productive and desirable thing. Exploring the shadow of hierarchy in the context of how geospatial data agencies govern their data, Lance et al. find that the shadow of hierarchy enables the networked governance approaches that agencies adopt. This is because operating in the shadow of institutions provides authority, confers bureaucratic legitimacy and top-down power, and offers financial support. The darkness of the shadow is thus less a moral or ethicopolitical statement (such as that suggested by Fisher and Bolter, who use the idea of darkness to unpack the morality of tourism involving death and human suffering), and instead a relationality; an expression of differing values, logics, and (techno)socialities internal and external to those infrastructures and institutions that cast it (Gehl and McKelvey). The shadow of hierarchy might therefore be thought of as a field of relational influences and power that a social body casts over society, by virtue of a privileged position vis-a-vis society. It modulates society’s “light”; the resources (Bourdieu) and power relationships (Foucault) that run through social life, as parsed through a certain institutional and infrastructural worldview (the thing that blocks the light to create the shadow). In this way the shadow of hierarchy is not a field of absolute blackness that obscures, but instead a gradient of light and dark that creates certain effects. The shadow of hierarchy is now, however, also being cast by decentralised, privately held, and non-hierarchal platforms that are replacing or merging with public infrastructure, creating new social effects. Platforms are digital, socio-technical systems that create relationships between different entities. They are most commonly built around a relatively fixed core function (such as a social media service like Facebook), that then interacts with a peripheral set of complementors (advertising companies and app developers in the case of social media; Baldwin and Woodard), to create new relationships, forms of value, and other interactions (van Dijck, The Culture of Connectivity). In creating these relationships, platforms become inherently political (Gillespie), shaping relationships and content on the platform (Suzor) and in embodied life (Ajunwa; Eubanks). While platforms are often associated with optional consumer platforms (such as streaming services like Spotify), they have increasingly come to occupy the place of public infrastructure, and act as a powerful enabler to different socio-technical, economic, and political relationships (van Dijck, Governing Digital Societies). For instance, Plantin et al. argue that platforms have merged with infrastructures, and that once publicly held and funded institutions and essential services now share many characteristics with for-profit, privately held platforms. For example, Australia has had a long history of outsourcing employment services (Webster and Harding), and nearly privatised its entire visa processing data infrastructure (Jenkins). Platforms therefore have a greater role in casting the shadow of hierarchy than before. In doing so, they cast a shadow that is qualitatively different, modulated through a different set of relational values and (techno)socialities. Scalability A key difference and selling point of platforms is their scalability; since they can rapidly and easily up- and down-scale their functionalities in a way that traditional infrastructure cannot (Plantin et al.). The ability to respond “on-demand” to infrastructural requirements has made platforms the go-to service delivery option in the neo-liberalised public infrastructure environment (van Dijck, Governing Digital Societies). For instance, services providers like Amazon Web Services or Microsoft Azure provide on demand computing capacity for many nations’ most valuable services, including their intelligence and security capabilities (Amoore, Cloud Ethics; Konkel). The value of such platforms to government lies in the reduced cost and risk that comes with using rented capabilities, and the enhanced flexibility to increase or decrease their usage as required, without any of the economic sunk costs attached to owning the infrastructure. Scalability is, however, not just about on-demand technical capability, but about how platforms can change the scale of socio-technical relationships and services that are mediated through the platform. This changes the relational quality of the shadow of hierarchy, as activities and services occurring within the shadow are now connected into a larger and rapidly modulating scale. Scalability allows the shadow of hierarchy to extend from those in proximity to institutions to the broader population in general. For example, individual citizens can more easily “reach up” into governmental services and agencies as a part of completing their everyday business through platform such as MyGov in Australia (Services Australia). Using a smartphone application, citizens are afforded a more personalised and adaptive experience of the welfare state, as engaging with welfare services is no-longer tied to specific “brick-and-mortar” locations, but constantly available through a smartphone app and web portal. Multiple government services including healthcare and taxation are also connected to this platform, allowing users to reach across multiple government service domains to complete their personal business, seeking information and services that would have once required separate communications with different branches of government. The individual’s capacities to engage with the state have therefore upscaled with this change in the shadow, retaining a productivity and capacity enhancing quality that is reminiscent of older infrastructures and institutions, as the individual and their lived context is brought closer to the institutions themselves. Scale, however, comes with complications. The fundamental driver for scalability and its adaptive qualities is datafication. This means individuals and organisations are inflecting their operational and relational logics with the logic of datafication: a need to capture all data, at all times (van Dijck, Datafication; Fourcade and Healy). Platforms, especially privately held platforms, benefit significantly from this, as they rely on data to drive and refine their algorithmic tools, and ultimately create actionable intelligence that benefits their operations. Thus, scalability allows platforms to better “reach down” into individual lives and different social domains to fuel their operations. For example, as public transport services become increasingly datafied into mobility-as-a-service (MAAS) systems, ride sharing and on-demand transportation platforms like Uber and Lyft become incorporated into the public transport ecosystem (Lyons et al.). These platforms capture geospatial, behavioural, and reputational data from users and drivers during their interactions with the platform (Rosenblat and Stark; Attoh et al.). This generates additional value, and profits, for the platform itself with limited value returned to the user or the broader public it supports, outside of the transport service. It also places the platform in a position to gain wider access to the population and their data, by virtue of operating as a part of a public service. In this way the shadow of hierarchy may exacerbate inequity. The (dis)benefits of the shadow of hierarchy become unevenly spread amongst actors within its field, a function of an increased scalability that connects individuals into much broader assemblages of datafication. For Eubank, this can entrench existing economic and social inequalities by forcing those in need to engage with digitally mediated welfare systems that rely on distant and opaque computational judgements. Local services are subject to increased digital surveillance, a removal of agency from frontline advocates, and algorithmic judgement at scale. More fortunate citizens are also still at risk, with Nardi and Ekbia arguing that many digitally scaled relationships are examples of “heteromation”, whereby platforms convince actors in the platform to labour for free, such as through providing ratings which establish a platform’s reputational economy. Such labour fuels the operation of the platform through exploiting users, who become both a product/resource (as a source of data for third party advertisers) and a performer of unrewarded digital labour, such as through providing user reviews that help guide a platform’s algorithm(s). Both these examples represent a particularly disconcerting outcome for the shadow of hierarchy, which has its roots in public sector institutions who operate for a common good through shared and publicly held infrastructure. In shifting towards platforms, especially privately held platforms, value is transmitted to private corporations and not the public or the commons, as was the case with traditional infrastructure. The public also comes to own the risks attached to platforms if they become tied to public services, placing a further burden on the public if the platform fails, while reaping none of the profit and value generated through datafication. This is a poor bargain at best. (Non)Participation Scalability forms the basis for a further predicament: a changing socio-technical dynamic of (non)participation between individuals and services. According to Star (118), infrastructures are defined through their relationships to a given context. These relationships, which often exist as boundary objects between different communities, are “loosely structured in common use, and become tightly bound in particular locations” (Star, 118). While platforms are certainly boundary objects and relationally defined, the affordances of cloud computing have enabled a decoupling from physical location, and the operation of platforms across time and space through distributed digital nodes (smartphones, computers, and other localised hardware) and powerful algorithms that sort and process requests for service. This does not mean location is not important for the cloud (see Amoore, Cloud Geographies), but platforms are less likely to have a physically co-located presence in the same way traditional infrastructures had. Without the same institutional and infrastructural footprint, the modality for participating in and with the shadow of hierarchy that platforms cast becomes qualitatively different and predicated on digital intermediaries. Replacing a physical and human footprint with algorithmically supported and decentralised computing power allows scalability and some efficiency improvements, but it also removes taken-for-granted touchpoints for contestation and recourse. For example, ride-sharing platform Uber operates globally, and has expressed interest in operating in complement to (and perhaps in competition with) public transport services in some cities (Hall et al.; Conger). Given that Uber would come to operate as a part of the shadow of hierarchy that transport authorities cast over said cities, it would not be unreasonable to expect Uber to be subject to comparable advocacy, adjudication, transparency, and complaint-handling requirements. Unfortunately, it is unclear if this would be the case, with examples suggesting that Uber would use the scalability of its platform to avoid these mechanisms. This is revealed by ongoing legal action launched by concerned Uber drivers in the United Kingdom, who have sought access to the profiling data that Uber uses to manage and monitor its drivers (Sawers). The challenge has relied on transnational law (the European Union’s General Data Protection Regulation), with UK-based drivers lodging claims in Amsterdam to initiate the challenge. Such costly and complex actions are beyond the means of many, but demonstrate how reasonable participation in socio-technical and governance relationships (like contestations) might become limited, depending on how the shadow of hierarchy changes with the incorporation of platforms. Even if legal challenges for transparency are successful, they may not produce meaningful change. For instance, O’Neil links algorithmic bias to mathematical shortcomings in the variables used to measure the world; in the creation of irritational feedback loops based on incorrect data; and in the use of unsound data analysis techniques. These three factors contribute to inequitable digital metrics like predictive policing algorithms that disproportionately target racial minorities. Large amounts of selective data on minorities create myopic algorithms that direct police to target minorities, creating more selective data that reinforces the spurious model. These biases, however, are persistently inaccessible, and even when visible are often unintelligible to experts (Ananny and Crawford). The visibility of the technical “installed base” that support institutions and public services is therefore not a panacea, especially when the installed base (un)intentionally obfuscates participation in meaningful engagement like complaints handling. A negative outcome is, however, also not an inevitable thing. It is entirely possible to design platforms to allow individual users to scale up and have opportunities for enhanced participation. For instance, eGovernance and mobile governance literature have explored how citizens engage with state services at scale (Thomas and Streib; Foth et al.), and the open government movement has demonstrated the effectiveness of open data in understanding government operations (Barns; Janssen et al.), although these both have their challenges (Chadwick; Dawes). It is not a fantasy to imagine alternative configurations of the shadow of hierarchy that allow more participatory relationships. Open data could facilitate the governance of platforms at scale (Box et al.), where users are enfranchised into a platform by some form of membership right and given access to financial and governance records, in the same way that corporate shareholders are enfranchised, facilitated by the same app that provides a service. This could also be extended to decision making through voting and polling functions. Such a governance form would require radically different legal, business, and institutional structures to create and enforce this arrangement. Delacoix and Lawrence, for instance, suggest that data trusts, where a trustee is assigned legal and fiduciary responsibility to achieve maximum benefit for a specific group’s data, can be used to negotiate legal and governance relationships that meaningfully benefit the users of the trust. Trustees can be instructed to only share data to services whose algorithms are regularly audited for bias and provide datasets that are accurate representations of their users, for instance, avoiding erroneous proxies that disrupt algorithmic models. While these developments are in their infancy, it is not unreasonable to reflect on such endeavours now, as the technologies to achieve these are already in use. Conclusions There is a persistent myth that data will yield better, faster, more complete results in whatever field it is applied (Lee and Cook; Fourcade and Healy; Mayer-Schönberger and Cukier; Kitchin). This myth has led to data-driven assemblages, including artificial intelligence, platforms, surveillance, and other data-technologies, being deployed throughout social life. The public sector is no exception to this, but the deployment of any technological solution within the traditional institutions of the shadow of hierarchy is fraught with challenges, and often results in failure or unintended consequences (Henman). The complexity of these systems combined with time, budgetary, and political pressures can create a contested environment. It is this environment that moulds societies' light and resources to cast the shadow of hierarchy. Relationality within a shadow of hierarchy that reflects the complicated and competing interests of platforms is likely to present a range of unintended social consequences that are inherently emergent because they are entering into a complex system – society – that is extremely hard to model. The relational qualities of the shadow of hierarchy are therefore now more multidimensional and emergent, and experiences relating to socio-technical features like scale, and as a follow-on (non)participation, are evidence of this. Yet by being emergent, they are also directionless, a product of complex systems rather than designed and strategic intent. This is not an inherently bad thing, but given the potential for data-system and platforms to have negative or unintended consequences, it is worth considering whether remaining directionless is the best outcome. There are many examples of data-driven systems in healthcare (Obermeyer et al.), welfare (Eubanks; Henman and Marston), and economics (MacKenzie), having unintended and negative social consequences. Appropriately guiding the design and deployment of theses system also represents a growing body of knowledge and practical endeavour (Jirotka et al.; Stilgoe et al.). Armed with the knowledge of these social implications, constructing an appropriate social architecture (Box and Lemon; Box et al.) around the platforms and data systems that form the shadow of hierarchy should be encouraged. This social architecture should account for the affordances and emergent potentials of a complex social, institutional, economic, political, and technical environment, and should assist in guiding the shadow of hierarchy away from egregious challenges and towards meaningful opportunities. To be directionless is an opportunity to take a new direction. The intersection of platforms with public institutions and infrastructures has moulded society’s light into an evolving and emergent shadow of hierarchy over many domains. With the scale of the shadow changing, and shaping participation, who benefits and who loses out in the shadow of hierarchy is also changing. Equipped with insights into this change, we should not hesitate to shape this change, creating or preserving relationalities that offer the best outcomes. Defining, understanding, and practically implementing what the “best” outcome(s) are would be a valuable next step in this endeavour, and should prompt considerable discussion. If we wish the shadow of hierarchy to continue to be productive, then finding a social architecture to shape the emergence and directionlessness of socio-technical systems like platforms is an important step in the continued evolution of the shadow of hierarchy. References Ajunwa, Ifeoma. “Age Discrimination by Platforms.” Berkeley J. Emp. & Lab. L. 40 (2019): 1-30. Amoore, Louise. Cloud Ethics: Algorithms and the Attributes of Ourselves and Others. Durham: Duke University Press, 2020. ———. “Cloud Geographies: Computing, Data, Sovereignty.” Progress in Human Geography 42.1 (2018): 4-24. Ananny, Mike, and Kate Crawford. “Seeing without Knowing: Limitations of the Transparency Ideal and Its Application to Algorithmic Accountability.” New Media & Society 20.3 (2018): 973–89. Attoh, Kafui, et al. “‘We’re Building Their Data’: Labor, Alienation, and Idiocy in the Smart City.” Environment and Planning D: Society and Space 37.6 (2019): 1007-24. Baldwin, Carliss Y., and C. Jason Woodard. “The Architecture of Platforms: A Unified View.” Platforms, Markets and Innovation. Ed. Annabelle Gawer. Cheltenham: Edward Elgar, 2009. 19–44. Barns, Sarah. “Mine Your Data: Open Data, Digital Strategies and Entrepreneurial Governance by Code.” Urban Geography 37.4 (2016): 554–71. Bourdieu, Pierre. Distinction: A Social Critique of the Judgement of Taste. Cambridge, MA: Harvard University Press, 1984. Box, Paul, et al. Data Platforms for Smart Cities – A Landscape Scan and Recommendations for Smart City Practice. Canberra: CSIRO, 2020. Box, Paul, and David Lemon. The Role of Social Architecture in Information Infrastructure: A Report for the National Environmental Information Infrastructure (NEII). Canberra: CSIRO, 2015. Chadwick, Andrew. “Explaining the Failure of an Online Citizen Engagement Initiative: The Role of Internal Institutional Variables.” Journal of Information Technology & Politics 8.1 (2011): 21–40. Conger, Kate. “Uber Wants to Sell You Train Tickets. And Be Your Bus Service, Too.” The New York Times, 7 Aug. 2019. 19 Jan. 2021. <https://www.nytimes.com/2019/08/07/technology/uber-train-bus-public-transit.html>. Dawes, Sharon S. “The Evolution and Continuing Challenges of E‐Governance.” Public Administration Review 68 (2008): 86–102. Delacroix, Sylvie, and Neil D. Lawrence. “Bottom-Up Data Trusts: Disturbing the ‘One Size Fits All’ Approach to Data Governance.” International Data Privacy Law 9.4 (2019): 236-252. Eubanks, Virginia. Automating Inequality: How High-Tech Tools Profile, Police, and Punish the Poor. New York: St. Martin’s Press, 2018. Fisher, Joshua A., and Jay David Bolter. “Ethical Considerations for AR Experiences at Dark Tourism Sites”. IEEE Explore 29 April. 2019. 13 Apr. 2021 <https://ieeexplore.ieee.org/document/8699186>. Foth, Marcus, et al. From Social Butterfly to Engaged Citizen: Urban Informatics, Social Media, Ubiquitous Computing, and Mobile Technology to Support Citizen Engagement. Cambridge MA: MIT Press, 2011. Fourcade, Marion, and Kieran Healy. “Seeing like a Market.” Socio-Economic Review, 15.1 (2017): 9–29. Gehl, Robert, and Fenwick McKelvey. “Bugging Out: Darknets as Parasites of Large-Scale Media Objects.” Media, Culture & Society 41.2 (2019): 219–35. Gillespie, Tarleton. “The Politics of ‘Platforms.’” New Media & Society 12.3 (2010): 347–64. Hall, Jonathan D., et al. “Is Uber a Substitute or Complement for Public Transit?” Journal of Urban Economics 108 (2018): 36–50. Henman, Paul. “Improving Public Services Using Artificial Intelligence: Possibilities, Pitfalls, Governance.” Asia Pacific Journal of Public Administration 42.4 (2020): 209–21. Henman, Paul, and Greg Marston. “The Social Division of Welfare Surveillance.” Journal of Social Policy 37.2 (2008): 187–205. Héritier, Adrienne, and Dirk Lehmkuhl. “Introduction: The Shadow of Hierarchy and New Modes of Governance.” Journal of Public Policy 28.1 (2008): 1–17. Janssen, Marijn, et al. “Benefits, Adoption Barriers and Myths of Open Data and Open Government.” Information Systems Management 29.4 (2012): 258–68. Jenkins, Shannon. “Visa Privatisation Plan Scrapped, with New Approach to Tackle ’Emerging Global Threats’.” The Mandarin. 23 Mar. 2020. 19 Jan. 2021 <https://www.themandarin.com.au/128244-visa-privatisation-plan-scrapped-with-new-approach-to-tackle-emerging-global-threats/>. Jirotka, Marina, et al. “Responsible Research and Innovation in the Digital Age.” Communications of the ACM 60.6 (2016): 62–68. Kitchin, Rob. The Data Revolution: Big Data, Open Data, Data Infrastructures and Their Consequences. Thousand Oaks, CA: Sage, 2014. Konkel, Frank. “CIA Awards Secret Multibillion-Dollar Cloud Contract.” Nextgov 20 Nov. 2020. 19 Jan. 2021 <https://www.nextgov.com/it-modernization/2020/11/exclusive-cia-awards-secret-multibillion-dollar-cloud-contract/170227/>. Lance, Kate T., et al. “Cross‐Agency Coordination in the Shadow of Hierarchy: ‘Joining up’Government Geospatial Information Systems.” International Journal of Geographical Information Science, 23.2 (2009): 249–69. Lee, Ashlin J., and Peta S. Cook. “The Myth of the ‘Data‐Driven’ Society: Exploring the Interactions of Data Interfaces, Circulations, and Abstractions.” Sociology Compass 14.1 (2020): 1–14. Lyons, Glenn, et al. “The Importance of User Perspective in the Evolution of MaaS.” Transportation Research Part A: Policy and Practice 121(2019): 22-36. MacKenzie, Donald. “‘Making’, ‘Taking’ and the Material Political Economy of Algorithmic Trading.” Economy and Society 47.4 (2018): 501-23. Mayer-Schönberger, V., and K. Cukier. Big Data: A Revolution That Will Change How We Live, Work and Think. London: John Murray, 2013. Michel Foucault. Discipline and Punish. London: Penguin, 1977. Nardi, Bonnie, and Hamid Ekbia. Heteromation, and Other Stories of Computing and Capitalism. Cambridge, MA: MIT Press, 2017. O’Neil, Cathy. Weapons of Math Destruction – How Big Data Increases Inequality and Threatens Democracy. London: Penguin, 2017. Obermeyer, Ziad, et al. “Dissecting Racial Bias in an Algorithm Used to Manage the Health of Populations.” Science 366.6464 (2019): 447-53. Plantin, Jean-Christophe, et al. “Infrastructure Studies Meet Platform Studies in the Age of Google and Facebook.” New Media & Society 20.1 (2018): 293–310. Rosenblat, Alex, and Luke Stark. “Algorithmic Labor and Information Asymmetries: A Case Study of Uber’s Drivers.” International Journal of Communication 10 (2016): 3758–3784. Sawers, Paul. “Uber Drivers Sue for Data on Secret Profiling and Automated Decision-Making.” VentureBeat 20 July. 2020. 19 Jan. 2021 <https://venturebeat.com/2020/07/20/uber-drivers-sue-for-data-on-secret-profiling-and-automated-decision-making/>. Services Australia. About MyGov. Services Australia 19 Jan. 2021. 19 Jan. 2021 <https://www.servicesaustralia.gov.au/individuals/subjects/about-mygov>. Star, Susan Leigh. “Infrastructure and Ethnographic Practice: Working on the Fringes.” Scandinavian Journal of Information Systems 14.2 (2002):107-122. Stilgoe, Jack, et al. “Developing a Framework for Responsible Innovation.” Research Policy 42.9 (2013):1568-80. Suzor, Nicolas. Lawless: The Secret Rules That Govern Our Digital Lives. Cambridge: Cambridge University Press, 2019. Thomas, John Clayton, and Gregory Streib. “The New Face of Government: Citizen‐initiated Contacts in the Era of E‐Government.” Journal of Public Administration Research and Theory 13.1 (2003): 83-102. Van Dijck, José. “Datafication, Dataism and Dataveillance: Big Data between Scientific Paradigm and Ideology.” Surveillance & Society 12.2 (2014): 197–208. ———. “Governing Digital Societies: Private Platforms, Public Values.” Computer Law & Security Review 36 (2020) 13 Apr. 2021 <https://www.sciencedirect.com/science/article/abs/pii/S0267364919303887>. ———. The Culture of Connectivity: A Critical History of Social Media. Oxford: Oxford University Press, 2013. Webster, Elizabeth, and Glenys Harding. “Outsourcing Public Employment Services: The Australian Experience.” Australian Economic Review 34.2 (2001): 231-42.

Emerging technologies, such as; Artificial Intelligence, extended reality technologies (MR, AR and VR), and sensor technologies are increasingly used during knowledge production within urban design processes. This thesis shows that the combination of different paradigms underlying experimental initiatives that bring together design and technology can lead to either vicious loops that continue to reinforce the modernist city, or virtuous loops where the interweaving of fundamentally different paradigms of knowledge production with emerging technology possess potential for radical urban transformation.  The key findings of this work include: Sensorial ways of knowing the city can be in synergy with these emerging technologies 1) augmenting our human senses and overcoming spatial-temporal constraints in experiencing ecological change, 2) they can create fully embodied experiences of sidelined or marginalized narratives in urban planning and the future of cities, 3) they open up new space for more collaboration in research design and planning. In the same way, democratization and co-production of knowledge for urban design, can be in synergy with these emerging technologies that; 1) allow for the expansion of collaboration and the integration of multiple sources of knowledge, 2) that promote an open relation with the future by rapid prototyping of alternative scenarios, and that 3) democratize the power over the process of data collection and data analysis all the way to ‘open-sourcing’ the tools themselves.

Cyber security is a constantly evolving area of interest. Many solutions are currently available and new methods and technologies are emerging. Although some solutions already exist in extended reality, a lack of engagement and storytelling is available, with a consequence of decreasing the probability of dissemination and awareness of the risks involved in cybersecurity. This chapter gives an overview of an extended reality platform that can be potentially used for the simulation of security threats and that combines artificial intelligence and game design principles. The main goal of this research is to develop an extended reality solution to simulate a story involving virtual characters and objects for the entertainment industry, with possible applications in other sectors such as education and training. After an introduction to extended reality, the chapter focuses on an overview on the available extended reality technologies in the context of cybersecurity.