Academic literature on the topic 'Image restoration, adaptive image processing, neural networks, human visual system'

The Pyramidal Neural Networks (PNN) are an example of a successful recently proposed model inspired by the human visual system and deep learning theory. PNNs are applied to computer vision and based on the concept of receptive fields. This paper proposes a variation of PNN, named here as Structured Pyramidal Neural Network (SPNN). SPNN has self-adaptive variable receptive fields, while the original PNNs rely on the same size for the fields of all neurons, which limits the model since it is not possible to put more computing resources in a particular region of the image. Another limitation of the original approach is the need to define values for a reasonable number of parameters, which can turn difficult the application of PNNs in contexts in which the user does not have experience. On the other hand, SPNN has a fewer number of parameters. Its structure is determined using a novel method with Delaunay Triangulation and k-means clustering. SPNN achieved better results than PNNs and similar performance when compared to Convolutional Neural Network (CNN) and Support Vector Machine (SVM), but using lower memory capacity and processing time.

Visual neuroprosthesis, that provide electrical stimulation along several sites of the human visual system, constitute a potential tool for vision restoration for the blind. Scientific and technological progress in the fields of neural engineering and artificial vision comes with new theories and tools that, along with the dawn of modern artificial intelligence, constitute a promising framework for the further development of neurotechnology. In the framework of the development of a Cortical Visual Neuroprosthesis for the blind (CORTIVIS), we are now facing the challenge of developing not only computationally powerful tools and flexible approaches that will allow us to provide some degree of functional vision to individuals who are profoundly blind. In this work, we propose a general neuroprosthesis framework composed of several task-oriented and visual encoding modules. We address the development and implementation of computational models of the firing rates of retinal ganglion cells and design a tool — Neurolight — that allows these models to be interfaced with intracortical microelectrodes in order to create electrical stimulation patterns that can evoke useful perceptions. In addition, the developed framework allows the deployment of a diverse array of state-of-the-art deep-learning techniques for task-oriented and general image pre-processing, such as semantic segmentation and object detection in our system’s pipeline. To the best of our knowledge, this constitutes the first deep-learning-based system designed to directly interface with the visual brain through an intracortical microelectrode array. We implement the complete pipeline, from obtaining a video stream to developing and deploying task-oriented deep-learning models and predictive models of retinal ganglion cells’ encoding of visual inputs under the control of a neurostimulation device able to send electrical train pulses to a microelectrode array implanted at the visual cortex.

Abstract The human eye has three rotational degrees of freedom: azimuthal, elevational, and torsional. Although torsional eye movements have the most limited excursion, Hering and Helmholtz have argued that they play an important role in optimizing visual information processing. In humans, the relationship between gaze direction and torsional eye angle is described by Listing's law. However, it is still not clear how this behavior initially develops and remains calibrated during growth. Here we present the first computational model that enables an autonomous agent to learn and maintain binocular torsional eye movement control. In our model, two neural networks connected in series: one for sensory encoding followed by one for torsion control, are learned simultaneously as the agent behaves in the environment. Learning is based on the Active Efficient Coding (AEC) framework, a generalization of Barlow's efficient coding hypothesis to include action. Both networks adapt by minimizing the prediction error of the sensory representation, subject to a sparsity constraint on neural activity. The policies that emerge follow the predictions of Listing’s law. Because learning is driven by the sensorimotor contingencies experienced by the agent as it interacts with the environment, our system can adapt to the physical configuration of the agent as it changes. We propose that AEC provides the most parsimonious expression to date of Hering’s and Helmholtz’s hypotheses. We also demonstrate that it has practical implications in autonomous artificial vision systems, by providing an automatic and adaptive mechanism to correct orientation misalignments between cameras in a robotic active binocular vision head. Our system's use of fairly low resolution ($~100 \times 100$ pixel) image windows and perceptual representations amenable to event-based input paves a pathway towards the implementation of adaptive self-calibrating robot control on neuromorphic hardware.

Whichever way you look at it, self is bound up with consciousness, so it seems useful to review some of the more significant existing conceptions of this relationship. A claim by Mikhail Bakhtin can serve as an anchoring point for this discussion. He firmly predicates the formation of self not just on the existence of an individual consciousness, but on what might be called a double or social (or dialogic) consciousness. Summarising his argument, Pam Morris writes: 'A single consciousness could not generate a sense of its self; only the awareness of another consciousness outside the self can produce that image.' She goes on to say that, 'Behind this notion is Bakhtin's very strong sense of the physical and spatial materiality of bodily being,' and quotes directly from Bakhtin's essay as follows: This other human being whom I am contemplating, I shall always see and know something that he, from his place outside and over against me, cannot see himself: parts of his body that are inaccessible to his own gaze (his head, his face and its expression), the world behind his back . . . are accessible to me but not to him. As we gaze at each other, two different worlds are reflected in the pupils of our eyes . . . to annihilate this difference completely, it would be necessary to merge into one, to become one and the same person. This ever--present excess of my seeing, knowing and possessing in relation to any other human being, is founded in the uniqueness and irreplaceability of my place in the world. (Bakhtin in Morris 6 Recent investigations in neuroscience and the philosophy of mind lay down a challenge to this social conception of the self. Notably, it is a challenge that does not involve the restoration of any variant of Cartesian rationalism; indeed, it arguably over--privileges rationalism's subjective or phenomenological opposite. 'Self' in this emerging view is a biologically generated but illusory construction, an effect of the operation of what are called 'neural correlates of consciousness' (NCC). Very briefly, an NCC refers to the distinct pattern of neurochemical activity, a 'neural representational system' -- to some extent observable by modern brain--imaging equipment – that corresponds to a particular configuration of sense--phenomena, or 'content of consciousness' (a visual image, a feeling, or indeed a sense of self). Because this science is still largely hypothetical, with many alternative terms and descriptions, it would be better in this limited space to focus on one particular account – one that is particularly well developed in the area of selfhood and one that resonates with other conceptions included in this discussion. Thomas Metzinger begins by postulating the existence within each person (or 'system' in his terms) of a 'self--model', a representation produced by neural activity -- what he calls a 'neural correlate of self--consciousness' -- that the individual takes to be the actual self, or what Metzinger calls the 'phenomenal self'. 'A self--model is important,' Metzinger says, 'in enabling a system to represent itself to itself as an agent' (293). The individual is able to maintain this illusion because 'the self--model is the only representational structure that is anchored in the brain by a continuous source of internally generated input' (297). In a manner partly reminiscent of Bakhtin, he continues: 'The body is always there, and although its relational properties in space and in movement constantly change, the body is the only coherent perceptual object that constantly generates input.' The reason why the individual is able to jump from the self--model to the phenomenal self in the first place is because: We are systems that are not able to recognise their subsymbolic self--model as a model. For this reason, we are permanently operating under the conditions of a 'naïve--realistic self--misunderstanding': We experience ourselves as being in direct and immediate epistemic contact with ourselves. What we have in the past simply called a 'self' is not a non--physical individual, but only the content of an ongoing dynamical process – the process of transparent self—modeling. (Metzinger 299) The question that nonetheless arises is why it should be concluded that this self--model emerges from subjective neural activity and not, say, from socialisation. Why should a self--model be needed in the first place? Metzinger's response is to say that there is good evidence 'for some kind of innate 'body prototype'' (298), and he refers to research that shows that even children born without limbs develop self--models which sometimes include limbs, or report phantom sensations in limbs that have never existed. To me, this still leaves open the possibility that such children are modelling their body image on strong identification with human others. But be that as it may, one of the things that remains unclear after this relatively rich account of contemporary or scientific phenomenology is the extent to which 'neural consciousness' is or can be supplemented by other kinds of consciousness, or indeed whether neural consciousness can be overridden by the 'self' acting on the basis of these other kinds of consciousness. The key stake in Metzinger's account is 'subjectivity'. The reason why the neural correlate of self--consciousness is so important to him is: 'Only if we find the neural and functional correlates of the phenomenal self will we be able to discover a more general theoretical framework into which all data can fit. Only then will we have a chance to understand what we are actually talking about when we say that phenomenal experience is a subjective phenomenon' (301). What other kinds of consciousness might there be? It is significant that, not only do NCC exponents have little to say about the interaction with other people, they rarely mention language, and they are unanimously and emphatically of the opinion that the thinking or processing that takes place in consciousness is not dependent on language, or indeed any signifying system that we know of (though conceivably, it occurs to me, the neural correlates may signify to, or 'call up', each other). And they show little 'consciousness' that a still influential body of opinion (informed latterly by post--structuralist thinking) has argued for the consciousness shaping effects of 'discourse' -- i.e. for socially and culturally generated patterns of language or other signification to order the processing of reality. We could usefully coin the term 'verbal correlates of consciousness' (VCC) to refer to these patterns of signification (words, proverbs, narratives, discourses). Again, however, the same sorts of questions apply, since few discourse theorists mention anything like neuroscience: To what extent is verbal consciousness supplemented by other forms of consciousness, including neural consciousness? These questions may never be fully answerable. However, it is interesting to work through the idea that NCC and VCC both exist and can be in some kind of relation even if the precise relationship is not measurable. This indeed is close to the case that Charles Shepherdson makes for psychoanalysis in attempting to retrieve it from the misunderstanding under which it suffers today: We are now familiar with debates between those who seek to demonstrate the biological foundations of consciousness and sexuality, and those who argue for the cultural construction of subjectivity, insisting that human life has no automatically natural form, but is always decisively shaped by contingent historical conditions. No theoretical alternative is more widely publicised than this, or more heavily invested today. And yet, this very debate, in which 'nature' and 'culture' are opposed to one another, amounts to a distortion of psychoanalysis, an interpretive framework that not only obscures its basic concepts, but erodes the very field of psychoanalysis as a theoretically distinct formation (2--3). There is not room here for an adequate account of Shepherdson's recuperation of psychoanalytic categories. A glimpse of the stakes involved is provided by Shepherdson's account, following Eugenie Lemoine--Luccione, of anorexia, which neither biomedical knowledge nor social constructionism can adequately explain. The further fact that anorexia is more common among women of the same family than in the general population, and among women rather than men, but in neither case exclusively so, thereby tending to rule out a genetic factor, allows Shepherdson to argue: [A]norexia can be understood in terms of the mother--daughter relation: it is thus a symbolic inheritance, a particular relation to the 'symbolic order', that is transmitted from one generation to another . . . we may add that this relation to the 'symbolic order' [which in psychoanalytic theory is not coextensive with language] is bound up with the symbolisation of sexual difference. One begins to see from this that the term 'sexual difference' is not used biologically, but also that it does not refer to general social representations of 'gender,' since it concerns a more particular formation of the 'subject' (12). An intriguing, and related, possibility, suggested by Foucault, is that NCC and VCC (or in Foucault's terms the 'visible' and the 'articulable'), operate independently of each other – that there is a 'disjunction' (Deleuze 64) or 'dislocation' (Shepherdson 166) between them that prevents any dialectical relation. Clearly, for Foucault, the lack of dialectical relation between the two modes does not mean that both are not at all times equally functional. But one can certainly speculate that, increasingly under postmodernity and media saturation, the verbal (i.e. the domain of signification in general) is influential. And if linguistic formations -- discourses, narratives, etc. -- can proliferate and feed on each other unconstrained by other aspects of reality, we get the sense of language 'running away with itself' and, at least for a time, becoming divorced from a more complete sense of reality. (This of course is basically the argument of Baudrillard.) The reverse may also be possible, in certain periods, although the idea that language could have no mediating effect at all on the production of reality (just inconsequential fluff on the surface of things) seems far--fetched in the wake of so much postmodern and media theory. However, the notion is consistent with the theories of hard--line materialists and genetic determinists. But we should at least consider the possibility that some sort of shaping interaction between NCC and VCC, without implicating the full conceptual apparatus of psychoanalysis, is continuously occurring. This possibility is, for me, best realised by Jacques Derrida when he writes of an irreducible interweaving of consciousness and language (the latter for Derrida being a cover term for any system of signification). This interweaving is such that the significatory superstructure 'reacts upon' the 'substratum of non--expressive acts and contents', and the name for this interweaving is 'text' (Mowitt 98). A further possibility is that provided by Pierre Bourdieu's notion of habitus -- the socially inherited schemes of perception and selection, imparted by language and example, which operate for the most part below the level of consciousness but are available to conscious reflection by any individual lucky enough to learn how to recognise that possibility. If the subjective representations of NCC exist, this habitus can be at best only partial; something denied by Bourdieu whose theory of individual agency is founded in what he has referred to as 'the relation between two states of the social' – i.e. 'between history objectified in things, in the form of institutions, and history incarnate in the body, in the form of that system of durable dispositions I call habitus' (190). At the same time, much of Bourdieu's thinking about the habitus seems as though it could be consistent with the kind of predictable representations that might be produced by NCC. For example, there are the simple oppositions that structure much perception in Bourdieu's account. These range from the obvious phenomenological ones (dark/light; bright/dull; male/female; hard/soft, etc.) through to the more abstract, often analogical or metaphorical ones, such as those produced by teachers when assessing their students (bright/dull again; elegant/clumsy, etc.). It seems possible that NCC could provide the mechanism or realisation for the representation, storage, and reactivation of impressions constituting a social model--self. However, an entirely different possibility remains to be considered – which perhaps Bourdieu is also getting at – involving a radical rejection of both NCC and VCC. Any correlational or representational theory of the relationship between a self and his/her environment -- which, according to Charles Altieri, includes the anti--logocentrism of Derrida -- assumes that the primary focus for any consciousness is the mapping and remapping of this relationship rather than the actions and purposes of the agent in question. Referring to the later philosophy of Wittgenstein, Altieri argues: 'Conciousness is essentially not a way of relating to objects but of relating to actions we learn to perform . . . We do not usually think about objects, but about the specific form of activity which involves us with these objects at this time' (233). Clearly, there is not yet any certainty in the arguments provided by neuroscience that neural activity performs a representational role. Is it not, then, possible that this activity, rather than being a 'correlate' of entities, is an accompaniment to, a registration of, action that the rest of the body is performing? In this view, self is an enactment, an expression (including but not restricted to language), and what self--consciousness is conscious of is this activity of the self, not the self as entity. In a way that again returns us towards Bakhtin, Altieri writes: '>From an analytical perspective, it seems likely that our normal ways of acting in the world provide all the criteria we need for a sense of identity. As Sidney Shoemaker has shown, the most important source of the sense of our identity is the way we use the spatio--temporal location of our body to make physical distinctions between here and there, in front and behind, and so on' (234). Reasonably consistent with the Wittgensteinian view -- in its focus on self--activity -- is that contemporary theorisation of the self that compares in influence with that posed by neuroscience. This is the self avowedly constructed by networked computer technology, as described by Mark Poster: [W]hat has occurred in the advanced industrial societies with increasing rapidity . . . is the dissemination of technologies of symbolisation, or language machines, a process that may be described as the electronic textualisation of daily life, and the concomitant transformations of agency, transformations of the constitution of individuals as fixed identities (autonomous, self--regulating, independent) into subjects that are multiple, diffuse, fragmentary. The old (modern) agent worked with machines on natural materials to form commodities, lived near other workers and kin in urban communities, walked to work or traveled by public transport, and read newspapers but engaged as a communicator mostly in face--to--face relations. The new (postmodern) agent works mostly on symbols using computers, lives in isolation from other workers and kin, travels to work by car, and receives news and entertainment from television. . . . Individuals who have this experience do not stand outside the world of objects, observing, exercising rational faculties and maintaining a stable character. The individuals constituted by the new modes of information are immersed and dispersed in textualised practices where grounds are less important than moves. (44--45) Interestingly, Metzinger's theorisation of the model--self lends itself to the self--mutability -- though not the diffusion -- favoured by postmodernists like Poster. [I]t is . . . well conceivable that a system generates a number of different self--models which are functionally incompatible, and therefore modularised. They nevertheless could be internally coherent, each endowed with its own characteristic phenomenal content and behavioral profile. . . this does not have to be a pathological situation. Operating under different self--models in different situational contexts may be biologically as well as socially adaptive. Don't we all to some extent use multiple personalities to cope efficiently with different parts of our lives? (295--6) Poster's proposition is consistent with that of many in the humanities and social sciences today, influenced variously by postmodernism and social constructionism. What I believe remains at issue about his account is that it exchanges one form of externally constituted self ('fixed identity') for another (that produced by the 'modes of information'), and therefore remains locked in a logic of deterministic constitution. (There is a parallel here with Altieri's point about Derrida's inability to escape representation.) Furthermore, theorists like Poster may be too quickly generalising from the experience of adults in 'textualised environments'. Until such time as human beings are born directly into virtual reality environments, each will, for a formative period of time, experience the world in the way described by Bakhtin – through 'a unified perception of bodily and personal being . . . characterised . . . as a loving gift mutually exchanged between self and other across the borderzone of their two consciousnesses' (cited in Morris 6). I suggest it is very unlikely that this emergent sense of being can ever be completely erased even when people subsequently encounter each other in electronic networked environments. It is clearly not the role of a brief survey like this to attempt any resolution of these matters. Indeed, my review has made all the more apparent how far from being settled the question of consciousness, and by extension the question of selfhood, remains. Even the classical notion of the homunculus (the 'little inner man' or the 'ghost in the machine') has been put back into play with Francis Crick and Christof Koch's (2000) neurobiological conception of the 'unconscious homunculus'. The balance of contemporary evidence and argument suggests that the best thing to do right now is to keep the questions open against any form of reductionism – whether social or biological. One way to do this is to explore the notions of self and consciousness as implicated in ongoing processes of complex co--adaptation between biology and culture -- or their individual level equivalents, brain and mind (Taylor Ch. 7). References Altieri, C. "Wittgenstein on Consciousness and Language: a Challenge to Derridean Literary Theory." Wittgenstein, Theory and the Arts. Ed. Richard Allen and Malcolm Turvey. New York: Routledge, 2001. Bourdieu, P. In Other Words: Essays Towards a Reflexive Sociology. Trans. Matthew Adamson. Stanford: Stanford University Press, 1990. Crick, F. and Koch, C. "The Unconscious Homunculus." Neural Correlates of Consciousness: Empirical and Conceptual Questions. Ed. Thomas Metzinger. Cambridge, Mass.: MIT Press, 2000. Deleuze, G. Foucault. Trans. Sean Hand. Minneapolis: University of Minnesota Press, 1988. Metzinger, T. "The Subjectivity of Subjective Experience: A Representationalist Analysis of the First-Person Perspective." Neural Correlates of Consciousness: Empirical and Conceptual Questions. Ed. Thomas Metzinger. Cambridge, Mass.: MIT Press, 2000. Morris, P. (ed.). The Bakhtin Reader: Selected Writings of Bakhtin, Medvedev, Voloshinov. London: Edward Arnold, 1994. Mowitt, J. Text: The Genealogy of an Interdisciplinary Object. Durham: Duke University Press, 1992. Poster, M. Cultural History and Modernity: Disciplinary Readings and Challenges. New York: Columbia University Press, 1997. Shepherdson, C. Vital Signs: Nature, Culture, Psychoanalysis. New York: Routledge, 2000. Taylor, M. C. The Moment of Complexity: Emerging Network Culture. Chicago: University of Chicago Press, 2001. Citation reference for this article Substitute your date of access for Dn Month Year etc... MLA Style Wallace, Derek. "'Self' and the Problem of Consciousness" M/C: A Journal of Media and Culture 5.5 (2002). [your date of access] < http://www.media-culture.org.au/mc/0210/Wallace.html &gt. Chicago Style Wallace, Derek, "'Self' and the Problem of Consciousness" M/C: A Journal of Media and Culture 5, no. 5 (2002), < http://www.media-culture.org.au/mc/0210/Wallace.html &gt ([your date of access]). APA Style Wallace, Derek. (2002) 'Self' and the Problem of Consciousness. M/C: A Journal of Media and Culture 5(5). < http://www.media-culture.org.au/mc/0210/Wallace.html &gt ([your date of access]).

Abstract This thesis describes research into the field of image restoration. Restoration is a process by which an image suffering some form of distortion or degradation can be recovered to its original form. Two primary concepts within this field have been investigated. The first concept is the use of a Hopfield neural network to implement the constrained least square error method of image restoration. In this thesis, the author reviews previous neural network restoration algorithms in the literature and builds on these algorithms to develop a new faster version of the Hopfield neural network algorithm for image restoration. The versatility of the neural network approach is then extended by the author to deal with the cases of spatially variant distortion and adaptive regularisation. It is found that using the Hopfield-based neural network approach, an image suffering spatially variant degradation can be accurately restored without a substantial penalty in restoration time. In addition, the adaptive regularisation restoration technique presented in this thesis is shown to produce superior results when compared to non-adaptive techniques and is particularly effective when applied to the difficult, yet important, problem of semi-blind deconvolution. The second concept investigated in this thesis, is the difficult problem of incorporating concepts involved in human visual perception into image restoration techniques. In this thesis, the author develops a novel image error measure which compares two images based on the differences between local regional statistics rather than pixel level differences. This measure more closely corresponds to the way humans perceive the differences between two images. Two restoration algorithms are developed by the author based on versions of the novel image error measure. It is shown that the algorithms which utilise this error measure have improved performance and produce visually more pleasing images in the cases of colour and grayscale images under high noise conditions. Most importantly, the perception based algorithms are shown to be extremely tolerant of faults in the restoration algorithm and hence are very robust. A number of experiments have been performed to demonstrate the performance of the various algorithms presented.