ORIGINAL RESEARCHARTICLE published: 17 June 2014 doi: 10.3389/fpsyg.2014.00577 Objects of consciousness DonaldD.Hoffman1*andChetanPrakash2 1 Department of Cognitive Sciences, University of California, Irvine, CA, USA 2 Department of Mathematics, California State University, San Bernardino, CA, USA Edited by: Current models of visual perception typically assume that human vision estimates true Chris Fields, New Mexico State properties of physical objects, properties that exist even if unperceived. However, recent University, USA (retired) studies of perceptual evolution, using evolutionary games and genetic algorithms, reveal Reviewedby: that natural selection often drives true perceptions to extinction when they compete John Serences, University of with perceptions tuned to fitness rather than truth: Perception guides adaptive behavior; California San Diego, USA David Marcus Appleby, University of it does not estimate a preexisting physical truth. Moreover, shifting from evolutionary Sydney, Australia biology to quantum physics, there is reason to disbelieve in preexisting physical truths: *Correspondence: Certain interpretations of quantum theory deny that dynamical properties of physical Donald D. Hoffman, Department of objects have definite values when unobserved. In some of these interpretations the Cognitive Sciences, University of observer is fundamental, and wave functions are compendia of subjective probabilities, California, Irvine, CA 92697, USA e-mail: ddhoff@uci.edu not preexisting elements of physical reality. These two considerations, from evolutionary biology and quantum physics, suggest that current models of object perception require fundamental reformulation. Here we begin such a reformulation, starting with a formal model of consciousness that we call a “conscious agent.” We develop the dynamics of interacting conscious agents, and study how the perception of objects and space-time can emerge from such dynamics. We show that one particular object, the quantum free particle, has a wave function that is identical in form to the harmonic functions that characterize the asymptotic dynamics of conscious agents; particles are vibrations not of stringsbutofinteractingconsciousagents.Thisallowsustoreinterpretphysicalproperties such as position, momentum, and energy as properties of interacting conscious agents, rather than as preexisting physical truths. We sketch how this approach might extend to the perception of relativistic quantum objects, and to classical objects of macroscopic scale. Keywords: consciousness, quantum theory, Markov chains, combination problem, geometric algebra INTRODUCTION brain exists when unperceived (Edelman, 2004). When Francis The human mind is predisposed to believe that physical objects, Crick asserted the “astonishing hypothesis” that “You’re noth- when unperceived, still exist with definite shapes and locations ing but a pack of neurons” he assumed that neurons exist when in space. The psychologist Piaget proposed that children start to unperceived (Crick, 1994). develop this belief in “object permanence” around 9 months of Object permanence underlies the standard account of evo- age, and have it firmly entrenched just 9 months later (Piaget, lution by natural selection. As James memorably put it, “The 1954). Further studies suggest that object permanence starts as point which as evolutionists we are bound to hold fast to is early as 3 months of age (Bower, 1974; Baillargeon and DeVos, that all the new forms of being that make their appearance are 1991). reallynothingmorethanresultsoftheredistributionofthe Belief in object permanence remains firmly entrenched into original and unchanging materials. The self-same atoms which, adulthood, even in the brightest of minds. Abraham Pais said of chaotically dispersed, made the nebula, now, jammed and tem- Einstein, “We often discussed his notions on objective reality. I porarily caught in peculiar positions, form our brains” (James, recall that on one walk Einstein suddenly stopped, turned to me 1890).Evolutionarytheory,inthestandardaccount,assumesthat and asked whether I really believed that the moon exists only atoms, and the replicating molecules that they form, exist when when I look at it” (Pais, 1979). Einstein was troubled by inter- unperceived. pretations of quantum theory that entail that the moon does not Object permanence underlies computational models of the exist when unperceived. visual perception of objects. David Marr, for instance, claimed Belief in object permanence underlies physicalist theories of “We ... very definitely do compute explicit properties of the the mind-body problem. When Gerald Edelman claimed, for real visible surfaces out there, and one interesting aspect of the instance, that “There is now a vast amount of empirical evi- evolution of visual systems is the gradual movement toward the dence to support the idea that consciousness emerges from the difficult task of representing progressively more objective aspects organization and operation of the brain” he assumed that the of the visual world” (Marr, 1982). For Marr, objects and their www.frontiersin.org June 2014 | Volume 5 | Article 577 | 1 Hoffman and Prakash Objects of consciousness surfaces exist when unperceived, and human vision has evolved perceptual representations that are genuine insights into the true to describe their objective properties. nature of the objective world. Bayesian theories of vision assume object permanence. They Evaluating object permanence on evolutionary grounds might model object perception as a process of statistical estimation of seem quixotic, or at least unfair, given that we just noted that object properties, such as surface shape and reflectance, that exist evolutionary theory, as it’s standardly described, assumes object when unperceived. As Alan Yuille and Heinrich Bülthoff put it, permanence(e.g., of DNA and the physical bodies of organisms). “Wedefinevisionasperceptualinference,theestimationofscene Howthencouldonepossiblyuseevolutionarytheorytotestwhat properties from an image or sequence of images ...”(Yuille and it assumes to be true? Bülthoff, 1996). However, Richard Dawkins and others have observed that the There is a long and interesting history of debate about which core of evolution by natural selection is an abstract algorithm properties of objects exist when unperceived. Shape, size, and with three key components: variation, selection, and retention position usually make the list. Others, such as taste and color, (Dennett, 1995; Blackmore, 1999). This abstract algorithm con- often do not. Democritus, a contemporary of Socrates, famously stitutes a “universal Darwinism” that need not assume object claimed, “by convention sweet and by convention bitter, by con- permanence and can be profitably applied in many contexts vention hot, by convention cold, by convention color; but in beyond biological evolution. Thus, it is possible, without beg- reality atoms and void” (Taylor, 1999). ging the question, to use formal models of evolution by natural Locke proposed that “primary qualities” of objects, such as selection to explore whether object permanence is an insight “bulk, figure, or motion” exist when unperceived, but that “sec- or not. ondaryproperties” of objects, such as “colors and smells” do not. Jerry Fodor has criticized the theory of natural selection itself, He then claimed that “...the ideas of primary qualities of bod- arguing, for instance, that it impales itself with an intensional fal- ies are resemblances of them, and their patterns do really exist lacy, viz., inferring from the premise that “evolution is a process in the bodies themselves, but the ideas produced in us by these inwhichcreatureswithadaptivetraitsareselected”totheconclu- secondary qualities have no resemblance of them at all” (Locke, sion that “evolution is a process in which creatures are selected 1690). for their adaptive traits” (Fodor and Piattelli-Palmarini, 2010). Philosophical and scientific debate continues to this day on However, Fodor’s critique seems wide of the mark (Futuyma, whether properties such as color exist when unperceived (Byrne 2010) and the evidence for evolution by natural selection is and Hilbert, 2003; Hoffman, 2006). But object permanence, cer- overwhelming(Coyne,2009;Dawkins,2009). tainly regarding shape and position, is so deeply assumed by the What,then,dowefindwhenweexploretheevolutionof scientific literature in the fields of psychophysics and computa- perception using evolutionary games and genetic algorithms? tional perception that it is rarely discussed. The standard answer, at least among vision scientists, is that we It is also assumed in the scientific study of consciousness and should find that natural selection favors veridical perceptions, the mind-body problem. Here the widely acknowledged failure i.e., perceptions that accurately represent objective properties of to create a plausible theory forces reflection on basic assump- the external world that exist when unperceived. Steven Palmer, tions, including object permanence. But few researchers in fact for instance, in a standard graduate-level textbook, states that give it up. To the contrary, the accepted view is that aspects “Evolutionarily speaking, visual perception is useful only if it is of neural dynamics—from quantum-gravity induced collapses reasonablyaccurate...Indeed,visionisusefulpreciselybecauseit of wavefunctions at microtubules (Hameroff, 1998)toinforma- is so accurate. By andlarge,whatyouseeiswhatyouget.Whenthis tional properties of re-entrant thalamo-cortical loops (Tononi, is true, we have what is called veridicalperception ...perception 2004)—cause, or give rise to, or are identical to, conscious- that is consistent with the actual state of affairs in the environ- ness.AsColinMcGinnputsit,“weknowthatbrainsarethe ment. This is almost always the case with vision ...”(Palmer, de facto causal basis of consciousness, but we have, it seems, 1999). no understanding whatever of how this can be so” (McGinn, Theargument,roughly,isthatthoseofourpredecessorswhose 1989). perceptions were more veridical had a competitive advantage over those whose perceptions were less veridical. Thus, the genes EVOLUTIONANDPERCEPTION that coded for more veridical perceptions were more likely to Thehumanmindispredisposedfromearlychildhoodtoassume propagate to the next generation. We are, with good probability, object permanence, to assume that objects have shapes and posi- the offspring of those who, in each succeeding generation, per- tions in space even when the objects and space are unperceived. It ceived more truly, and thus we can be confident that our own is reasonable to ask whether this assumption is a genuine insight perceptions are, in the normal case, veridical. into the nature of objective reality, or simply a habit that is The conclusion that natural selection favors veridical percep- perhaps useful but not necessarily insightful. tionsiscentraltocurrentBayesianmodelsofperception,inwhich We can look to evolution for an answer. If we assume that perceptual systems use Bayesian inference to estimate true prop- ourperceptual and cognitive capacities have been shaped, at least erties of the objective world, properties such as shape, position, in part, by natural selection, then we can use formal models of motion, and reflectance (Knill and Richards, 1996; Geisler and evolution, such as evolutionary game theory (Lieberman et al., Diehl,2003).Objectsexistandhavethesepropertieswhenunper- 2005; Nowak, 2006) and genetic algorithms (Mitchell, 1998), to ceived, and the function of perception is to accurately estimate exploreif, and under whatcircumstances,naturalselectionfavors pre-existing properties. Frontiers in Psychology | Perception Science June 2014 | Volume 5 | Article 577 | 2 Hoffman and Prakash Objects of consciousness However, when we actually study the evolution of perception THEINTERFACETHEORYOFPERCEPTION usingMonteCarlosimulationsofevolutionarygamesandgenetic Natural selection favors perceptions that are useful though not algorithms, we find that natural selection does not, in general, true. This might seemcounterintuitive, even to experts in percep- favor perceptions that are true reports of objective properties of tion. Palmer, for instance, in the quote above, makes the plausible the environment. Instead, it generally favors perceptual strategies claim that “vision is useful precisely because it is so accurate” that are tuned to fitness (Mark et al., 2010; Hoffman et al., 2013; (Palmer, 1999). Geisler and Diehl agree, taking it as obvious that Marion,2013;Mark,2013). “In general, (perceptual) estimates that are nearer the truth have Why? Several principles emerge from the simulations. First, greater utility than those that are wide of the mark” (Geisler and there is no free information. For every bit of information one Diehl, 2002). Feldman also takes it as obvious that “it is clearly obtains about the external world, one must pay a price in energy, desirable (say from an evolutionary point of view) for an organ- e.g., in calories expended to obtain, process and retain that infor- ism to achieve veridical percepts of the world” (Feldman, 2013). mation. And for every calorie expended in perception, one must Knill and Richards concur that vision “... involves the evolu- go out and kill something and eat it to get that calorie. So tion of an organism’s visual system to match the structure of the natural selection tends to favor perceptual systems that, ceteris world ...”(Knill and Richards, 1996). paribus, use fewer calories. One way to use fewer calories is This assumption that perceptions are useful to the extent that to see less truth, especially truth that is not informative about theyaretrueisprimafacieplausible,anditcomportswellwiththe fitness. assumption of object permanence. For if our perceptions report Second, for every bit of information one obtains about the to us a three-dimensional world containing objects with specific external world, one must pay a price in time. More information shapes and positions, and if these perceptual reports have been requires, in general, more time to obtain and process. But in the shaped by evolution to be true, then we can be confident that real world where predators are on the prowl and prey must be those objects really do, in the normal case, exist and have their wary, the race is often to the swift. It is the slower gazelle that positions and shapes even when unperceived. becomes lunch for the swifter cheetah. So natural selection tends So we find it plausible that perceptions are useful only if true, to favor perceptual systems that, ceteris paribus, take less time. and we find it deeply counterintuitive to think otherwise. But One way to take less time is, again, to see less truth, especially studies with evolutionary games and genetic algorithms flatly truth that is not informative about fitness. contradict this deeply held assumption. Clearly our intuitions Third, in a world where organisms are adapted to niches and needalittle help here. How can we try to understand perceptions require homeostatic mechanisms, the fitness functions guiding that are useful but not true? their evolution are generally not monotonic functions of struc- Fortunately, developments in computer technology have pro- tures or quantities in the world. Too much salt or too little can vided a convenient and helpful metaphor: the desktop of a win- be devastating; something in between is just right for fitness. The dowsinterface(Hoffman,1998,2009,2011,2012,2013;Mausfeld, same goldilocks principle can hold for water, altitude, humidity, 2002; Koenderink, 2011a; Hoffman and Singh, 2012; Singh and andsoon.Inthesecases,perceptions that are tuned to fitness are Hoffman, 2013). Suppose you are editing a text file and that the ipso facto not tuned to the true structure of the world, because the icon for that file is a blue rectangle sitting in the lower left corner two are not monotonically related; knowing the truth is not just of the desktop. If you click on that icon you can open the file and irrelevant, it can be inimical, to fitness. revise its text. If you drag that icon to the trash, you can delete the Fourth, in the generic case where noise and uncertainty are file.Ifyoudragittotheiconforanexternalharddrive,youcan endemictotheperceptualprocess,astrategythatestimatesatrue create a backup of the file. So the icon is quite useful. state of the world and then uses the utility associated to that state Butisittrue?Well,theonlyvisiblepropertiesoftheiconareits to govern its decisions must throw away valuable information position, shape, and color. Do these properties of the icon resem- aboututility. It will in general be driven to extinction by a strategy ble the true properties of the file? Clearly not. The file is not blue that does not estimate the true state of the world, and instead uses orrectangular, and it’s probably not in the lower left corner of the all the information about utility (Marion, 2013). computer. Indeed, files don’t have a color or shape, and needn’t Fifth, more complex perceptual systems are more difficult to have a well-defined position (e.g., the bits of the file could be evolve. Monte Carlo simulations of genetic algorithms show that spread widely over memory). So to even ask if the properties of there is a combinatorial explosion in the complexity of the search the icon are true is to make a category error, and to completely required to evolve more complex perceptual systems. This com- misunderstand the purpose of the interface. One can reasonably binatorial explosion itself is a selection pressure toward simpler askwhethertheiconisusefullyrelatedtothefile,butnotwhether perceptual systems. it truly resembles the file. In short, natural selection does not favor perceptual systems Indeed, a critical function of the interface is to hide the truth. that see the truth in whole or in part. Instead, it favors per- Mostcomputerusersdon’twanttoseethecomplexityoftheinte- ceptions that are fast, cheap, and tailored to guide behaviors grated circuits, voltages, and magnetic fields that are busy behind needed to survive and reproduce. Perception is not about truth, the scenes when they edit a file. If they had to deal with that it’s about having kids. Genes coding for perceptual systems that complexity, they might never finish their work on the file. So increase the probability of having kids are ipso facto the genes the interface is designed to allow the user to interact effectively that are more likely to code for perceptual systems in the next with the computer while remaining largely ignorant of its true generation. architecture. www.frontiersin.org June 2014 | Volume 5 | Article 577 | 3 Hoffman and Prakash Objects of consciousness Ignorant, also, of its true causal structure. When a user drags perception, our perceptions primarily resonate to “affordances,” a file icon to an icon of an external drive, it looks obvious that those aspects of the objective world that have important con- themovementofthefileicontothedriveiconcauses the file to sequences for fitness (Gibson, 1979). While we disagree with be copied. But this is just a useful fiction. The movement of the Gibon’s direct realism and denial of information processing in file icon causes nothing in the computer. It simply serves to guide perception, we agree with his emphasis on the tuning of percep- the user’s operation of a mouse, triggering a complex chain of tion to fitness. causal events inside the computer, completely hidden from the So we must clarify the relationship between truth and fitness. user. Forcing the user to see the true causal chain would be an In evolutionary theory it is as follows. If W denotes the objec- impediment,notahelp. tive world then, for a fixed organism, state, and action, we can Turning now to apply the interface metaphor to human per- think of a fitness function to be a function f:W → [0,1], which ception, the idea is that natural selection has not shaped our per- assigns to each state w of W a fitness value f(w). If, for instance, ceptions to be insights into the true structure and causal nature the organism is a hungry cheetah and the action is eating, then f of objective reality, but has instead shaped our perceptions to be might assign a high fitness value to world state w in which fresh a species-specific user interface, fashioned to guide the behav- raw meat is available; but if the organism is a hungry cow then f iors that we need to survive and reproduce. Space and time are mightassign a low fitness value to the same state w. the desktop of our perceptual interface, and three-dimensional If the true probabilities of states in the world are given by a objects are icons on that desktop. probability measuremonW,thenonecandefineanewprobabil- Ourinterfacegivestheimpressionthatitrevealstruecauseand ity measuremf onW,whereforanyeventAofW,mf(A)issimply effect relations. When one billiard ball hits a second, it certainly the integral of f over A with respect to m; mf must of course be looksasthoughthefirstcausesthesecondtocareenaway.Butthis normalized so that mf(W) = 1. appearance of cause and effect is simply a useful fiction, just as it And here is the key point. A perceptual system that is tuned is for the icons on the computer desktop. to maximize the mutual information with m will not, in gen- Thereisanobviousrejoinder:“Ifthatcobraisjustaniconof eral,maximizemutualinformationwithmf (CoverandThomas, yourinterfacewithnocausalpowers,whydon’tyougrabitbythe 2006). Being tuned to truth, i.e., maximizing mutual information tail?” The answer is straightforward: “I don’t grab the cobra for with m, is not the same as being tuned to fitness, i.e., maximiz- thesamereasonIdon’tcarelesslydragmyfileicontothetrash—I ingmutualinformationwithmf.Indeed,dependingonthefitness could lose a lot of work. I don’t take my icons literally:Thefile, functionf, a perceptual system tuned to truth might carry little or unlike its icon, is not literally blue or rectangular. But I do take noinformationaboutfitness,andviceversa.Itisinthissensethat myiconsseriously.” the interface theory of perception claims that our perceptions are Similarly, evolution has shaped us with a species-specific inter- tunedtofitnessratherthantruth. face whose icons we must take seriously. If there is a cliff, don’t There is another rejoinder: “The interface metaphor is noth- stepover.Ifthereisacobra,don’tgrabitstail.Naturalselection ingnew.Physicistshavetoldusformorethanacenturythat has endowed us with perceptions that function to guide adaptive solid objects are really mostly empty space. So an apparently solid behaviors, and we ignore them at our own peril. stone isn’t the true reality, but its atoms and subatomic particles But, given that we must take our perceptions seriously, it does are.” Physicists have indeed said this since Rutherford published not follow that we must take them literally. Such an inference is his theory of the atomic nucleus in 1911 (Rutherford, 1911). But natural, in the sense that most of us, even the brightest, make it the interface metaphor says something more radical. It says that automatically.WhenSamuelJohnsonheardBerkeley’stheorythat space and time themselves are just a desktop, and that anything “To be is to be perceived” he kicked a stone and said, “I refute it in space and time, including atoms and subatomic particles, are thus!” (Boswell, 1986) Johnson observed that one must take the themselves simply icons. It’s not just the moon that isn’t there stone seriously or risk injury. From this Johnson concluded that when one doesn’t look, it’s the atoms, leptons and quarks them- onemusttakethestoneliterally. But this inference is fallacious. selves that aren’t there. Object permanence fails for microscopic Onemightobjectthattherestillisanimportantsenseinwhich objects just as it does for macroscopic. our perceptual icon of, say, a cobra does resemble the true objec- This claim is, to contemporary sensibilities, radical. But there tive reality: The consequences for an observer of grabbing the tail is a perspective on the intellectual evolution of humanity over the of the cobra are precisely the consequences that would obtain if last few centuries for which the interface theory seems a natural the objective reality were in fact a cobra. Perceptions and internal next step. According to this perspective, humanity has gradually information-bearing structures are useful for fitness-preserving been letting go of the false belief that the way H. sapiens sees the orenhancingbehaviorbecausethereissomemutualinformation worldisaninsightintoobjectivereality. between the predicted utility of a behavior (like escaping) and its Many ancient cultures, including the pre-Socratic Greeks, actualutility. If there’s no mutual information and no mechanism believed the world was flat, for the obvious reason that it looks for increasing mutual information, fitness is low and stays that that way. Aristotle became persuaded, on empirical grounds, that way. Here we use mutual information in the sense of standard the earth is spherical, and this view gradually spread to other cul- information theory (Cover and Thomas, 2006). tures. Reality, we learned, departed in important respects from This point is well-taken. Our perceptual icons do give us gen- someofourperceptions. uine information about fitness, and fitness can be considered an Butthenageocentricmodeloftheuniverse,inwhichtheearth aspect of objective reality. Indeed, in Gibson’s ecological theory of is at the center and everything revolves around it, still held sway. Frontiers in Psychology | Perception Science June 2014 | Volume 5 | Article 577 | 4 Hoffman and Prakash Objects of consciousness Why? Because that’s the way things look to our unaided percep- halt states, and (7) a finite set of simple transition rules (Hopcroft tions. The earth looks like it’s not moving, and the sun, moon, et al., 2006). planets, and stars look like they circle a stationary earth. Not until Turingandothersthenconjecturedthatafunctionisalgorith- the work of Copernicus and Kepler did we recognize that once mically computable if and only if it is computable by a Turing again reality differs, in important respects, from our perceptions. machine. This “Church-Turing Thesis” can’t be proven, but it This was difficult to swallow. Galileo was forced to recant in the could in principle be falsified by a counterexample, e.g., by some Vatican basement, and Giordano Bruno was burned at the stake. exampleofaprocedurethateveryoneagreedwascomputablebut Butwefinally, and painfully, accepted the mismatch between our for which no Turing machine existed. No counterexample has yet perceptions and certain aspects of reality. been found, and the Church-Turing thesis is considered secure, Theinterfacetheoryentailsthatthesefirsttwostepsweremere even definitional. warmup.Thenextstepintheintellectual history of H. sapiens is Similarly, to construct a theory of consciousness we propose a a big one. We must recognize that all of our perceptions of space, simplebutrigorousformalismcalledaconsciousagent,consisting time and objects no more reflect reality than does our perception ofsixcomponents.Wethenstatetheconsciousagentthesis,which ofaflatearth.It’snotjustthisorthataspectofourperceptions claims that every property of consciousness can be represented that must be corrected, it is the entire framework of a space-time by some property of a conscious agent or system of interacting containing objects, the fundamental organization of our percep- conscious agents. The hope is to start with a small and simple tual systems, that must be recognized as a mere species-specific set of definitions and assumptions, and then to have a complete modeofperceptionratherthananinsightintoobjectivereality. theory of consciousness arise as a series of theorems and proofs Bythistimeitshouldbeclearthat,iftheargumentsgivenhere (or simulations, when complexity precludes proof). We want a are sound, then the current Bayesian models of object perception theory of consciousness qua consciousness, i.e., of consciousness needmorethantinkeringaroundtheedges,theyneedfundamen- onitsownterms,notassomethingderivativeoremergentfroma tal transformation. And this transformation will necessarily have prior physical world. ramifications for scientific questions well-beyond the confines of Nodoubtthisapproachwillstrikemanyasprimafacieabsurd. computational models of object perception. It is a commonplace in cognitive neuroscience, for instance, that One example is the mind-body problem. A theory in which most of our mental processes are unconscious processes (Bargh objects and space-time do not exist unperceived and do not have and Morsella, 2008). The standard account holds that well more causal powers, cannot propose that neurons—which by hypoth- than 90% of mental processes proceed without conscious aware- esis do not exist unperceived and do not have causal powers— ness. Therefore, the proposal that consciousness is fundamental cause any of our behaviors or conscious experiences. This is so is, to contemporary thought, an amusing anachronism not worth contrary to contemporary thought in this field that it is likely to serious consideration. be taken as a reductio of the view rather than as an alternative Thiscritique is apt. It’s clear from many experiments that each direction of inquiry for a field that has yet to construct a plausible of us is indeed unaware of most of the mental processes underly- theory. ingouractionsandconsciousperceptions.Butthisisnosurprise, given the interface theory of perception. Our perceptual inter- DEFINITIONOFCONSCIOUSAGENTS faces have been shaped by natural selection to guide, quickly and If our reasoning has been sound, then space-time and three- cheaply, behaviors that are adaptive in our niche. They have not dimensional objects have no causal powers and do not exist been shaped to provide exhaustive insights into truth. In con- unperceived.Therefore,weneedafundamentallynewfoundation sequence, our perceptions have endogenous limits to the range from which to construct a theory of objects. Here we explore the andcomplexityoftheir representations. It was not adaptive to be possibility that consciousness is that new foundation, and seek a awareofmostofourmentalprocessing,justasitwasnotadaptive mathematicallyprecise theory. The idea is that a theory of objects tobeawareofhowourkidneysfilterblood. requires, first, a theory of subjects. We must be careful not to assume that limitations of our Thisis, of course, a non-trivial endeavor. Frank Wilczek, when species-specific perceptions are insights into the true nature of discussing the interpretation of quantum theory, said, “The rel- reality. My friend’s mind is not directly conscious to me, but that evant literature is famously contentious and obscure. I believe it does not entail that my friend is unconscious. Similarly, most of will remain so until someone constructs, within the formalism of my mental processes are not directly conscious to me, but that quantummechanics, an “observer,” that is, a model entity whose does not entail that they are unconscious. Our perceptual sys- states correspondtoarecognizablecaricatureofconsciousaware- tems have finite capacity, and will therefore inevitably simplify ness ...Thatisaformidableproject,extendingwell-beyondwhat and omit. We are well-advised not to mistake our omissions and is conventionally considered physics” (Wilczek, 2006). simplifications for insights into reality. The approach we take toward constructing a theory of con- There are of course many other critiques of an approach sciousness is similar to the approach Alan Turing took toward that takes consciousness to be fundamental: How can such an constructing a theory of computation. Turing proposed a simple approach explain matter, the fundamental forces, the Big Bang, but rigorous formalism, now called the Turing machine (Turing, the genesis and structure of space-time, the laws of physics, 1937; Herken, 1988). It consists of six components: (1) a finite evolution by natural selection, and the many neural correlates set of states, (2) a finite set of symbols, (3) a special blank sym- of consciousness? These are non-trivial challenges that must be bol, (4) a finite set of input symbols, (5) a start state, (6) a set of faced by the theory of conscious agents. But for the moment we www.frontiersin.org June 2014 | Volume 5 | Article 577 | 5 Hoffman and Prakash Objects of consciousness will postpone them and develop the theory of conscious agents output alphabet Y and a probability transition matrix p(x|y)that itself. expresses the probability of observing the output symbol y given Conscious agent is a technical term, with a precise mathemat- that we send the symbol x”(Cover and Thomas, 2006). Thus, a ical definition that will be presented shortly. To understand the discrete channel is simply a Markovian kernel. technicalterm,itcanbehelpfultohavesomeintuitionsthatmoti- So, each time a conscious agent interacts with the world and, vate the definition. The intuitions are just intuitions, and if they in consequence, has a conscious experience, we can think of this don’t help they can be dropped. What does the heavy lifting is the interaction as a message being passed from the world to the con- definition itself. scious agent over a channel. Similarly, each time the conscious Akeyintuition is that consciousness involves three processes: agent has a conscious experience and, in consequence, decides on perception, decision,andaction. anactiontotake,wecanthinkofthisdecisionasamessagebeing In the process of perception, a conscious agent interacts with passedoverachannelwithintheconsciousagentitself.Andwhen the world and, in consequence, has conscious experiences. the conscious agent then takes the action and, in consequence, In the process of decision, a conscious agent chooses what alters the state of the world, we can think of this as a message actions to take based on the conscious experiences it has. being passed from the conscious agent to the world over a chan- In the process of action, the conscious agent interacts with the nel. In the discrete case, we can keep track of the number of times world in light of the decision it has taken, and affects the state of each channel is used. That is, we can count the number of mes- the world. sages that are passed over each channel. Assuming that all three Another intuition is that we want to avoid unnecessarily channels (perception, decision, action) all work in lock step, we restrictive assumptions in constructing a theory of consciousness. can use one counter, N,tokeeptrackofthenumberofmessages Our conscious visual experience of nearby space, for instance, that are passed. is approximately Euclidean. But it would be an unnecessary These are some of the intuitions that underlie the definition restriction to require that all of our perceptual experiences be of conscious agent that we will present. These intuitions can be represented by Euclidean spaces. represented pictorially in a diagram, as shown in Figure1.The However it does seem necessary to discuss the probability of channel P transmits messages from the world W, leading to con- having a conscious experience, of making a particular decision, scious experiences X. The channel D transmits messages from X, and of making a particular change in the world through action. leading to actions G. The channel A transmits messages from G Thus, it seems necessary to assume that we can represent the that are received as new states of W. The counter N is an inte- world, our conscious experiences, and our possible actions with ger that keeps track of the number of messages that are passed on probability spaces. each channel. We also want to avoid unnecessarily restrictive assumptions In what follows we will be using the notion of a measurable about the processes of perception, decision, and action. We might space. Recall that a measurable space, (X, X), is a set X together find, for instance, that a particular decision process maximizes withacollectionXofsubsetsofX,calledevents,thatsatisfiesthree expected utility, or minimizes expected risk, or builds an explicit properties: (1) X is in X;(2)X is closed under complement(i.e., if model of the self. But it would be an unnecessary restriction to asetAisinXthenthecomplementofAisalsoinX);and(3)Xis require this of all decisions. closed under countable union. The collection of events X is a σ- However, when considering the processes of perception, deci- algebra (Athreya and Lahiri, 2006). A probability measure assigns sion and action, it does seem necessary to discuss conditional a probability to each event in X. probability. It seems necessary, for instance, to discuss the con- Withtheseintuitions, we now present the formal definition of ditional probability of deciding to take a specific action given a a conscious agent where, for the moment, we simply assume that specificconsciousexperience,theconditionalprobabilityofapar- the world is a measurable space (W, W). ticular change in the world given that a specific action is taken, Definition1.Aconsciousagent,C,isasix-tuple andtheconditionalprobability of a specific conscious experience given a specific state of the world. C=((X,X),(G,G),P,D,A,N)), (1) A general way to model such conditional probabilities is by the mathematicalformalismofMarkoviankernels(Revuz,1984). where: One can think of a Markovian kernel as simply an indexed list of probability measures. In the case of perception, for instance, (1) (X, X)and(G,G)aremeasurablespaces; a Markovian kernel might specify that if the state of the world is (2) P : W ×X→[0,1],D:X ×G→[0,1],A:G×W→[0,1] w1,thenhereisalistoftheprobabilitiesforthevariousconscious are Markovian kernels; and experiencesthatmightresult,butifthestateoftheworldisw2, (3) N is an integer. thenhereisadifferentlistoftheprobabilitiesforthevariouscon- scious experiences that might result, and so on for all the possible states of the world. A Markovian kernel onafinitesetofstatescan For convenience we will often write a conscious agent C as bewritten as matrix in which the entries in each row sum to 1. A Markovian kernel can also be thought of as an informa- C=(X,G,P,D,A,N), (2) tion channel. Cover and Thomas, for instance, define “a discrete channel to be a system consisting of an input alphabet X and omitting the σ-algebras. Frontiers in Psychology | Perception Science June 2014 | Volume 5 | Article 577 | 6 Hoffman and Prakash Objects of consciousness FIGURE2|Twoconsciousagents,C andC .EachispartoftheworldW 1 2 for the other conscious agent. The lower part of the diagram represents C 1 and the upper part represents C2. This creates an undirected combination of C1 and C2, a concept we define in section The Combination Problem. existing data and theories, and make predictions that are novel, FIGURE1|Adiagramofaconsciousagent.Aconsciousagenthassix interesting and testable. components as illustrated here. The maps P, D,andA can be thought of as communication channels. TWOCONSCIOUSAGENTS Conscious realism can be expressed mathematically in a simple Given that P, D,andA are channels, each has a channel form.Considertheelementarycase,inwhichtheworldW ofone capacity, viz., a highest rate of bits per channel use, at which conscious agent, information can be sent across the channel with arbitrarily low C =(X ,G ,P ,D ,A ,N ), (3) chance of error (Cover and Thomas, 2006). 1 1 1 1 1 1 1 Theformalstructureofaconsciousagent,likethatofaTuring machine,issimple.Nevertheless,wewillpropose,inthenextsec- contains just C1 and one other agent, tion, a “conscious-agent thesis” which, like the Church-Turing C =(X ,G ,P ,D ,A ,N ), (4) thesis, claims wide application for the formalism. 2 2 2 2 2 2 2 CONSCIOUSREALISM andviceversa. This is illustrated in Figure 2. One glaring feature of the definition of a conscious agent is that Observe that although W is the world it cannot properly be it involves the world, W. This is not an arbitrary choice; W is called, in this example, the external world of C1 or of C2 because required to define the perceptual map P andactionmapA of the C1andC2areeachpartofW.ThisconstructionofW requiresthe conscious agent. compatibility conditions This raises the question: What is the world? If we take it to be P =A , (5) the space-time world of physics, then the formalism of conscious 1 2 agents is dualistic, with some components (e.g., X and G) refer- P =A , (6) ring to consciousness and another, viz., W, referring to a physical 2 1 world. N =N . (7) 1 2 We want a non-dualistic theory. Indeed, the monism we want takes consciousness to be fundamental. The formal- These conditions mean that the perceptions of one conscious ism of conscious agents provides a precise way to state this agentareidenticaltotheactionsoftheother,andthattheircoun- monism. ters are synchronized. To understand this, recall that we can think Hypothesis1.Consciousrealism:TheworldW consistsentirely of P , P , A ,andA asinformationchannels.Sointerpreted,con- 1 2 1 2 of conscious agents. ditions (5) and (6) state that the action channel of one agent is Consciousrealismisaprecisehypothesisthat,ofcourse,might the same information channel as the perception channel of the be precisely wrong. We can explore its theoretical implications other agent. Condition (7) states that the channels of both agents in the normal scientific manner to see if they comport well with operate in synchrony. www.frontiersin.org June 2014 | Volume 5 | Article 577 | 7 Hoffman and Prakash Objects of consciousness FIGURE3|Twoadjacentconsciousagents,C andC .Eachagent 1 2 receives messages from the other (indicated by the concave receivers) and sends messages to the other (indicated by the semicircular transmitters). Arrows show the direction of information flow. If two conscious agents C1 and C2 satisfy the commuting dia- gramofFigure2,thenwesaythattheyarejoinedoradjacent:the experiences and actions of C1 affect the probabilities of experi- ences and actions for C2 andviceversa.Figure3 illustrates the ideassofar. WecansimplifythediagramsfurtherandsimplywriteC1—C2 to represent two adjacent conscious agents. THREECONSCIOUSAGENTS Anynumberofconsciousagentscanbejoined.Considerthecase of three conscious agents, C =(X,G,P,D,A,N),i=1,2,3. (8) i i i i i i i This is illustrated in Figure 4,andcompactlyinFigure5. Because C1 interacts with C2 and C3, its perceptions are affected by both C2 and C3. Thus, its perception kernel, P , must reflect the inputs of C and C .Wewriteitas 1 2 3 follows: P =P ⊗P :(G ×G )×X →[0,1], (9) FIGURE4|Threeadjacentconsciousagents.Thethirdagentis 1 12 13 2 3 1 replicated at the top and bottom of the diagram for visual simplicity. where X =σ(X ×X ), (10) receive from C2 need not be the same as the kinds of perceptions 1 12 13 that C1 can receive from C3. (X , X ) is the measurable space of perceptions that C can Because C1 interacts with C2 and C3, its actions affect both. 12 12 1 However, the way C1 acts on C2 might differ from how it acts on receive from C ,and(X , X ) is the measurable space of 2 13 13 C3, and the definition of its action kernel, A1, must allow for this perceptions that C1 can receive from C3,andσ(X12 × X13) difference of action. Therefore, we define the action kernel, A1,to denotes the σ-algebra generated by the Cartesian product of bethetensorproduct X and X . The tensor product P of (9) is given by the 12 13 1 formula A =A ⊗A :G ×σ(X ×X )→[0,1], (12) 1 12 13 1 2 3   P (g ,g ),(x ,x ) = P (g ,x )P (g ,x ), (11) 1 2 3 12 13 12 2 12 13 3 13 where whereg ∈ G ,g ∈ G ,x ∈X ,andx ∈X .Notethat(11) G =G ×G , (13) 2 2 3 3 12 12 13 13 1 12 13 allows that the perceptions that C1 gets from C2 could be entirely different from those it gets from C3, and expresses the probabilis- (G12, G12) is the measurable space of actions that C1 can take on tic independence of these perceptual inputs. In general, X need C ,and(G ,G )isthemeasurablespaceofactionsthatC can 12 2 13 13 1 not be identical to X13, since the kinds of perceptions that C1 can take on C3. Frontiers in Psychology | Perception Science June 2014 | Volume 5 | Article 577 | 8 Hoffman and Prakash Objects of consciousness FIGURE7|Threeconsciousagentswithdirectedjoins.Herewe assumeA =P ,A =P ,andA =P . 1 2 2 3 3 1 FIGURE5|Threeadjacentconsciousagents.Thisisacompact representation of the diagram in Figure 4. FIGURE8|Simplifiedgraphofthreeconsciousagentswithdirected joins. FIGURE6|Threeconsciousagentswhosegraphiscomplete. message from C2 and sends no message to C3. Similar remarks hold, mutatis mutandis,forC2 and C3. In this situation, the three conscious agents have the property Figure7canbesimplifiedasshowninFigure8. thateverypairisadjacent;wesaythatthegraphofthethreeagents Directed joins can model the standard situation in visual is complete.ThisisillustratedinFigure6. perception, in which there are multiple levels of visual represen- Sofarwehaveconsideredjoinsthatareundirected,inthe tations,onelevelbuildingonothersbelowit.Forinstance,atone sense that if C1 sends a message to C2 then C2 sends a message level there could be the construction of 2D motions based on a to C1. However, it is also possible for conscious agents to have solution to the correspondence problem; at the next level there directed joins.ThisisillustratedinFigure7.Inthiscase,C1 sends could be a computation of 3D structure from motion, based on amessagetoC2 and receives a message from C3, but receives no the 2D motions computed at the earlier level (Marr, 1982). So www.frontiersin.org June 2014 | Volume 5 | Article 577 | 9 Hoffman and Prakash Objects of consciousness anagentC1 mightsolvethecorrespondenceproblemandpassits There are total of 24 = 16 possible states for the dynamics of solution to C2, which solves the structure-from-motion problem, the two agents, which we can write as |0000, |0001, |0010, ... andthenpassesitssolution to C3, which does object recognition. |1111, where the leftmost digit is the state of X1, the next digit Wecan join any number of conscious agents into any multi- the state of G1,thenextofX2, and the rightmost of G2. graph, where nodes denote agents and edges denote directed or The asymptotic (i.e., long-term) dynamics of these two con- undirectedjoinsbetweenagents(ChartrandandPing,2012).The scious agents can be characterized by its absorbing sets and their nodes can have any finite degree, i.e., any finite number of edges. periods. Recall that an absorbing set for such a dynamics is As a special case, conscious agents can be joined to form deter- a smallest set of states that acts like a roach motel: once the ministic or non-deterministic cellular automata (Ceccherini- dynamics enters the absorbing set it never leaves, and it forever Silberstein and Coornaert, 2010) and universal Turing machines cycles periodically through the states within that absorbing set. (Cook,2004). It is straightforward to verify that for the simple dynamics of DYNAMICSOFTWOCONSCIOUSAGENTS conscious agents just described, the asymptotic behavior is as follows: Twoconsciousagents C =(X ,G ,P ,D ,A ,N ), (14) (1) {|0000} is absorbing with period 1; 1 1 1 1 1 1 1 (2) {|1111} is absorbing with period 1; and (3) {|0101, |1010}isabsorbingwithperiod2; (4) {|0001, |1000, |0100, |0010} is absorbing with period 4, andcycles in that order; C =(X ,G ,P ,D A ,N ), (15) 2 2 2 2 2, 2 2 (5) {|0011, |1001, |1100, |0110} is absorbing with period 4, andcycles in that order; can be joined, as illustrated in Figure2, to form a dynamical (6) {|0111, |1011, |1101, |1110} is absorbing with period 4, system. Here we discuss basic properties of this dynamics. andcycles in that order. Thestatespace,E,ofthedynamicsisE = X ×G ×X ×G , 1 1 2 2 with product σ-algebra E. The idea is that for the current step, SECONDEXAMPLEOFASYMPTOTICBEHAVIOR t ∈ N, of the dynamics, the state can be described by the vec- If we alter this dynamics by simply changing the kernel D1 from tor (x (t),g (t),x (t),g (t)), and based on this state four actions 1 1 2 2 an identity matrix to the matrix D1 = ((0,1),(1,0)), then the happen simultaneously: (1) agent C1 experiences the perception asymptotic behavior changes to the following: x (t) ∈ X and decides, according to D , on a specific action 1 1 1 g (t) ∈ G to take at step t + 1; (2) agent C , using A , takes 1 1 1 1 (1) {|0000, |0100, |0110, |0111, |1111, |1011, |1001, |1000} the action g1(t) ∈ G1;(3)agentC2 experiences the perception is absorbing with period 8, and cycles in that order; x (t) ∈ X and decides, according to D , on a specific action 2 2 2 (2) {|0001, |1100, |0010, |0101, |1110, |0011, |1101, |1010} g (t) ∈ G to take at step t + 1; (4) agent C , using A , takes the 2 2 2 2 is absorbing with period 8, and cycles in that order. action g2(t) ∈ G2. Thus, the state evolves by a kernel If instead of changing D we changed D (or A or A )to 1 2 1 2 L : E × E →[0,1], (16) ((0,1),(1,0)), we would get the same asymptotic behavior. Thus, in general, an asymptotic behavior corresponds to an equivalence whichisgiven,forstatee = (x (t),g (t),x (t),g (t)) ∈ E at time class of interacting conscious agents. 1 1 2 2 The range of possible dynamics of pairs of conscious agents t and event B ∈ E, comprised of a measurable set of states of the is huge, and grows as one increases the richness of the state form(x (t +1),g (t +1),x (t +1),g (t +1)),by 1 1 2 2 space E and, therefore, the set of possible kernels. The possibil-  ities increase as one considers dynamical systems of three or more L(e,B) = A (g (t),dx (t + 1))D (x (t),dg (t + 1))A (g (t), conscious agents, with all the possible directed and undirected 2 2 1 1 1 1 1 1 B joins among them, forming countless connected multi-graphs or dx (t + 1))D (x (t),dg (t + 1)). (17) amenablegroups. 2 2 2 2 With this brief introduction to the dynamics of conscious This is not kernel composition; it is simply multiplication of the agents we are now in a position to state another key hypothesis. four kernel values. The idea is that at each step of the dynamics Hypothesis 2. Conscious-agent thesis. Every property of con- eachofthefourkernelsactssimultaneouslyandindependentlyof sciousness can be represented by some property of a dynamical the others to transition the state (x (t),g (t),x (t),g (t)) to the system of conscious agents. 1 1 2 2 next state (dx (t + 1), dg (t + 1), dx (t + 1), dg (t + 1)). 1 1 2 2 THECOMBINATIONPROBLEM FIRSTEXAMPLEOFASYMPTOTICBEHAVIOR Conscious realism and the conscious-agent thesis are strong For concreteness, consider the simplest possible case where (1) claims, and face a tough challenge: Any theory that claims con- X , G , X ,andG each have only two states which, using Dirac sciousness is fundamental must solve the combination problem 1 1 2 2 notation, we denote |0 and |1,and(2)eachofthekernelsA2, (Seager, 1995; Goff, 2009; Blamauer, 2011; Coleman, 2014). D , A ,andD isa2×2identitymatrix. William Seager describes this as “the problem of explaining how 1 1 2 Frontiers in Psychology | Perception Science June 2014 | Volume 5 | Article 577 | 10 Hoffman and Prakash Objects of consciousness the myriad elements of ‘atomic consciousness’ can be combined Theorem1.(Undirected Join Theorem.) An undirected join of into a new, complex and rich consciousness such as that we twoconsciousagentscreates a new conscious agent. possess” (Seager, 1995). Proof.(By construction.) Let two conscious agents William James saw the problem back in 1890: “Where the ele- mental units are supposed to be feelings, the case is in no wise C =((X ,X ),(G ,G ),P ,D ,A ,N ), (18) 1 1 1 1 1 1 1 1 1 altered. Take a hundred of them, shuffle them and pack them as close together as you can (whatever that may mean); still each and remains the same feeling it always was, shut in its own skin, win- dowless, ignorant of what the other feelings are and mean. There C =((X ,X ),(G ,G ),P ,D ,A ,N ), (19) would be a hundred-and-first feeling there, if, when a group or 2 2 2 2 2 2 2 2 2 series of such feelings were set up, a consciousness belonging to have an undirected join. Let the group as such should emerge. And this 101st feeling would be a totally new fact; the 100 original feelings might, by a curious physical law, be a signal for its creation, when they came together; C=((X,X),(G,G),P,D,A,N)) (20) but they would have no substantial identity with it, nor it with them, and one could never deduce the one from the others, or where (in any intelligible sense) say that they evolved it. ... The pri- vate minds do not agglomerate into a higher compound mind” X = X ×X , (21) 1 2 (James, 1890/2007). G=G ×G, (22) There are really two combination problems. The first is 1 2 the combination of phenomenal experiences, i.e., of qualia. For T P = P ⊗P :G ×X→[0,1], (23) 1 2 instance, one’s taste experiences of salt, garlic, onion, basil and D=D ⊗D :X×G→[0,1], (24) tomato are somehow combined into the novel taste experience 1 2 of a delicious pasta sauce. What is the relationship between one’s T A=A ⊗A :G×X →[0,1], (25) experiences of the ingredients and one’s experience of the sauce? 1 2 N =N =N , (26) The second problem is the combination of subjects of expe- 1 2 riences. In the sauce example, a single subject experiences the ingredients and the sauce, so the problem is to combine experi- where superscript T indicates transpose, e.g., XT = X ×X ; 2 1 ences within a single subject. But how can we combine subjects where X is the σ-algebra generated by the Cartesian product of themselves to create a new unified subject? Each subject has its X1andX2;whereGistheσ-algebragenerated by G1and G2;and point of view. How can different points of view be combined to where the Markovian kernels P, D,andA are given explicitly, in give a new, single, point of view? the discrete case, by No rigorous theory has been given for combining phenome- nal experiences, but there is hope. Sam Coleman, for instance, P((g ,g ),(x ,x )) = P ⊗P ((g ,g ),(x ,x )) 2 1 1 2 1 2 2 1 1 2 is optimistic but notes that “there will have to be some sort of =P (g ,x )P (g ,x ), (27) qualitative blending or pooling among the qualities carried by 1 2 1 2 1 2 each ultimate: if each ultimate’s quality showed up as such in the D((x ,x ),(g ,g )) = D ⊗D ((x ,x ),(g ,g )) 1 2 1 2 1 2 1 2 1 2 macro-experience,itwouldlackthenotablehomogeneityof(e.g., =D(x ,g )D (x ,g ), (28) color experience, and plausibly some mixing of basic qualities is 1 1 1 2 2 2 required to obtain the qualities of macro-experience” (Coleman, A((g ,g ),(x ,x )) = A ⊗A ((g ,g ),(x ,x )) 1 2 2 1 1 2 1 2 2 1 2014). =A(g ,x )A (g ,x ), (29) Likewise, no rigorous theory has been given for combining 1 1 2 2 2 1 subjects. But here there is little hope. Thomas Nagel, for instance, where g ∈ G , g ∈ G , x ∈ X ,andx ∈ X .ThenC satisfies says “Presumably the components out of which a point of view 1 1 2 2 1 1 2 2 is constructed would not themselves have to have points of view” the definition of a conscious agent.  (Nagel, 1979). Coleman goes further, saying, “it is impossible to Thus, the undirected join of two conscious agents (illustrated explainthegenerationofamacro-subject(likeoneofus)interms in Figure2) creates a single new conscious agent that we call of the assembly of micro-subjects, for, as I show, subjects cannot their undirected combination. It is straightforward to extend the combine”(Coleman,2014). construction in Theorem 1 to the case in which more than So at present there is the hopeful, but unsolved, problem of two conscious agents have an undirected join. In this case the combining experiences and the hopeless problem of combining joined agents create a single new agent that is their undirected subjects. combination. Thetheory of conscious agents provides two ways to combine Theorem 2.(Directed Join Theorem.) A directed join of two conscious agents: undirected combinations and directed combi- conscious agents creates a new conscious agent. nations. We prove this, and then consider the implications for Proof.(By construction.) Let two conscious agents solving the problems of combining experiences and combining subjects. C =((X ,X ),(G ,G ),P ,D ,A ,N ), (30) 1 1 1 1 1 1 1 1 1 www.frontiersin.org June 2014 | Volume 5 | Article 577 | 11 Hoffman and Prakash Objects of consciousness and the phenomenology of decision making is intimately connected with the spaces of perceptual experiences that are integrated in C =((X ,X ),(G ,G ),P ,D ,A ,N ), (31) the decision process. This is an interesting prediction of the for- 2 2 2 2 2 2 2 2 2 malism of conscious agents, and suggests that solution of the have the directed join C1 → C2.Let combination problem for experience will necessarily involve the integration of experience with decision-making. C=((X,X),(G,G),P,D,A,N)) (32) We turn now to the combination of subjects. Coleman describes subjects as follows: “The idea of being a subject goes where with being an experiential entity, something conscious of phe- nomenal qualities. That a given subject has a particular phe- X = X , (33) 1 nomenological point of view can be taken as saying that there G=G2, (34) exists a discrete ‘sphere’ of conscious-experiential goings-on cor- P = P , (35) respondingtothissubject,withregardtowhichothersubjectsare 1 distinct in respect of the phenomenal qualities they experience, D=DAD :X ×G →[0,1], (36) andtheyhavenodirect(i.e.,experiential)accesstothequalitative 1 1 2 1 2 A=A, (37) fieldenjoyedbythefirstsubject.Asubject,then,canbethoughtof 2 asapointofviewannexedtoaprivatequalitativefield”(Coleman, N =N =N , (38) 2014). 1 2 A conscious agent Ci is a subject in the sense described by where D A D denotes kernel composition. Then C satisfies the Coleman. It has a distinct sphere, X , of “conscious-experiential 1 1 2 i definition of a conscious agent.  goings-on”andhasnodirectexperientialaccesstothesphere,Xj, Thus,thedirectedjoinoftwoconsciousagentscreatesasingle of experiences of any other conscious agent Cj. Moreover, a con- new conscious agent that we call their directed combination.Itis scious agent is a subject in the further sense of being an agent, i.e., straightforward to extend the construction in Theorem 2 to the makingdecisions and taking actions on its own. Thus, according case in which more than one conscious agent has a directed join to the theory being explored here a subject, a point of view, is a to C2. In this case, all such agents, together with C2,createanew six-tuple that satisfies the definition of a conscious agent. agent that is their directed combination. The problem with combining subjects is, according to Goff, GivenTheorems1and2,wemakethefollowing that “It is never the case that the existence of a number (one or Conjecture 3:(Combination Conjecture.) Given any pseu- more) of subjects of experience with certain phenomenal char- dograph of conscious agents, with any mix of directed and acters a priori entails the existence of some other subject of undirected edges, then any subset of conscious agents from the experience” (Goff, 2009). pseudograph, adjacent to each other or not, can be combined to Coleman goes further, saying that “The combination of sub- create a new conscious agent. jects is a demonstrablyincoherentnotion,notjustonelackingina How do these theorems address the problems of combining prioriintelligibility...”(Coleman,2014).Heexplainswhy:“...a experiences and subjects? We consider first the combination of set of points of view have nothingtocontributeassuchtoasingle, experiences. unified successor point of view. Their essential property defines Suppose C1 has a space of possible perceptual experiences X1, themagainst it: in so far as they are points of view they are expe- and C2 has a space of possible perceptual experiences X2.Then rientially distinct and isolated—they have different streams of their undirected join creates a new conscious agent C that has consciousness. The diversity of the subject-set, of course, derives a space of possible perceptual experiences X = X × X .Inthis from the essential oneness of any given member: since each sub- 1 2 case, C has possible experiences that are not possible for C1 or ject is essentially a oneness, a set of subjects are essentially diverse, C2. If, for instance, C1 can see only achromatic brightness, and for they must be a set of onenesses. Essential unity from essential C2 canseeonlyvariationsinhue,thenC can see hues of varying diversity ...is thus a case of emergence ...” brightness. Although C’s possible experiences X are the Cartesian Thetheoryofconsciousagentsproposesthatasubject,apoint product of X and X ,neverthelessC might exhibit perceptual of view, is a six-tuple that satisfies the definition of conscious 1 2 dependence between X and X , due to feedback inherent in an agent. The directed and undirected join theorems give construc- 1 2 undirected join (Maddox and Ashby, 1996; Ashby, 2000). tive proofs of how conscious agents and, therefore, points of view, For a directed join C1 → C2, the directed-combination agent can be combined to create a new conscious agent, and thus a C has a space of possible perceptual experiences X = X1.This new point of view. The original agents, the original subjects, are might suggest that no combination of experiences takes place. not destroyed in the creation of the new agent, the new sub- However, C has a decision kernel D that is given by the kernel ject. Instead the original subjects structurally contribute in an product D A D . This product integrates (in the literal sense of understandable, indeed mathematically definable, fashion to the 1 1 2 integral calculus) over the entire space of perceptual experiences structureandpropertiesofthenewagent.Theoriginalagentsare, X , making these perceptual experiences an integral part of the indeed, influenced in the process, because they interact with each 2 decision process. This comports well with evidence that there is other. But they retain their identities. And the new agent has new something it is like to make a decision (Nahmias et al., 2004; properties not enjoyed by the constituent agents, but which are BayneandLevy,2006),andsuggeststheintriguingpossibilitythat intelligible from the structure and interactions of the constituent Frontiers in Psychology | Perception Science June 2014 | Volume 5 | Article 577 | 12 Hoffman and Prakash Objects of consciousness agents. In the case of undirected combination, for instance, we • Eachinvariant event ρ is partitioned into a finite number dρof have seen that the new agent can have periodic asymptotic prop- “asymptotic” events, indexed by ρ and by δ = 1, ..., dρ,so erties that are not possessed by the constituent agents but that are that once the chain enters the asymptotic event δ,itwillthen intelligible—and thus not emergent in a brute sense—from the proceed, with certainty, to δ + 1, δ + 2, and so on, cyclically structures and interactions of the constituent agents. aroundthesetofasymptoticeventsfortheinvarianteventρ. Thus,inshort,thetheoryofconsciousagentsprovidesthefirst rigorous theoretical account of the combination of subjects. The Then there is a correspondence between eigenfunctions of formalismisrichwithdeductiveimplicationstobeexplored.The L and harmonic functions of Q (Revuz, 1984, p. 210) discussion here is just a start. But one hint is the following. The Welet undirected combination of two conscious agents is a single con- λ =exp(2iπk/dρ), (41) scious agent whose world, W, is itself. This appears to be a model ρ,k of introspection, in which introspection emerges, in an intelligible fashion, from the combination of conscious agents. and MICROPHYSICALOBJECTS dρ f =(λ )δU (42) Wehavesketched a theory of subjects. Now we use it to sketch a ρ,k ρ,k ρ,δ theoryofobjects,beginningwiththemicroscopicandproceeding δ=1 to the macroscopic. whereρistheindexovertheinvariantevents(i.e.,absorbingsets), The idea is that space-time and objects are among the sym- the variable k is an integer modulo d ,andU is the indicator bols that conscious agents employ to represent the properties and ρ ρ,δ interactions of conscious agents. Because each agent is finite, but function of the asymptotic event with index ρ, δ. For instance, the realm of interacting agents is infinite, the representations of in the example of section First Example of Asymptotic Behavior, each agent, in terms of space-time and objects, must omit and there are 6 absorbing sets, so ρ = 1,2,...,6. The first absorbing simplify. Hence the perceptions of each agent must serve as an set has only one state, so d1 = 1. Similarly, d2 = 1, d3 = 2, d4 = interface to that infinite realm, not as an isomorphic map. d5 = d6 = 4.ThefunctionU1,1 hasthevalue1onthestate|0000 Interacting conscious agents form dynamical systems, with and 0 for all other states; U5,3 has the value 1 on the state |1100 asymptotic (i.e., long-term) behaviors. We propose that micro- and0forallotherstates. physical objects represent asymptotic properties of the dynamics Thenitisatheoremthat of conscious agents, and that space-time is simply a convenient Lf =λ f , (43) framework for this representation. Specifically, we observe that ρ,k ρ,k ρ,k the harmonic functions of the space-time chain that is associated withthedynamicsofasystemofconsciousagentsareidenticalto i.e., that fρ,k is an eigenfunction of L with eigenvalue λρ,k,and the wave function of a free particle; particles are vibrations not of that strings but of interacting conscious agents. −n Consider, for concreteness, the system of two conscious agents g (· , n) = (λ ) f , (44) ρ,k ρ,k ρ,k of section Dynamics of Two Conscious Agents, whose dynam- icsisgovernedbythekernelL of (17). This dynamics is clearly is Q-harmonic (Revuz, 1984). Then, using (41–42), we have Markovian, because the change in state depends only on the cur- rent state. The space-time chain associated to L has, by definition, dρ the kernel g (·,n) = exp(2iπk/d )−n exp(2iπk/d )δU ρ,k ρ ρ ρ,δ Q:(E×N)×(E⊗2N)→[0,1], (39) δ=1 dρ given by =exp(2iπk δ −2iπk n )Uρ,δ δ=1 dρ dρ  dρ L e, A if m = n +1,  Q((e, n), A×{m}) = ( ) (40) kδ kn = cis(2π −2π )U 0, otherwise, d d ρ,δ δ=1 ρ ρ wheree∈E,n,m∈N,andA∈E(Revuz,1984). dρ Thenit is a theorem (Revuz, 1984) that, if Q is quasi-compact =cis(2π δ −2π n )U (45) d d ρ,δ (this is true when the state space is finite, as here), the asymptotic δ=1 ρ,k ρ,k dynamicsoftheMarkovchaintakesonacyclicalcharacter: where dρ,k = dρ/k. This is identical in form to the wavefunction • Thereareafinitenumberofinvarianteventsorabsorbingsets: of the free particle (Allday, 2009, §7.2.3): once the chain lands in any of these, it stays there forever. And  the union of these events exhausts the state space E.Wewill ψ(x,t) = A cis(2π x − 2π t )|x (46) index these events with the letter ρ. x λ T www.frontiersin.org June 2014 | Volume 5 | Article 577 | 13 Hoffman and Prakash Objects of consciousness ThisleadsustoidentifyA 1, Uρ,δ |x, δ x, n t,and OBJECTIONSANDREPLIES dρ,k λ = T.Thenthemomentumoftheparticleisp = h/dρ,k Here we summarize helpful feedback from readers of earlier and its energy is E = hc/dρ,k,whereh is Planck’s constant and c drafts, in the form of objections and replies. is the speed of light. Thus, we are identifying (1) a wavefunction ψ of the free par- (1) Your definition of conscious agents could equally well-apply ticle with a harmonic function g of a space-time Markov chain to unconscious agents. Thus, your theory says nothing about of interacting conscious agents, (2) the position basis |x of the consciousness. particle with indicator functions Uρ,δ of asymptotic events of the agentdynamics,(3)thepositionindexxwiththeasymptoticstate Even if the definition could apply to unconscious agents, that index δ, (4) the time parameter t with the step parameter n,(5) would not preclude it from applying to consciousness, any more the wavelength λ and period T with the number of asymptotic than using the integers to count apples would preclude using events dρ,k in the asymptotic behavior of the agents, and (6) the themtocountoranges. momentum p and energy E as functions inversely proportional to dρ,k. Note that wavelength and period are identical here: in these (2) Howcanconsciousness be cast in a mathematical formalism units, the speed of the wave is 1. without losing something essential? This identification is for non-relativistic particles. For the rel- ativistic case we sketch a promising direction to explore, starting The mathematics does lose something essential, viz., conscious- with the dynamics of two conscious agents in an undirected join. ness itself. Similarly, mathematical models of weather also lose In this case, the state of the dynamics has six components: N1, somethingessential, viz., weather itself. A mathematical model of N , X , X , G , G .Weidentifythesewiththegeneratingvectors hurricanes won’t create rain, and a mathematical model of con- 2 1 2 1 2 of a geometric algebra (2, 4) (Doran and Lasenby, 2003). The sciousness won’t create consciousness. The math is not the terri- components N and N have positive signature, and the remain- tory. But, properly constructed, mathematicsrevealsthestructure 1 2 ing have negative signature. (2, 4) is the conformal geometric of the territory. algebra for a space-time with signature (1, 3), i.e., the Minkowski space of special relativity. The conformal group includes as a (3) WhydoyourepresentqualiabyaprobabilityspaceX? subgroupthePoincaregroupofspace-timetranslationsandrota- tions; but the full conformal group is needed for most massless Probability spaces can be used, of course, to represent a diverse relativistic theories, and appears in theories of supersymmetry range of content domains, from the outcomes of coin-flips to the and supergravity. The Lie group SU(2, 2) is isomorphic to the long-term behavior of equity markets. But this does not preclude rotor group of (2, 4), which provides a connection to the usingprobabilityspacestorepresentqualia.Aprobabilityspaceis twistor program of Roger Penrose for quantum gravity (Penrose, notitself identical to qualia (or to coin flips or equity markets). To 2004). proposethatwerepresentthepossiblequaliaofaconsciousagent Thus,theideaistoconstructageometricalgebra  (2,4)from by a probability space is to propose that qualia convey informa- the dynamics of two conscious agents, and from this to con- tion, since probability and information are (as Shannon showed) struct space-time and massless particles. Each time we take an transforms of each other. It is also to propose that qualia need undirected join of two conscious agents, we get a new geometric not,ingeneral,exhibitotherstructures,suchasmetricsordimen- algebra (2, 4) with new basis vectors as described above. Thus, sions. Nowcertainqualiaspaces,suchasthespaceofphenomenal weget a nested hierarchy of such geometric algebras from which colors, do exhibit metrical and dimensional properties. These wecanbuildspace-timefromthePlanckscaleuptomacroscopic properties are not precluded. They are allowed but not required. scales. The metric would arise from the channel capacity of the All that is required is that we can meaningfully talk about the joined agents. information content of qualia. Themassivecaseinvolvessymmetrybreaking,andapromising The qualia X of a conscious agent C are private, in the sense direction to explore here involves hierarchies of stopping times that no other conscious agent C can directly experience X. in the Markovian dynamics of conscious agents. The idea is that i one system of conscious agents might infrequently interact with Instead each Ci experiences its own qualia Xi. Thus, the qualia another system, an interaction that can be modeled using stop- Xare“inside”theconsciousagentC.The“outside”forC isW,or ping times. Such interactions can create new conscious agents, moreprecisely, W-C. using the combination theorems presented earlier, whose “time” is moving more slowly than that of the original systems of agents (4) A conscious agent should have free will. Where is this mod- involved in the combination. This hierarchy of stopping times eled in your definition? proceeds all the way up to the slow times of our own con- sciousexperiencesashumanobservers(roughly1040 timesslower The kernel D represents the free will choices of the conscious than the Planck time). The hierarchy of stopping times is linked agentC.ForanyparticularqualexinX,thekernelDgivesaprob- to a hierarchy of combinations of conscious agents, leading up ability measure on possible actions in the set G that the conscious to the highest level of conscious agents that constitute us, and agent might choose to perform. We take this probability measure beyond. to represent the free will choice of the conscious agent. Thus, we Frontiers in Psychology | Perception Science June 2014 | Volume 5 | Article 577 | 14 Hoffman and Prakash Objects of consciousness interpret the probabilities as objective probabilities, i.e., as rep- defeater for belief in natural selection. See Alvin Plantinga’s resenting a true nondeterminism in nature. We are inclined to argumentonthis(Plantinga,2002). interpret all the other probabilities as subjective, i.e., as reflections of ignorance and degrees of belief. Evolutionarygamesandgeneticalgorithmsdemonstratethatnat- ural selection does not, in general, favor true perceptions. But (5) Aconsciousagentshouldhavegoalsandgoal-directedbehav- this entails nothing about the reliability of our cognitive facul- iors. Where are these modeled in your definition? ties more generally. Indeed, selection pressures might favor more accurate logic and mathematics, since these are critical for the Goals and goal-directed behaviors are not in the definition of proper estimation of the fitness consequences of actions. The conscious agent. This allows the possibility of goal-free conscious selection pressures on each cognitive faculty must be studied agents, and reflects the view that goals are not a definitional prop- individually before conclusions about reliability are drawn. erty of consciousness. However, since one can construct universal Turing machines from dynamical systems of conscious agents, (10) The undirected join of conscious agents doesn’t really solve it follows that one can create systems of conscious agents that the problem of combining subjects, because the decision exhibit goal-directed behaviors. Goals experienced as conscious kernel of the combination is just the product of the deci- desires can be represented as elements of a qualia space X. sion kernels of the two conscious agents that are combined. This product only models two separate agents making sep- (6) Your theory doesn’t reject object permanence, because con- arate decisions, not two subjects combined into a single scious agents are the “objects” that give rise to our percep- decision-making subject. tions of size and shape, and those agents are permanent even whenwe’renotlooking. It’s true that the decision kernel, D, of the combination starts out as a product, indicating independent decisions. But as the Conscious realism proposes that conscious agents are there even conscious agents in the combination continue to interact, the when one is not looking, and thus rejects solipsism. But it also decisions become less and less independent. In the asymptotic rejects object permanence, viz., the doctrine that 3D space and n physical objects exist when they are not perceived. To claim that limit, the decision kernel D as n →∞of the combination cannot, in general, be written as a product. In this limit, the conscious agents exist unperceived differs from the claim that combinationnowhasasingleunifieddecisionkernel,notdecom- unconscious objects and space-time exist unperceived. posable as a product of the original decision kernels. And yet the two conscious agents in the combination still retain their iden- (7) If our perceptions of space-time and objects don’t resem- tities. Thus, the undirected join models a combination process ble objective reality, if they’re just a species-specific interface, which starts off as little more than the product of the constituent then science is not possible. agents but ends up with those agents fully entangled to form a newconsciousagentwithagenuinelynewandintegrateddecision The interface theory of perception poses no special problems for kernel. science. The normal process of creating theories and testing pre- dictions continues as always. A particularly simple theory, viz., (11) IfIhaveanobjectionitisthattheauthors’proposalismaybe that our perceptions resemble reality, happens to be false. Fine. notcrazyenough.Iamwiththem100%whentheycompare Wecandevelop other theories of perception and reality, and test neuronstoiconsonacomputerscreen.But(ifIhaveunder- them. Science always faces the problem, well-known to philoso- stood them correctly) they then go on to attribute absolute phers of science, that no collection of data uniquely determines existencetoconsciousness.Myowninclinationistopropose the correct theory. But that makes science a creative and engaging that consciousness is also just an icon on a computer screen. process. Conscious realism is the hypothesis that the objective world W (8) Your proposal that consciousness, rather than physics, is consists of conscious agents. The theory of conscious agents fundamentalplaces consciousness outside of science. is a mathematical theory of consciousness that quantifies over qualia that it assumes really exist. So this theory does assume the Absolutely not. The onus is on us to provide a mathematically existence of consciousness. rigorous theory of consciousness, to show how current physics However,itdoesnotassumeincorrigibilityofqualia(tobelieve falls out as a special case, and to make new testable predictions one has a quale is to have one) or infallibility about the con- beyondthoseofcurrentphysics.Todismissthephysicalisttheory tents of one’s consciousness. Psychophysical studies provide clear that space-time and objects are fundamental is not to reject the evidence against incorrigibility and infallibility [see, e.g., the lit- methodologyofscience. It is just to dismiss a specific theory that erature on change blindness (Simons and Rensink, 2005)]. Nor is false. does it assume that the mathematics of conscious agents is itself identical to consciousness; a theory is just a theory. (9) You argue that natural selection does not favor true per- One might try to interpret the theory of conscious agents as ceptions. But this entails that the reliability of our cognitive describing a psychophysical monism, in which matter and con- faculties is low or inscrutable, and therefore constitutes a sciousness are two aspects of a more abstract reality. Such an www.frontiersin.org June 2014 | Volume 5 | Article 577 | 15 Hoffman and Prakash Objects of consciousness interpretation,ifpossible,mightstillbeunpalatabletomostphys- that triggers the organism to perceive a lion, but whatever that icalists since it entails that dynamical physical properties, such as something is, it almost surely doesn’t resemble a lion. A lion is position, momentum and spin, have definite values only when simply a species-specific adaptive symbol, not an insight into they are observed. objective reality. (12) One problem with section Evolution and Perception is that (13) In section Evolution and Perception, the authors’ argument theauthorsneverdefineeithertheirnotionofTruth,ortheir seemstobe:Argument1:(1)Naturalselectionfavorsfitness notionofPerception.Theyseemtobelievethatifyoustartle inperceptualsystems.(2)Fitnessisincompatiblewithtruth. at anysoundofrustlingleaves(asasortofsensitivepredator (3) Therefore, natural selection favors perceptions that do avoidance system), then when you run from a real predator, not see truth in whole or in part. you are not in any way in touch with the truth. But this is With some minor tweaking, Argument 1 can be made incorrect. valid. But premise 2 is completely implausible. If a tiger is charging you with lunch on his mind, truth works in the Forsakeofbrevity,weomittedourdefinitionsoftruthandper- service of fitness. (The authors’ treatment here raises the ception from this paper. But they are defined precisely in papers question of why we have perceptual systems at all and not thatstudytheevolutionofperceptioninMonteCarlosimulations just kaleidoscope eyes. They never address this.) of evolutionary games and genetic algorithms (Mark et al., 2010; The authors would object that premise 2 is too strong. Hoffmanetal.,2013;Marion,2013;Mark,2013). They don’t subscribe to premise 2, they would say. They Briefly, we define a perceptual strategy as a measurable func- wouldperhapsholdoutforArgument2: tion (or, more generally, a Markovian kernel) p : W → X,where Argument 2: (1) Natural selection favors fitness in per- Wis a measurable space denoting the objective world and X is ceptual systems. (2) Fitness need not always coincide with a measurable space denoting an organism’s possible perceptions. truth. (3) Therefore, natural selection favors perceptions If X = W and p is an isomorphism that preserves all structures that do not see truth in whole or in part. on W,thenp is a naïve realist perceptual strategy. If X ⊂ W and But Argument 2 is not valid and not tweakable into a p is structure preserving on this subset, then p is a strong critical valid argument. The conclusion is a lot stronger than the realist strategy. If X need not be a subset of W and p is structure premises. preserving, then p is a weak critical realist strategy. If X need not Worse, any weaker premise doesn’t give the authors their be a subset of W and p need not be structure preserving, then p needed/wantedradicalthesis: Perception is not about truth, is an interface strategy. These strategies form a nested hierarchy: it is about havingkids.Whichtheyinsistmustbeinterpreted naïve realist strategies are a subset of strong critical realist, which as Perception is never about truth, but about having kids. are a subset of weak critical realist, which are a subset of interface. Butthisinterpretationisobviouslyfalse.Foronething,ifan Naïve realist strategies see all and only the truth. Strong criti- ancientancestorofours(callher,Ug)issuccessfulinhaving cal realist strategies see some, but in general not all, of the truth. kids, she needs to know the truth: that she has kids! Why? Weakcriticalrealist strategies in general see none of the truth, but Because Ug needs to take care of them! the relationships among their perceptions genuinely reflect true relationships in the structure of the objective world W.Interface Wedonotuseeitherargument.WesimplyuseMonteCarlosim- strategies in general see none of the truth, and none of the true ulations of evolutionary games and genetic algorithms to study relationships in the structure of W. Thus, our mathematical for- the evolution of perceptual strategies (as discussed in Objection mulationofperceptualstrategies allows a nuanced exploration of 12). We find, empirically, that strategies tuned to truth almost the role of truth in perception. always go extinct, or never even arise, in hundreds of thousands We let these perceptual strategies compete in hundreds of of randomly chosen worlds. thousands of evolutionary games in hundreds of thousands of The key to understanding this finding is the distinction randomlychosenworlds, and find that strategies which see some between fitness and truth. If W denotes the objective world (i.e., or all of the truth have a pathetic tendency to go extinct when the truth), O denotes an organism, S the state of that organ- competing against interface strategies that are tuned to fitness ism, and A an action of that organism, then one can describe rather than truth. The various truth strategies don’t even get a fitness as a function f : W × O × S × A → R.Inotherwords, chancetocompeteinthegeneticalgorithms,becausetheyarenot fitness depends not only on the objective truth W,butalso fit enough even to get on the playing field. on the organism, its state and the action. Thus, fitness and Thus, natural selection favors interface strategies that are truth are quite distinct. Only if the fitness function happens tuned to fitness, rather than truth. If an organism with an to be a monotonic function of some structure in W, i.e., so interface perceptual strategy perceives, say, a predatory lion, then that truth and fitness happen to coincide, will natural selection it really does perceive a lion in the same sense that someone allow a truth strategy to survive. In the generic case, where truth having a headache really does have a headache. However, this and fitness diverge, natural selection sends truth strategies to does not entail that the objective world, W, contains an observer- extinction. independentlion,anymorethanabluerectangulariconona Tophrasethisasanargumentofthekindgivenintheobjection computer desktop entails that there is a blue rectangular file we would have Argument 3: (1) Natural selection favors fitness inthecomputer.ThereissomethingintheobjectiveworldW in perceptual systems. (2) Truth generically diverges from fitness. Frontiers in Psychology | Perception Science June 2014 | Volume 5 | Article 577 | 16 Hoffman and Prakash Objects of consciousness (3)Therefore,naturalselectiongenerically favorsperceptionsthat properties when they are not observed. These predictions are in diverge from the truth. factcompatiblewithquantumtheory,andarepartofthestandard The word generically here is a technical term. Some property interpretation of quantum theory. holdsgenerically if it holds everywhere except on a set of measure SupposeEisfalse.Thenourperceptionsdidnotevolvebynat- zero. So, for instance, the cartesian coordinates (x, y)ofapoint ural selection. At present, science has no other theory on offer in the plane generically have a non-zero y coordinate. Here we are for the development of our perceptual systems. So, in this case, assuming an unbiased (i.e., uniform) measure on the plane, in science cannot at present make an informed prediction about whichthemeasureofasetisproportionaltoitsarea.Sincetheset whether our perceptions are true or not. But this is not a logical of points with a zero y coordinate is the x-axis line, and since lines contradiction. have no area, it follows that generically a point in the plane has So there is no liar paradox. And there’d better not be. Science a non-zero y coordinate. Note, however, that there are infinitely cannot be precluded apriorifrom questioning the veridicality manypoints with a zero y coordinate, even though this property of the perceptions of H. sapiens, any more than it can be pre- is non-generic. cluded from questioning the veridicality of the perceptions of Soourargumentisthat,foranappropriateunbiasedmeasure, other species. David Marr, for instance, argues that “... it is fitness functions generically diverge from truth, and thus natural extremely unlikely that the fly has any explicit representation of selection generically favors perceptions that diverge from truth. thevisualworldaroundhim—notrueconceptionofasurface,for This does not entail the stronger conclusion that natural selec- example, but just a few triggers and some specifically fly-centered tion never favors truth. That conclusion is indeed stronger than parameters ...” and that the fly’s perceptual information “...is our premises and stronger than required for the interface theory all very subjective” (Marr, 1982, p. 34). Science has no trouble of perception. Perhaps H. sapiens is lucky and certain aspects of investigating the veridicality of the perceptions of other species our perceptual evolution has been shaped by a non-generic fit- and concluding, e.g., in the case of the fly, that they fail to be nessfunctionthatdoesnotdivergefromtruth.Inthiscasesome veridical. Its methods apply equally well to evaluating the veridi- aspects of our perceptions might be shaped to accurately report cality of the perceptions of H. sapiens (Koenderink et al., 2010; the truth, in the same sense that your lottery ticket might be the Koenderink, 2011b, 2013). winner. But the smart money would bet long odds against it. That’s what non-generic means. (15) Section The Interface Theory of Perception fares no better. The account of the interface theory about Ug’s perception of Here they say Reality, we learned, departed in important herkidsisthesameastheaccountinObjection12forthepercep- respects from some of our perceptions. This is true. But it tion of lions. There are no public physical objects. Lions and kids is true because other perceptions of ours won out because arenomorepublicandobserverindependentthanareheadaches. they were true. E.g., the Earth is not a flat disk or plane. Lions and kids (and space-time itself) are useful species-specific Otherperceptionsindeedwonout—notbecausetheyaretruebut perceptions that have been shaped by natural selection not to becausetheyareadaptiveinawiderrangeofcontexts.Flatearthis report the truth but simply to guide adaptive behavior. We must adequateformanyeverydayactivities,butifonewantstocircum- take them seriously, but it is a logical error to conclude that we navigate the earth by boat then a spherical earth is more adaptive. musttakethemliterally. If one wants to control satellites in orbit or navigate strategic sub- Although our eyes do not report the truth, they are not marines then a spherical earth is inadequate and a more complex kaleidoscope eyes because they do report what matters: fitness. modelisrequired. (14) We see then that the authors are caught in version of the Perceived 3D space is simply a species-specific perceptual Liar: Science shows that perception never cares about truth. interface, not an insight into objective reality; we have argued Let this statement be L. L is derived via perception. So is L for this on evolutionary grounds, and researchers in embod- (together with its perceptual base) true or false? If it is one, ied cognition have arrived at a similar conclusion (Laflaquiere then it is the other. Contradiction. et al., 2013; Terekhov and O’Regan, 2013). Space as modeled in physics extends perceived space via the action of groups, e.g., the This is not our argument. We claim that perception evolved by Euclideangroup,Poincaregroup,orarbitrarydifferentiablecoor- naturalselection. Call this statement E.NowE isindeedinformed dinate transformations (Singh and Hoffman, 2013). Any objects by the results of experiments, and thus by our perceptions. We embedded in space, including earth and its 3D shape, are thus observe, from evolutionary game theory, that one mathematical descriptions in a species-specific vocabulary, not insights into predictionofEisthatnaturalselectiongenericallydrivestrueper- objective reality. ceptionstoextinctionwhentheycompetewithperceptionstuned (16) Also, I don’t understand their interface theory of percep- to fitness. tion. I not only take my icons seriously, but literally: they Suppose E is true. Then our perceptions evolved by natural are icons. I’m prepared to wager the farm on this: they are selection. This logically entails that our perceptions are generi- indeed icons. cally about fitness rather than truth. Is this a contradiction? Not at all. It is a scientific hypothesis that makes testable predic- We would agree that icons are indeed icons. When I open my tions. For instance, it predicts that (1) physical objects have no eyes and see a red apple, that red apple is indeed an icon of my causalpowersand(2)physicalobjectshavenodynamicalphysical perceptual interface. When I close my eyes that icon disappears; I www.frontiersin.org June 2014 | Volume 5 | Article 577 | 17 Hoffman and Prakash Objects of consciousness see just a mottled gray field. Now some physicalists would like to their model, they discuss this very objection (objection 10). claim that even when my eyes are closed, an objective red apple Unfortunately, their resolution to this objection is mere still exists, indeed the very red apple that triggered my perceptual handwaving:Butastheconsciousagentsinthecombination interface to have a red apple icon. It is this claim that is generically continuetointeract,thedecisionsbecomelessandlessinde- incorrect, if our perceptual systems evolved by natural selection. pendent.Thisismerewishfulthinking.Theauthorshaveno reasontobelievethislessandlessbusinessandthey’vegiven (17) The authors make too much of the Humean idea that the the reader no reason to think this either. In fact, if this less appearance of cause and effect is simply a useful fiction andlessbusinessweretrue,theirmodelwouldn’trequirethe (section The Interface Theory of Perception). They like all Cartesianproductinthefirstplace.Frankly,thisobjection mammalsandperhapsmostanimalscannotfailtoseecau- and their failure to handle it guts their model. In this same sation in the deepest aspects of their lives. The authors paragraph, in the next couple of sentences, the authors just believe in causation as deeply as anyone in the world. Why? assert (using proof by blatant assertion) that in some unde- Becauseweareallhardwiredtoseecausation.Andwhileitis finedlimit,atruenewconsciousentityemerges.Thismakes true that causation goes away at the quantum level, we have the complex presentation of their model otiose. Why not no reason to believe that it doesn’t really exist at the macro just write a haiku asserting that the combination problem level. These two levels don’t live well together, but pretend- isnotaproblem? ing that there’s no such thing as causation is silly, at least it is silly without a lot of argument. Even Hume admitted that The limit we speak of (for the emergence of a new combined causation was perfectly real when he had left his study and conscious agent) is the asymptotic limit. Asymptotic behavior wenttoplaybackgammonwithhisfriends. is a precise technical concept in the theory of Markov chains (see, e.g., Revuz, 1984, chapter 6). We have given, in sections There is indeed good evidence that belief in causation is either First Example of Asymptotic Behavior and Second Example of innate or learned early in life (Carey, 2009; Keil, 2011). And of Asymptotic Behavior, concrete examples of undirected joins for course we, the authors, are no exception; we, no less than oth- which,asymptotically,anewcombinedconsciousagentiscreated ers, have a psychological penchant toward causal reasoning about that is not just a Cartesian product of the original agents. the physical world. But, equally, we no less than others have a Intuitively, the reason that the undirected combination of two psychological penchant toward assuming that space, time and agents creates a new agent that is not just a product is that there is physical objects are not merely icons of a species-specific percep- feedback between the two agents (this is illustrated in Figure 2). tual interface, but are instead real insights into the true nature Thus, the decisions and actions of one agent influence those of ofobjectivereality.Sciencehasahabitofcorrectingourpen- the other. This influence is not fully felt in the first step of the chants, even those deeply held. Evolutionary games and genetic dynamics, but in the asymptotic limit of the dynamics it com- algorithms convinced us, against our deeply held convictions to pletely dominates, carving the state space of the dynamics into the contrary, that perceptions are, almost surely, interfaces not various absorbing sets with their own periodic behaviors, in a insights; they also convinced us that the appearance of causality fashion that is not reducible to a simple product of the original amongphysicalobjectsis a useful fiction. twoagents. Perceptual icons do, we propose, inform thebehaviorofthe The degree to which the new conscious agent is not reducible perceiver, and in this sense might be claimed to have causal pow- to a simple product of the original agents can be precisely quan- ers. This sense of causality, however, differs from that typically tifiedusing,forinstance,themeasureofintegrated information attributed to physical objects. developed by Tononi and others (Tononi and Edelman, 1998; Hume’s ideas on causation had little influence on us, in part TononiandSpoorns,2003;Tononi,2008;TononiandKoch,2008; because exegesis of his ideas is controversial, including projec- Barrett and Seth, 2011). It is straightforward to compute, for tivist, reductionist and realist interpretations (Garrett, 2009). instance, that the new agent in Second Example of Asymptotic Our views on causality are consistent with interpretations of Behavior has 2 bits of integrated information, i.e., of new infor- quantum theory that abandon microphysical causality, such as mation that is not reducible to that of the two original agents. the Copenhagen,quantumBayesianand(arguably)many-worlds Thus, there is a precise and quantifiable sense in which the undi- interpretations, (Allday, 2009; Fuchs, 2010; Tegmark, 2014). The rected combination of conscious agents creates a new conscious burdenofproofissurelyononewhowouldabandonmicrophys- agent with its own new information. ical causation but still cling to macrophysical causation. Weshould note, however, that our use here of Tononi’s mea- sure of integrated information does not imply that we endorse (18) Their treatment of the combination problem is worth read- his theory of consciousness. Tononi is a reductive functionalist, ing. Thereishoweveraverylargeproblemwiththeirmodel: proposing that consciousness is identical to integrated infor- It relies on the Cartesian product of X and X (this is right mation and that qualia are identical to specific informational 1 2 after Conjecture 3). The Cartesian product is not conducive relationships (Tononi, 2008). Consistent with this view he asserts, to real combination (this problem is all over mathematics, for instance, that spectrum inversion is impossible (Tononi, 2008, by the way—mathematicians don’t care about it because footnote 8). However, a recent theorem proves that all reductive they only care about high level abstractions). In section functionalist theories of consciousness are false (Hoffman, 2006). Objections and Replies, where they discuss objections to A fortiori, Tononi’s theory is false. His measure of integrated Frontiers in Psychology | Perception Science June 2014 | Volume 5 | Article 577 | 18 Hoffman and Prakash Objects of consciousness information and his analyses of informational relationships are evolutionarytheorytobetrue,butinsteadtobea“boundarycon- valuable. But his next move, of identifying consciousness with dition” on the new evolutionary theory. Standard evolutionary integrated information, is provably false. He could fix this by theory is simply how the new evolutionary theory appears when makingtheweakerclaimthatconsciousnessiscausedbyorresults it is shoehorned into the perceptual framework that H. sapiens from integrated information. His theory would no longer be nec- happenstohave. essarily false. But then he would need to offer a scientific theory The process we are describing here is standard procedure in about how integrated information causes or gives rise to con- science. Wealwaysuseourcurrentbesttheoryasaladdertoabet- sciousness. No such theory is currently on offer and, we suspect, ter theory, whereupon we can, if necessary, kick away the ladder. nosuchtheoryispossible. However, we needn’t take our best theory to be true. It’s simply the best ladder we have to our next theory. We are here adopting (19) The paper explicitly commits a fallacy: it privileges the a philosophy of instrumentalism in regards to scientific theories. authors’ take on reality while denying that there is any such The development of a new generalized theory of evolution is thing as reality. For example: The authors say “There are no not just an abstract possibility, but is in fact one of our current public physical objects. Lions and kids are no more pub- projects. We are investigating the possibility of keeping the core lic and observer independent than are headaches. Lions and ideas of standard evolutionary theory that are sometimes referred kids (and space-time itself) are useful species-specific per- to as “Universal Darwinism,” ideas that include abstract notions ceptions that have been shaped by natural selection not to of variation, selection and retention. We plan to apply Universal report the truth but simply to guide adaptive behavior. We Darwinismtointeractingsystemsofconsciousagentstomodel musttakethemseriously,butitisalogicalerrortoconclude their evolution. that we must take them literally.” The new limited resource that is the source of competition Natural selection, which the authors clearly think is the would be information, which is the measure we use to quantify truth, is just as susceptible to their arguments as headaches the channel capacity of conscious agents. This is a promising or truth itself. So by their own reasoning, natural selection direction, since information is equivalent to energy, and informa- is not true; neither are their computer programs/models. tion can be converted into energy (Toyabe et al., 2010). Limited So the reader doesn’t have to take natural selection or their energy resources, e.g., in the form of food, are a clear source of models either seriously or literally. So their paper is now competition in standard evolutionary theory. exposedasself-refuting. Thenewevolutionarytheorythatweconstructshouldexplain why the standard evolutionary theory was a good ladder to the If we indeed proposed a “take on reality while denying that there newtheory,andwhywearejustifiedinkickingawaythatladder. is any such thing as reality,” we would of course be self-refuting. However, we do not deny that there is any such thing as real- (20) The authors say, “In short, natural selection does not favor ity. We cheerfully admit that there is a reality. We simply inquire perceptual systems that see the truth in whole or in part. into the relationship between reality and the perceptions of a Instead, it favors perceptions that are fast, cheap, and tai- particular species, H. sapiens. Such inquiry is surely within the lored to guide behaviors needed to survive and reproduce. purview of science. Moreover all currently accepted theories in Perception is not about truth, it’s about having kids.” This is science, including evolutionary theory, are appropriate tools for afalsedichotomy. such inquiry. Wefindthatevolutionarytheoryentails a low probability that The distinction between truth and fitness, between truth and our perceptions are veridical, and thus a high probability that having more kids, is not a false dichotomy to evolutionary biol- reality is not isomorphictoourperceptions,e.g.,ofspacetimeand ogists. It is a distinction that is central to their theory. The same objects. This promptsustoproposeanewtheoryofreality,which objectively true world can have an infinite variety of different fit- we have done by defining conscious agents and proposing con- ness functions, corresponding to the variety of organisms, states scious realism, viz., that reality consists of interacting conscious and actions. A steak that conveys substantial fitness benefits to agents. a hungry lion conveys no benefits to a cow. Each distinct fitness This proposal invites us to revisit evolutionary theory itself. function drives natural selection in a different direction. The standard formulation of evolutionary theory, i.e., the neo- Darwiniansynthesis,iscouchedintermsofspacetimeandobjects (21) In response to the claim that “Your definition of conscious (suchasorganismsandgenes),whichwenowtaketobeaspecies- agentscouldequallywell-applytounconsciousagents;thus, specific perceptual representation, not an insight into reality. But your theory says nothing about consciousness.” the authors we are not forced into self-refutation at this point. It is open to reply that “Even if the definition could apply to unconscious us to formulate a new generalized theory of evolution that oper- agents,thatwouldnotprecludeitfromapplyingtocon- ates on what we now take to be reality, viz., interacting systems of sciousness, any morethanusingtheintegerstocountapples conscious agents. wouldprecludeusingthemtocountoranges.” Akeyconstraintonournewevolutionarytheoryisthis:When However, the very fact that the integers can be used to the new evolutionary theory is projected onto the spacetime count apples and oranges and peace treaties, etc., is pre- perceptual interface of H. sapiens wemustgetbackthestan- cisely WHY the integers are not a theory of either apples or dard evolutionary theory. Thus, we do not take the standard orangesorpeacetreaties,etc.Thesameistrueofdefinitions. www.frontiersin.org June 2014 | Volume 5 | Article 577 | 19 Hoffman and Prakash Objects of consciousness If my definition of integer applies equally well to the com- is, of course, a tall order. We have taken some first steps by plex numbers as well as to the integers, then I do not have (1) proposing the formalism of conscious agents, (2) using that a definition of integers. Instead I have a definition of com- formalism to find solutions to the combination problem of con- plex numbers. So their definition is useless; all they’ve done sciousness, and (3) sketching how the asymptotic dynamics of is define an agent. Consciousness is not present, except conscious agents might lead to particles and space-time itself. accidentally. Much work remains to flesh out this account. But if it succeeds, H. sapiens might just replace object permanence with objects of Theintegers are not considered a theory of peace treaties because consciousness. they don’t have the appropriate mathematical structure to model peace treaties—not because they can be used to count apples and ACKNOWLEDGMENTS peace treaties. For helpful discussions and comments on previous drafts we If one has a mathematical structure that is rich enough to pro- thank Marcus Appleby, Wolfgang Baer, Deepak Chopra, Federico vide a useful theory of some subject, this does not entail that Faggin, Pete Foley, Stuart Hameroff, David Hoffman, Menas the same structure cannot be a useful theory of a different sub- Kafatos, Joachim Keppler, Brian Marion, Justin Mark, Jeanric ject. The group SU(3), for instance, models an exact symmetry of Meller, Julia Mossbridge, Darren Peshek, Manish Singh, Kyle quark colors and an approximate symmetry of flavors. No physi- Stephens, and an anonymous reviewer. cist would insist that because SU(3) is a useful theory of quark coloritcannotalsobeausefultheoryofflavor.AgivenMarkovian REFERENCES kernel P can modelastochasticdynamics,butalsoacommunica- Allday, J. (2009). Quantum Reality: Theory and Philosophy. Boca Raton, FL: CRC tion channel. The fact that P applies to both does not entail that Press. it’s a theory of neither. Ashby, F. G. (2000). A stochastic version of general recognition theory. J. Math. Similarly, a measurable space X might properly represent the Psychol. 44, 310–329. doi: 10.1006/jmps.1998.1249 conscious color experiences of a human observer, and also the Athreya, K. B., and Lahiri, S. N. (2006). Measure Theory and Probability Theory. NewYork,NY:Springer. unconscious color judgments of a robotic vision system designed Baillargeon, R., and DeVos, J. (1991). Object permanence in young infants: further to mimic that observer. No vision scientist would insist that evidence. Child Dev. 62, 1227–1246. doi: 10.2307/1130803 because X properly represents the unconscious color judgments Bargh, J. A., and Morsella, E. (2008). The unconscious mind. Perspect. Psychol. Sci. of the robotic vision system that therefore X cannot model the 3, 73–79. doi: 10.1111/j.1745-6916.2008.00064.x conscious color experiences of the human observer. Barrett, A. B., and Seth, A. K. (2011). Practical measures of integrated information for time-series data. PLoS Comput. Biol. 7:1. doi: 10.1371/journal.pcbi.1001052 Scientists do not reject a model because it has multiple Bayne, T., and Levy, N. (2006). “The feeling of doing: deconstructing the phe- domainsofusefulapplication.Theydorejectamodelifitsstruc- nomenology of agency,” in Disorders of Volition, eds N. Sebanz and W. Prinz ture is inappropriate to the domain, or if it makes predictions (Cambridge,MA:MITPress),49–68. that are empirically false. These are the appropriate grounds to Blackmore, S. (1999). The Meme Machine. Oxford: Oxford University Press. judge whether the formalism of conscious agents provides an Blamauer, M. (2011). Is the panpsychist better off as an idealist? Some Leibnizian remarksonconsciousnessandcomposition.Eidos15,48–75. adequate model for consciousness. The possibility that this for- Boswell, J., (1986). The Life of Samuel Johnson, ed H. Christopher (New York, NY: malism applies well to other domains does not entail that it PenguinClassics), 122. cannot apply to consciousness. Bower, T. G. R. (1974). Development in Infancy. San Francisco, CA: Freeman. Byrne, A., and Hilbert, D. R. (2003). Color realism and color science. Behav. Brain CONCLUSION Sci. 26, 3–26. doi: 10.1017/S0140525X03000013 Carey, S. (2009). The Origin of Concepts. New York, NY: Oxford University Press. Belief in object permanence commences at 3 months of age and doi: 10.1093/acprof:oso/9780195367638.001.0001 continues for a lifetime. It inclines us to assume that objects exist Ceccherini-Silberstein, T., and Coornaert, M. (2010). Cellular Automata and without subjects to perceive them, and therefore that an account Groups. Berlin: Springer. doi: 10.1007/978-3-642-14034-1 Chartrand, G., and Ping, Z. (2012). A First Course in Graph Theory. Mineola, NY: of objects can be given without a prior account of subjects. Dover. However, studies with evolutionary games and genetic algo- Coleman, S. (2014). The real combination problem: panpsychism, micro-subjects, rithmsindicatethatselectiondoesnotfavorveridicalperceptions, andemergence.Erkenntnis79,19–44.doi:10.1007/s10670-013-9431-x and that therefore the objects of our perceptual experiences are Cook, M. (2004). Universality in elementary cellular automata. Complex Syst. better understood as icons of a species-specific interface rather 15, 1–40. Cover, T. M., and Thomas, J. A. (2006). Elements of Information Theory. New York, than as an insight into the objective structure of reality. This NY:Wiley. requires a fundamental reformulation of the theoretical frame- Coyne,J. A. (2009). Why Evolution is True. New York, NY: Viking. workforunderstandingobjects. Crick, F. (1994). The Astonishing Hypothesis: the Scientific Search for the Soul. New This reformulation cannot assume that physical objects have York, NY: Scribners. genuine causal powers, nor that space-time is fundamental, since Dawkins, R. (2009). The Greatest Show on Earth: the Evidence for Evolution.New York, NY: Free Press. objects and space-time are simply species-specific perceptual Dennett, D. (1995). Darwin’s Dangerous Idea. New York, NY: Touchstone. adaptions. Doran, C., and Lasenby, A. (2003). Geometric Algebra for Physicists. New York, NY: If we assume that conscious subjects, rather than unconscious CambridgeUniversityPress. doi: 10.1017/CBO9780511807497 objects,arefundamental,thenwemustgiveamathematicallypre- Edelman, G. M. (2004). Wider than the Sky: the Phenomenal Gift of Consciousness. cise theory of such subjects, and show how objects, and indeed NewHaven,CT:YaleUniversityPress. Feldman, J. (2013). Tuning your priors to the world. Top. Cogn. Sci. 5, 13–34. doi: all physics, emerges from the theory of conscious subjects. This 10.1111/tops.12003 Frontiers in Psychology | Perception Science June 2014 | Volume 5 | Article 577 | 20 Hoffman and Prakash Objects of consciousness Fodor,J.,andPiattelli-Palmarini,M.(2010).WhatDarwingotWrong,Farrar,Straus space,” in IEEE Conference on Intelligent Robots and Systems (Tokyo), andGiroux.NewYork,NY:Farrar,StrausandGiroux. 1230–1235. Fuchs, C. (2010). QBism, the Perimeter of Quantum Bayesianism. Available online Lieberman, E., Hauert, C., and Nowak, M. A. (2005). Evolutionary dynamics on at: arXiv:1003.5209v51. graphs. Nature 433, 312–316. doi: 10.1038/nature03204 Futuyma, D. J. (2010). Two critics without a clue. Science 328, 692–693. doi: Locke, J. (1690). An Essay Concerning Human Understanding. London: Tho Basset. 10.1126/science.1189090 Maddox, W. T. S., and Ashby, F. G. (1996). Perceptual separability, decisional Garrett, D. (2009). “Hume,” in The Oxford Handbook of Causation, eds H. Beebee, separability, and the identification-speeded classification relationship. J. Exp. C. Hitchcock, and P. Menzies (New York, NY: Oxford University Press), 53–71. Psychol. 22, 795–817. Geisler, W. S., and Diehl, R. L. (2003). A Bayesian approach to the evo- Marion, B. B. (2013). The Impact of Utility on the Evolution of Perceptions.Ph.D. lution of perceptual and cognitive systems. Cogn. Sci. 27, 379–402. doi: Dissertation, University of California, Irvine, CA. 10.1207/s15516709cog2703_3 Mark,J.T.(2013).EvolutionaryPressuresonPerception:WhendoesNaturalSelection Geisler, W. W., and Diehl, R. L. (2002). Bayesian natural selection and the Favor Truth? Ph.D. Dissertation, University of California, Irvine, CA. evolution of perceptual systems. Philos. Trans. R. Soc. B 357, 419–448. doi: Mark,J.T.,Marion,B.B.,andHoffman,D.D.(2010).Naturalselectionandveridi- 10.1098/rstb.2001.1055 cal perceptions. J. Theor. Biol. 266, 504–515. doi: 10.1016/j.jtbi.2010.07.020 Gibson, J. J. (1979). The Ecological Approach to Visual Perception. Boston, MA: Marr, D. (1982). Vision: a Computational Investigation into the Human HoughtonMifflin. Representation and Processing of Visual Information. San Francisco, CA: Goff, P. (2009). Why panpsychismdoesn’thelpusexplainconsciousness.Dialectica Freeman. 63, 289–311. doi: 10.1111/j.1746-8361.2009.01196.x Mausfeld, R. (2002). “The physicalist trap in perception theory,” in Perception and Hameroff, S. R. (1998). Quantum computation in brain microtubules? The the Physical World: Psychological and Philosophical Issues in Perception,edsD. Penrose-Hameroff “Orch OR” model of consciousness”. Philos. Trans. R. Soc. HeyerandR.Mausfeld(NewYork,NY:JohnWiley&Sons),75–112. A356,1869–1896.doi:10.1098/rsta.1998.0254 McGinn,C.(1989).Canwesolvethemind-bodyproblem?Mind98,349–366.doi: Herken,R.(ed.).(1988).TheUniversalTuringMachine:aHalf-centurySurvey.New 10.1093/mind/XCVIII.391.349 York, NY: Springer. Mitchell, M. (1998). An Introduction to Genetic Algorithms. Cambridge, MA: Hoffman,D.D.(1998).VisualIntelligence:HowWeCreateWhatWeSee.NewYork, BradfordBooksMITPress. NY:W.W.Norton. Nagel, T. (ed.). (1979). “Panpsychism,” in Mortal Questions (New York, NY: Hoffman, D. D. (2006). The scrambling theorem: a simple proof of the log- CambridgeUniversityPress), 181–195. ical possibility of spectrum inversion. Conscious. Cogn. 15, 31–45. doi: Nahmias,E.,Morris,S.,Nadelhoffer,T.,andTurner,J.(2004).Thephenomenology 10.1016/j.concog.2005.06.002 of free will. J. Conscious. Stud. 11, 162–179. Hoffman, D. D. (2009). “The interface theory of perception,” in Object Nowak, M. (2006). Evolutionary Dynamics: Exploring the Equations of Life. Categorization: Computer and Human Vision Perspectives, eds S. Dickinson, M. Cambridge,MA:BelknapPressofHarvardUniversityPress. Tarr,A.Leonardis,andB.Schiele(NewYork,NY:CambridgeUniversityPress), Pais, A. (1979). Einstein and the quantum theory. Rev. Mod. Phys. 51, 863–914. 148–165. doi: 10.1017/CBO9780511635465.009 (Quoteonp.907).doi:10.1103/RevModPhys.51.863 Hoffman, D. D. (2011). “The construction of visual reality,” in Hallucinations: Palmer, S. E. (1999). Vision Science: Photons to Phenomenology.Cambridge,MA: Theory and Practice, eds J. Blom and I. Sommer (New York, NY: MITPress. Springer), 7–15. Penrose, R. (2004). The Road to Reality,NewYork,NY:Knopf. Hoffman,D.D.(2012).“Thesensorydesktop,”inThisWillMakeYouSmarter:New Piaget, J. (1954). The Construction of Reality in the Child. NewYork,NY:Basic.doi: Scientific Concepts to Improve Your Thinking,edJ.Brockman(NewYork,NY: 10.1037/11168-000 HarperPerennial), 135–138. Plantinga, A. (2002). Naturalism Defeated? Ithaca, NY: Cornell University Press. Hoffman, D. D. (2013). “Public objects and private qualia: the scope and lim- Revuz, D. (1984). Markov Chains, Amsterdam: North-Holland. its of psychophysics,” in The Wiley-Blackwell Handbook of Experimental Rutherford,E.(1911).Thescatteringofαandβ particlebymatterandthestructure Phenomenology, ed L. Albertazzi (New York, NY: Wiley-Blackwell), of the atom. Philos. Mag. 6, 21. 71–89. Seager, W. (1995). Consciousness, information, and panpsychism. J. Conscious. Hoffman, D. D., and Singh, M. (2012). Computational evolutionary perception. Stud. 2, 272–288. Perception 41, 1073–1091. doi: 10.1068/p7275 Simons, D. J., and Rensink, R. A. (2005). Change blindness: past, present, and Hoffman, D. D., Singh, M., and Mark, J. T. (2013). “Does natural selection favor future. Trends Cogn. Sci. 9, 16–23. doi: 10.1016/j.tics.2004.11.006 true perceptions,” in Proceedings of the SPIE (Bellingham, WA). Singh, M., and Hoffman, D. D. (2013). “Natural selection and shape percep- Hopcroft, J. E., Motwani, R., and Ullman, J. D. (2006). Introduction to Automata tion: shape as an effective code for fitness,” in Shape Perception in Human and Theory, Languages, and Computation, 3rd Edn. Upper Saddle River, NJ: Prentice Computer Vision: an Interdisciplinary Perspective, eds S. Dickinson and Z. Pizlo Hall. (NewYork,NY:Springer),171–185. James, W. (1890). The Principles of Psychology, Vols 1, 2. London: MacMillan. doi: Taylor, C. C. W. (1999). “The atomists,” in The Cambridge Companion to Early 10.1037/10538-000 Greek Philosophy, ed A. A. Long (New York, NY: Cambridge University Press), James,W.,(1890/2007).ThePrinciplesofPsychology,Vol.1,NewYork,NY:Cosimo. 181–204. doi: 10.1017/CCOL0521441226.009 Keil, F.C. (2011). Science starts early. Science 33, 1022–1023. doi: 10.1126/sci- Tegmark,M.(2014).OurMathematicalUniverse:MyQuestfortheUltimateNature ence.1195221 of Reality. New York, NY: Knopf. Knill, D., and Richards, W. A. (eds.). (1996). Perception as Bayesian Inference.New Terekhov,A.,andO’Regan,K.O.(2013).SpaceasanInventionofBiologicalSystems. York, NY: Cambridge University Press. doi: 10.1017/CBO9780511984037 Available online at: arxiv.org/abs/1308.2124. Koenderink,J.J.(2011a).“Visionandinformation,”inPerceptionBeyondInference: Tononi, G. (2004). An information integration theory of consciousness. BMC TheInformationContentofVisualProcesses,edsL.Albertazzi, G.Tonder,andD. Neurosci. 5:42. doi: 10.1186/1471-2202-5-42 Vishnawath(Cambridge,MA:MITPress),27–58. Tononi, G. (2008). Consciousness as integrated information: a provisional mani- Koenderink,J.J.(2011b).“Visionasauserinterface,”inProceedingsofSPIE,Human festo. Biol. Bull. 215, 216–242. doi: 10.2307/25470707 VisionandElectronicImagingXVI,Vol.7865(Bellingham,WA).doi:10.1117/12. Tononi, G., and Edelman, G. (1998). Consciousness and complexity. Science 282, 881671 1846–1851. doi: 10.1126/science.282.5395.1846 Koenderink, J. J. (2013). “World, environment, umwelt, and inner-world: a biolog- Tononi,G.,andKoch,C.(2008).Theneuralcorrelatesofconsciousness:anupdate. ical perspective on visual awareness,” in Proceedings of SPIE, Human Vision and Ann.N.Y.Acad.Sci.1124,239–261.doi:10.1196/annals.1440.004 Electronic Imaging XVIII, Vol. 8651 (Bellingham, WA). doi: 10.1117/12.2011874 Tononi, G., and Spoorns, O. (2003). Measuring information integration. BMC Koenderink, J. J., van Doorn, A., de Ridder, H., and Oomes, S. (2010). Visual rays Neurosci. 4:31. doi: 10.1186/1471-2202-4-31 are parallel. Perception 39, 1163–1171. doi: 10.1068/p6530 Toyabe,S.,Sagawa,T.,Ueda,M.,Muneyuki,E.,andSano,M.(2010).Experimental Laflaquiere, A., Terekhov, A. V., Gas, B., and O’Regan, K. O. (2013). demonstration of information-to-energy conversion and validation of the gen- “Learning an internal representation of the end-effector configuration eralized Jarzynski equality. Nat. Phys. 6, 988–992. doi: 10.1038/nphys1821 www.frontiersin.org June 2014 | Volume 5 | Article 577 | 21 Hoffman and Prakash Objects of consciousness Turing, A.M. (1937). On computable numbers, with an application to the Received: 22 January 2014; accepted: 23 May 2014; published online: 17 June 2014. Entscheidungs problem. Proc. Lond. Math. Soc. 2, 42, 230–265. doi: Citation: HoffmanDDandPrakashC(2014)Objectsofconsciousness.Front.Psychol. 10.1112/plms/s2-42.1.230 5:577. doi: 10.3389/fpsyg.2014.00577 Wilczek, F. (2006). Fantastic Realities: 49 Mind Journeys and a Trip to Stockholm. This article was submitted to Perception Science, a section of the journal Frontiers in Singapore: World Scientific. Psychology. Yuille, A., and Bülthoff, H. (1996). “Bayesian decision theory and psychophysics,” Copyright © 2014 Hoffman and Prakash. This is an open-access article dis- in Perception as Bayesian inference, eds D. Knill and W. Richards (New York, tributed under the terms of the Creative Commons Attribution License (CC BY). NY:CambridgeUniversityPress),123–162. The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publica- Conflict of Interest Statement: The authors declare that the research was con- tion in this journal is cited, in accordance with accepted academic practice. No ducted in the absence of any commercial or financial relationships that could be use, distribution or reproduction is permitted which does not comply with these construed as a potential conflict of interest. terms. Frontiers in Psychology | Perception Science June 2014 | Volume 5 | Article 577 | 22